Gems are able to be found on other maps too,

extinction: has fragmented green gems, crystallized blue sap and crystallized red sap which can all be obtained from Gachas most efficiently

Valgeuro has a very large underground area named the emerald forest Which has many green gems on the ground level and very few blue gems on ledges close to the ceiling

Genesis has every type of gem in the ocean biome in the underwater structures that you can enter and walk around in