The problem in my case was that I wanted to combine a pattern with my objects that I precomputed using an external program because it contained a lot of curved surfaces. As I already knew OpenSCAD basically becomes unusable if you try to use a png image as heightmap I decided to instead generate a .scad file containing an array with precomputed coordinates, and then used that to build and extrude 2d polygons.

This kind of works for the 3d preview, but the stl renderer was not happy regardless. It was basically the worst case for OpenSCAD. But still, the fact I could just auto-generate a text file and use it directly for building models is fantastic. With tools like FreeCAD I wouldn't even know where to start, here it was basically a case of "it just works" and with a few more minutes it can be combined with a bash script for batch rendering.