Idiomatic will probably entail a lot of objects for the pixel data, I'm assuming? I've seen some raytracers written with arrays of simple "bags" of numbers, which would increase performance.

I wrote a real-time 3D polygonal rasterizer in JS that supports texture mapping. After switching from regular arrays to Float32array to store coordinates, it got a huge (~50%) jump in performance.