You could pack the raytracer and the scene into a GLSL fragment shader without any trouble whatsoever and achieve something greater than 30fps out of the gate.

Check out https://www.shadertoy.com/ - all of this crazy stuff is generated using two triangles and a fancy shader. Almost every 3D scene there is generated using some form of ray casting or ray tracing on the GPU.