NeRFs in VR

A couple months ago, I started a project where I wanted to visualize a NeRF in VR (VR Art Museum). In that project, I converted the NeRF into a mesh and used Unity to view it in VR. In the recent months, many of the NeRF frameworks have made it possible to view NeRFs in VR. Here are two different approaches.

NVIDIA’s instant-ngp just recently dropped the ability to stereoscopically real-time render a trained NeRF. On top of that, the rendering is foveated. The video on the left is NVIDIA’s. They also recently made the building of instant-ngp even easier. It runes as a .exe now!

The video on the right is a different approach to NeRFs in VR. I used NeRF Studio to make an equirectangular video rendering (360 field of view) and displayed it in my VR headset. The downside with this approach is that the camera trajectory is on a predetermined path. However, all 360 degrees are viewable per frame. You can see me do a 180 in this video at the end.