During the third semester of my studies I worked on the vfx short "OBOLUS" with a big team. I decided to do our previz in the Unreal Engine, but i wanted to take it one step further. We have the NCAM camera tracking system at our university, and a friend and I decided to write a C++ plugin for the Unreal Engine to visualize our previz in realtime on the shooting day.

The plugin uses the raw 4k camera footage and combines it with a realtime tracked camera in the 3D scene - everything else is also done in the engine: greenscreen keying, basic compositing features, real camera depth of field (the focus can be extracted with NCAM). The rendered image is then given out to the camera operator without any visible lag (a few milliseconds).

I presented our Unreal Engine plugin at FMX 2017.

During the production of OBOLUS i also did a lot of texturing in substance painter. You can watch the "Making Of" here: