2020-2023
-Unreal Engine
-Virtual Production
-SteamVR
-Arduino
-Film Production
I Reverse-engineered the virtual production pipeline at the time wihen no tutorials by breaking down its core components and drawing from related fields (VR game development, A/V systems), and finally comeup with a test shot with my own Virtual Production Stage. This is the document about the process
1_Camera Tracking
When I think about Camera tracking, I immediatly thought about VR games. With a help of a VR game dev tutorial. I managed to use SteamVR and my HTC vive Tracker as Camera Tracking system. I have to write a custom blueprint to assign the tracking data to the camera. This is my first trial to programming with unreal engine blueprint.
2_nDisplay
I tried live compositing workflow using green screen. But then I decided to dive deep into ICVFX virtual production. I believe it is representing the future. After research, I found ou the correct way to line up everything. At the UE4's era, ndisplay editor is still a text based configurator, but I still managed to work out the corret way to do it.
3_The Lens Encoder
By the early time of Virtual Production, I can't find a consumer grade lens encoder product on the market. So in order to align the focal distance, I have to come up with my own Idea. This a arudino based project. I choose the QY1503-CDZ mini incremental encoder, with the resolusion of 256 step per rotation. A 0.8mod gear is attached to the input shaft to engage with the lens gear ring.
I used 3D printing to make it
For each lens, I made a table for alignment. The example is a Nikkor AIS 50mm F1.2 Lens. The out put is directly from Arduino IDE Serial Monitor
And then with the help of a Unreal engine plugin for 4.25 called UE4Duino, I was able to control the camera focus as a camera component.
4_The Stage and the Shot
For the first shooting test, I want to keep it as simple as possible, so I designed a one shot with no plot. I dont have the budget for renting a LED wall, so I come up with the idea of using a TV cover the window. And the following is the stage planning sketch.
With the help my friend, I built a "minimalistc virtual production stage" in a studio of School of Visual Art. The 'LED wall' is a TV I borrowed from my neibour.
