Lighting CGI without HDRIs for VFX is possible to a certain extent. But why do this? Well, sometimes, you might not have the luxury of taking 360 HDRIs of a physical scene/set. We can get around this with images of the light source to a reasonably accurate extent. I say ‘reasonably’ because although this method will cast the correct spectrum of light in the scene, it does not produce accurate shadows.
With that in mind, here’s how you can light a CGI scene without HDRIs.
General Overview
- Shoot on-set lighting plate
- Conform photos to ACES
- Create lighting planes/projection with images
- Render 🙂
The usual workflow for plating and setting up the layout of the 3D scene still applies. We won’t cover that here. This workflow is just to get an overview of bringing on-set color and lighting information to our 3D scene with a regular camera.
For this demonstration, we will be rendering a soldier man from turbosquid, onto a plate of our office space. The main light source would be the daylight from our window.
Shooting on-set lighting plates
Equipment for shooting lighting plates
Most modern day semi-professional mirrorless cameras generally come with their own LOG color profiles for video. We can use this for photos too! It’s a little unorthodox, but the nature of LOG allows us to capture a high dynamic range in a single photo. This means not having to set up a tripod, and do bracketing for HDR (more data, more work).
Why not shoot in RAW? To keep things short and simple, whilst RAW offers more latitude and flexibility in post, it also complicates the process of linearizing the image into an ACES workflow.
Shooting lighting plates in LOG helps us retain a considerable amount of dynamic range, whilst maintaining a standardized color science that is incredibly easy to unpack into ACES for use in a VFX pipeline. All you need is the right IDT (Input Device Transform).
Important note: When photographing data for video production, remember to match the main production camera’s settings as much as possible. Mainly exposure and white balance — Auto White Balance is a no-no.
Image of our main light source shot in S-Log3/S-Gamut.Cine
Reading these lighting plates in Nuke or Resolve is easy as we just have to select the correct IDT, which in our case is S-Log3/S-Gamut.Cine.
Here the same LOG image is converted into ACES, viewed in sRGB. You can see the amount of dynamic range we have when we feed this into a CG light source. Next we want match this to the color and exposure of the main footage. Once we’ve matched exposure and color, we can export this as an OpenEXR in ACEScg file for use in production.
Lighting in CGI
At this point, without diving into nuke to comp, this render is already very deliverable. Granted this is a relatively easy setup, the point is to demonstrate the viability of lighting a scene without an HDRI.
Testing our lighting plate.
Here you can see that lighting information is well maintained in the reflections.
If we shade the guy in an all metal shader, we can see the highlight retention better.
So again, why not RAW? Isn’t it better than LOG?
The actual issue is not that RAW is not a good format to use for capturing on-set lighting data. The problem is the post-production software available to process photo RAW files for VFX production.
Photography software like Adobe Camera RAW or Photoshop/Lightroom are great for developing RAW photo files for final display output/manipulation and print. However in our case, we want to translate the RAW data into a linearized color space for use in VFX/3D production. To my knowledge, there’s no easy way to do this without arbitrarily eyeballing everything.
So instead of eyeballing color science, we rely on camera manufacturers who develop consistent color science workflows for video production. Shooting image plates in LOG ensures we stay within this realm of consistency. The only downside is that these plates would be in 8-bit.
8-bit.. JPEGs… Isn’t that a bit-bad for production?
If there was an option to shoot lighting plates in 10-bit+ LOG images, we would. Unfortunately, our camera only shoots in either RAW or JPEG. The banding and binning that happens when we unpack our 8-bit LOG photo into ACES is something we have to live with. As long as we expose correctly when taking the LOG images, we should be alright.
Conclusion: It’s a hack at best.
Okay, so you can see that the examples here are very limited to lighting a CG character in an environment, on a plate that doesn’t require any contact shadowing. lol. And also, the character is also not highly reflective. You get it, some CG subjects are easier to work with than others.
At the end of the day, this is a hack at best because in order for a CG object to be lit correctly and match the plate, the lighting of the environment has to be accurate. And the most efficiently accurate lighting we can get is from a 360 spherical HDR image. This is why 360 HDRIs are crucial for VFX production.
But just in case you don’t have or can’t get one, this is a decent alternative for casting accurate light information into a 3D scene. And it works pretty well; if you don’t require accurate shadows or mirror-like reflections.
A hidden benefit to lighting with LOG images.
There’s an added benefit to understanding how LOG images can light a scene; The original footage plate can also be fed into a light source as an HDR image and provide accurate light information. This means we can project the footage onto a surface/light and use that as a source of light for our CG elements. This reiterates the point on why we don’t do VFX on graded footage.
To sum it all up, all these are tools available for us to light a CG scene to match, and it’s not so much one or the other, but a decisive mix of all depending on the situation. But yes, at the end of the day, we can light a CGI scene without an HDRI for VFX production. As long as we understand the value of maintaining exposure and color in our images.
