My Pipeline

Finally here.
This is the primary workflow for my projects. It's pretty much second-nature to follow it at this point, and I tend to not backtrack (e.g. from lighting back to modeling) because of the time it wastes. This process has worked pretty efficiently for me over the past two years, and is actually a refinement of what I was doing in college. It's a bit of an amalgamation of production pipelines used at Sony Pictures Imageworks, Pixar and by a college professor of mine, plus bits and pieces of what works best for me.

Thesis Texturing

One of the important parts of my thesis project will be texture mapping. I want things to age over the course of my film, and I plan to use a combination of photographs and procedural shading. I have compiled a list of the rooms that will be in my project, as well as the textures I will need for them according to the decade in which they appear. Additionally, I've included columns for audio, mood and other components relative to each shot. It's a way for me to organize my ideas and also "pre-storyboard," in a way.
I've used this spreadsheet to help myself figure out which textures I need to photograph and in turn plan where I should take them. For example, I knew that I would need plenty of brick and concrete to texture map the exterior shots of the apartment building, so I headed to downtown Providence, RI and also downtown Los Angeles because I knew there were plenty of industrial buildings. I was shooting with a digital SLR that had RAW format in 8 megapixels, so I knew I would have plenty of resolution to work with when it came time to creating my texture maps for Maya.

I tried to get a variation of surfaces, ranging from pristine to worn down, to help me indicate the passage of time. Finding dirty textures was actually somewhat fun, and it was also nice to know that I would have to do much less work creating grunge and imperfections with shaders in Maya.

I tried to photograph textures as orthographically as possible, and get as far away as possible so that I would not have to repeat them when I applied them to geometry. Additionally, I tried to shoot them so that the lighting was consistent across them and would therefore give me the ability to use them at all times of day and in any lighting conditions.

I'm organizing my textures by type rather than by scene so that I can easily locate materials I need. This will help to streamline my texturing process in that I will probably end up with dozens, if not hundreds, of materials that I will have to manage.

Although most of my textures were shot outdoors, I do plan on using many of them, especially the plaster and wood ones, in my interiors. It's just a matter of simple color correction, and chances are that I will even just use some as bump maps.

Concrete


Brick

PlasterWood
Misc.
I also photographed some objects so that I could camera project textures in Maya, and create 2D camera moves from static 2D images. My Visual Effects Society Mentor, Van Ling, gave me some great advice on how to incorporate camera mapping into my project. For one thing, it would help to save time in lighting in rendering. Secondly, it would be great for using as backgrounds so that the CG objects would not look so CG when seen in the same shot.

Potential Camera Projections

Ambient Occlusion

The way I've been rendering ambient occlusion is mapping an Mib_amb_occlusion node to the Color of a Surface shader, assigning it to all of my geometry and using a white background for the camera. I've been outputting 32-bit EXR files for maximum control over the contrast and exposure of the rendered image, which is then multiplied on top of the beauty pass.

One of the initial problems I was encountering was that given the default settings of the occlusion shader, I was getting renders that were completely black except for the window. Additionally, the renders would still take about a minute to calculate even at 400x225. I figured that it had something to do with the fact that my scene was nearly completely enclosed. After talking to a graduate of my program, Ben Hendricks, it turned out that this was partly the case, but he said that understanding how the shader was working would allow me to control it.

The camera shoots out rays at the geometry that then bounce off. Samples determine how many rays are shot out from the point on the geometry hit by the camera.
If they hit another surface, they return black. If they don't hit another surface, they return white. Since multiple rays are being shot out, the resulting pixels can be anything between black and white. The Max Distance determines how far each ray will radiate (i.e. look) from the geometry for another surface. A value of 0.000 is actually infinity; this explained why my renders for my interior space were turning out black. The window did provide an opening to infinity (which would give a white pixel), but since the return values for the rays were being averaged, the resulting pixels were almost completely black.

Ben said that to determine what setting I should use for Max Distance, I should use a value of about 1/3 the longest distance in the scene. I used the Distance Tool in Maya to figure this out:

I got a value of about 21, so I used a Max Distance of 6 (hey I know it's not 1/3 of 21, I can count) and rendered out my occlusion passes successfully. Additionally, my render time was drastically reduced to about 1 1/2 minutes for a 1920x1080 frame. Because I was exporting 32-bit EXRs, it was easy to crush values and adjust the contrast without degrading the quality of the image.

Ambient Occlusion (Wikipedia)

Renderman vs. Mental Ray Tesselation

One of my personal attractions to RmF is its subpixel tesselation of subdivision and NURBS surfaces. It renders both out extremely quickly, and there are never any jagged or sharp facets.

I have had mixed results with the tesselation of these surfaces with Mental Ray, both in terms of quality and time, but I was very pleased with the renders I received from my bathroom model. After rendering out my first beauty pass of the night scene with RmF, it was quickly evident that I needed to use MR to create certain passes, such as ambient occlusion and possibly light fog. When I rendered out my ambient occlusion with MR, I was lucky that it actually only took about a minute to render, and matched up perfectly with my RfM beauty pass - even at 1920x1080 resolution.

I'm currently running Maya 2008 and know that MR has had good improvements in quality and efficiency since even Maya 8.5. I guess the reason for this post is that I just want to confirm it is indeed possible to seamlessly composite passes from both RmF and MR, without adjusting parameters for either renderer.

Overcast Lighting - Compositing

I only waited about 8 minutes for this render, which was 1920x1080:












The results were pretty satisfying, but I knew that they were just a jumping point for the final image. The parts that were working for me were the very even lighting and wide specularity of the objects, including those that had originally exhibited sharp spec highlights in the moonlight scene. Also, the light had a beautifully smooth falloff from the window to the foreground. This falloff worked very well with the variant specularity of the walls and floor especially, with blurry "blobs" that created the illusion of slightly dirty surfaces.

I rendered out a quick ambient occlusion pass to help carve out crevices and add some depth to the scene (even though the diffuse lighting was meant to flatten out the lighting quite a bit):
In Nuke, I adjusted its histogram to reduce the contrast and also redistribute the occlusion to be less spread out:

I color-corrected the beauty pass and comped it in Nuke with its ambient occlusion pass and depth pass, as well as with an image for the background behind the window:







One of my favorite things about the image was that it was lit by a single light and did not carry the CG quality of looking too consistent and clean. However, I felt that it looked too flat because of its lack of specularity, and I wanted to bring back the shininess of specific surfaces, such as the sink and pipes, bathtub, shower rod, mirror and radiator.

I knew that if I were to be rendering out an image sequence, I would prefer to create a specular pass instead of adding lights to the scene and facing a possibly drastically higher render time. From my experience with rendering, I've found that doing so would probably have more of an exponential rather than additive increase in render time. I created a very simple Blinn with the following settings, including a Solid Fractal mapped to its Specular Color node:





















I was satisfied with the lighting of my moonlight scene and knew that I had lit the scene to accent the spec highlights of the scene. So, I ended up re-using the exact same light rig, knowing that I could adjust the render's color and exposure in the compositing stage. I used Mental Ray to render the scene with this light setup and the Blinn mapped to the necessary objects. All of the objects whose specularity I wanted to remain untouched in the final composite were assigned a black surface shader. (In this screenshot, I've made the surface shader green to help distinguish the specular-shaded objects from what did not have spec attached):
The rendered spec pass (of course with the surface shader turned to black again):
This pass took about 5 minutes to render, versus what I predicted to be at least 15 minute render if I had rendered out the photon-mapped scene with this light rig - not to mention some tedious adjustment of the light rig in order to get it to play well with the photon mapping.

I also adjusted the hue of the spec pass to make it less blue and a bit more yellow, and screened it on top of the beauty pass.





















My final composited image:
I was very happy with the outcome of the final image. Using a combination of photon mapping, direct lighting and basic compositing, I felt that I was able to effectively immitate the overcast light of my reference photos. Lighting was nicely gradated but not flat to the point of looking perfectly CG, the green wall color radiated off of the bathtub, and the specular pass really helped to sell the believability of the scene. I'll definitely be using this approach in my thesis project, and I know that with better shaders, I'll be able to get even better results.

Overcast Lighting - Photon Mapping

Given the massive failure of my testing with Incandescence and Colorbleeding in RmF, I knew that I should pursure other options. I talked to Eric and he told me to use Slim (the actual RenderMan interface for shading and lighting) for photon mapping. I tried using it but due to software issues (and software issues alone - it was not in anyone's control), I was unable to get very far. I knew that I had to switch over to Mental Ray to get the job done. Luckily, Eric had taught us how to control photon mapping in our lighting class last semester. Here I'm regurgitating what I have in my notes, so if anything is wrong or unclear, it's all me.

I used just one single light in the entire scene. I created an Area Light with an Intensity of 0.000 and under the Mental Ray tab, checked on Emit Photons:





















I gave the Photon Color a light, desaturated blue hue to match the color of a cloudy afternoon. Leaving everything else at default values, and using the render preset Preview: Global Illumination in my Mental Ray render globals, I ran a test render at 400x225:
The light looked as if it had been placed directly next to the window instead of far away, out in the sky. I felt that things were too bright near the window and did not look like the even diffuse light I had observed in my reference photos. Photon Intensity determines how hard the light is "thrown" from the Area Light, so I decreased it to 4000 because the default value of 8000 was causing the lighting to be too intense where the source was. I ran another test render and got more satisfactory results:
The lighting was darker overall, but the falloff was not as drastic coming from the window, which I liked. Just as a test, I decreased the Exponent from its default at 2.0 to 1.5 in order to increase the "intensity" of the "light" (I know this gets contradictory quickly.) Decreasing the Exponent allows the "light" to travel farther from the source; my goal was the brighten up the foreground. I got this render:

I felt that the foreground had been brightened nicely but not enough, and that the area surrounding the window was starting to get a bit blown out. I figured that between values of 2.0 and 1.5, there would definitely not be a happy medium to give me a brighter foreground and a background that was not too hot. I knew that I would ultimately be rendering out an EXR, and in turn I would be able to color correct and brighten things easily without degrading the image quality, so I decided to return the Exponent's value to 2.0.

The splotchiness on the walls and ceiling was partly due to the shader (which had a fractal mapped to its spec color), but I also knew that some of it was due to the photon mapping. In the render globals, Caustics and Global Illumination > Global Illumination Options, I increased the Radius from its default 0.000 to 2.000:





















Increasing the radius serves to help smooth out the splotchiness and other rendering artifacts. The result was very similar to the one before it (the second render in this post):

The only discernable differences were that I could see a bit more occlusion in the corners of the walls, and things looked a little bit darker. However, some of the blobs had been smoothed away. Knowing I was going to be rendering at full 1080, I felt that it was worth keeping the Radius at 2.

The important thing to know about adjusting the Radius is that it can smooth things out at the expense of accuracy. Increasing the Accuracy can help fix light leaking (i.e. where lights seems to be passing through objects) and also help get rid of splotchiness. However, I chose to leave it at 500 because I was fairly satisfied with the results (the only leak I noticed was where the shower wall facing us joins the ceiling ), and I knew I was going to be rendering an ambient occlusion pass to help darken crevices and hide any leaks.

The most recent render image in this post ended up satisfying what I had been looking for in a beauty pass, so I rendered out an EXR at 1920x1080.

Overcast Lighting - Render Testing

After using the direct lighting strategy with the moonlight scene, I wanted to use a different strategy in order to create a very diffuse type of lighting. I knew that photon mapping in Mental Ray would probably be my best option, but I wanted to test the waters using a tactic in RfM in which I would actually light with a shader instead of an area light.

In my workflow, I tend to run tests on scenes that have very limited and very simple geometry. I do this to save time and in turn leave plenty of room for messing around with parameters to see if an idea is even worth trying on more crowded scenes with more complex geometry. I can also render at a higher test resolution (800x450 instead of 200x112) so that I can get a better estimation of the quality of antialiasing as well as splotches that sometimes show up when lighting.

My setup was what you see here: just a sphere, cone, torus, plane and a surrounding cube. Everything except for the plane was subdivided and had the default Lambert shader.













The plane had a Blinn with all of its defaults, except for its Color had been turned completely black and its Incandescence had been turned red...






















...and given a Value of 1.000:






















I then tried rendering out a scene with raytracing turned on, but the only thing that came out visible was the red plane, and nothing else was illuminated. I then figured that I should try using some kind of Global Illumination, as with HDRIs, in order to get the plane to light the other objects. The only GI in RmF with which I was familiar was the Environment Light, so I created one...






















and set its Shadowing to Colorbleeding:























Turning on Colorbleeding in RmF is one way of turning on radiosity, or color play between objects. In my Render Globals, I originally started with a Shading Rate of 5.0 as a test, but I was getting chunky results despite having any bitmaps attached to any of my shaders. I was also only rendering test images at 400x225, so I had a feeling that my test was failing pretty quickly. Still, I then turned my shading rate down to 1.0, and gave it another go. I was only somewhat impressed by the results - and mostly because the antialiasing improved slightly - but I was still looking at a render time of about 2 1/2 minutes for an image that was tiny in comparison to the 1080p I would ultimately be outputting. There was certainly some radiosity going on, but not bright enough to illuminate the scene efficiently or to satisfaction.

Out of stubbornness and nothing better to do, I rendered out a full 1920x1080 image with the incandescence's Value still at 1.0:












The lighting was poor, there was lower-than-expected contrast because dark spots weren't completely black (they looked as if their gamma had been adjusted to a distracting gray), and I was getting a rendering artifact behind the plane (see the white spots.) Also, this render took about 10 minutes for my computer to calculate, and there was really nothing intricate about the scene.

A closer crop shows some splotchiness on the floor, and also gives a better sense of the antialising of each of the three objects in the foreground:










I decided to kick out another render, but this time adjusting the Value of the incandescence to 10.000:























The result was equally an improvement as it was a disappointment:













The illumination drastically shot up and reached farther than before, but I was getting awful rendering artifacts (gummy spots were showing up all over the walls and floor), and it looked like chunks of dirt had been caked on every surface.

Needless to say, the antialiasing was horrible:








Probably the worst thing about the render was that it took a solid 40 minutes to come out. I didn't even want to know how long this simple lighting setup would take in my bathroom scene. The results were terrible, and things were taking forever. I knew at this point I had to try something else.

Overcast Lighting Study

Since most of my experience has come from lighting exteriors and with hard lighting, I wanted to give myself the challenge of lighting a scene with diffuse illumination. I first looked at reference photos to study the objects cast shadows and received bounced light. I started with a photo I had been using as reference for modeling a scene for my thesis project:
















I'd originally taken it nearly four years ago only as a snapshot for texture references , and not for anything artistic or compositionally driven. In this way, the lighting was completely natural and was pretty much as close as one could get to looking at things as they are and just capturing that site.

I observed the image more and also tried to make mental notes ever time I saw diffuse light in real life. I tried to pick out what gave such lighting its qualities: color, intensity, spread, its shadows, the bounced light, and its interaction with surfaces of various mattes and glosses. In this image, the window was a translucent glass that diffused the sunlight and spread it evenly around. I noticed how the specular highlights on surfaces such as the linoleum table top and floor were very wide, and how the dirt and scuffs on the floor looked very flat. The shadows were all very soft, and rather than lighting helping to outline the profile of objects, color was more responsible.

I also looked at a stock photo of a rather pristine environment (as opposed to the worn and grungy one I'd photographed) from the internet as a way to help myself better understand diffuse light:












Although the objects were all very clean and polished, I still saw many of the same properties as in the other image. Shadows were very soft; although the chair and counter's shadows are clearly being cast, they are all very transparent and are not bold and sharp. The lighting is still very even across all surfaces; the refrigerator door to the left and the lower-right floor are both good examples. Considering that this was a professional photograph, it still displayed qualities that my simple snapshot did.

I took my observations into consideration as I tried to figure out what my 3D lighting and 2D compositing approach would be.

If it smells like fish..

Back in college, my computer animation professor told our class one day that if it looks like CG, it's bad CG. I'm almost positive it was on a day we were discussing lighting. She then told us about how a CG ball that had been comped into a shot in the movie "Toys" was blurred and covered in noise, among other things, to merge it into the scene.

Last semester, my professor Eric Hanson taught our class about creating organically lit and shaded renders. He showed us photographs and CG renders that all involved light that had been broken up and was unevenly cast across the objects in view. In my last post I noted that among the quick approaches he gave us in order to achieve this was to map fractals to our lights, as well as map fractals to the specularity of our shaders. I have since relied on these methods and have been very satisfied with the quickly calculated and more organic renders.

As much as these strategies helped to create the sense of realism with lighting and rendering, I noticed that when I could employ them on geometry that was not beveled or smoothed correctly, or perfectly symmetrically angled geometry, there was no way of hiding these obviously CG characteristics.

Here is a scene I shaded, lit and comped (but did not model) last semester:












Notice the very sharp edges along the base, as well as where the side meets the front of the cab:



















I tried to hide the sharp edges with the shaders, lighting and comping, but it's something that still bothers me to look at. Maybe it's just my personal preference, but I do think that smoother edges help to sell the image so much better, especially when it comes to the way light hits them.

When it came to modeling the bathroom scene, I decided early on that I would avoid creating sharp edges so that I would not have to spend extra time hiding them with the lights, shaders and 2D compositing. Additionally, I knew that I did not want geometry that looked too "perfect."

My solution was to use another one of Eric’s methods that he had taught us: pushing vertices around so that edges were not so straight and angles did not line up so well at 90 degrees. Using the poly-to-subD strategy, I would create the basic geometry with quads, and then make crooked angles, asymmetrical shapes and slightly offset faces before performing Edge Ring cuts and subdividing.

Here's a closeup from the night shot:













I offset the individual uprights of the radiator so that they were at slightly varying heights and distances from one another, and also rotated them at different angles. The window sill was also modeled to be bowed downwards and towards us in the middle. I was able to light all of these objects with just a single spotlight. By randomizing the scene in the modeling stage, I was able to avoid doing any extra lighting, shading or comping to hide the seams.

Lighting Rig - Moonlight

As I mentioned before, I used a direct lighting rig for my moonlight scenario. I used a combination of spot and point lights with varying intensities and white or blue hues. I began with the spot lights to create the main "moon" light at first, and then carve out the important geometry with rim lighting. After I was satisfied with the main shapes I was seeing, filled in the dark (sometimes virtually black) areas with point lights.

The render camera's view of the scene from within Maya.













Wireframe of the same view.













Only one single light (a spot) had a shadow attached to it. Most of the other spot lights were linked to specific geometry so that shadowing was not so much an issue. Most of them also had Penumbra angles of no more than 10 to soften their edges a bit.

Spot lights are highlighted in light green.













Additionally, one of the most effective (and also efficient) ways of creating more organic light play was to map Solid Fractals to their Color channels.





















The point lights all had their Decay Rate set to Cubic, so there was a finite range that each one would affect. One of the nice conveniences that comes with using a cubic decay is that you do not have to spend time linking lights to geometry in order to limit what they are hitting. Additionally, you do not have to create shadows because the unlit areas can simply serve this purpose. These lights are perfect for creating soft lighting, and are a great cheap replacement for raytracing. Of course, you need to spend more time lighting and less time rendering, whereas raytracing requires the opposite. I do enjoy lighting with point arrays because they allow more art direction and more control over the rendered image, but I know that one of the limitations is their lack of control to make the lighting dynamic.

Point lights are highlighted in light green.













All of the lights were either white or had a blue hue to mimic moonlight and its subsequent bounced light. Naturally, the white lights were aimed at the silhouette of objects, whereas the light blue were placed as fills, and the dark blues served to break up completely black shadows. I found myself using the spot lights more for carving out the profiles of objects, and saving the point lights for softening shadows.

Although I was duplicating lights so that I would not have to keep setting colors and intensities, I was sure to vary both of these parameters slightly, as well as to rotate and offset the lights' positions to keep things as organic and real-worldly as possible. One of the common mistakes that I saw myself making when I first started using point arrays was that I was spacing them too evenly and coloring them too consistently; the result was very flat, CG-looking scenes.

Here are some other views to give a better sense of how lights were offset from one another. The main "moon" light is highlighted in green.