Posted on March 6, 2014 by

Normal Map Experiment

Interactive Re-Lighting of a photograph

A few months ago, I was brainstorming visual ideas to accompany a live music performance and I ended up creating this little demo. The techniques used aren’t original, but I think the combination of them are interesting. First off, check out the experiment, move your mouse to move the light source:

This effect is achieved using two images. The first is a fully lit black and white image, the second is a normal map:

exampleImages

If you aren’t familiar with normal maps you can find a ton of information online. I first learned about them from this Video Copilot tutorial, which does a good job explaining how they work to relight a scene. Put simply, the red and green color channels represent the direction the image’s surface is facing. When using a normal map to relight an image a little math is performed that determines if the surface is facing towards a light source. If so it will receive light, the amount of which is determined by the distance to the light source and by how precisely the surface is facing the light. The technique is called phong shading, the code for this experiment is almost 100% copy + pasted from this fabulous post by Jonas Wagner.

Normally a normal map is rendered by a 3d program, but I thought I might be able to approximate one by shooting multiple photos of a subject with light coming from above, below, left and right. As it turns out, I was not the first to think of doing this and I found this article by Ryan Clark which helped out a lot. I had my wife sit as still as possible while I ran around with a flash light and took these:

exampleImages

The “full light” image used in the demo above is a composite of the separate images, and I followed Ryan Clarks instructions to create the normal map. Since it took a few minutes to shoot the photos they don’t line up perfectly, but good enough for a proof of concept.

Since creating this experiment I’ve been working on ways to build upon it. I’ve already built an automated rig for taking the photos quickly which I think will result in a more convincing effect. I’m also planning to rebuild the demo in WebGL which I hope to explore in another post.