Baking lightmaps for Photoshop.

[Herpes Melodie]

Hi all, 

I was wondering how I should create light maps, in Maya, that can be used in Photoshop by using the blending modes. Like ambient occlusion maps, I want seperate light maps that can be used in conjunction with texture maps. 

Thanks in advance

[Chosen Few]

First, build your model, UV it, and light the scene.  After that, the specific steps to generate the light map will depend on what renderer you're using. 

If you're using Mental Ray, there are several methods you might employ, but this one is among the simplest:

1. In the rendering section of Maya, click Texture/Lighting -> Batch Backe (mental ray) -> Options Box.  The Mental Ray Backing Options dialog will open. 

2.  Apply the following settings:

• Objects to bake: Selected
• Bake to: Texture
• Bake shadows: Checked
• Keep original shading network: checked  (This is super important.  If you uncheck it, your existing shader(s) will end up replaced by the new baked one.  Needless to say, this could ruin your whole day.  Don't do that.)
• Use bake set override: checked
• Color mode: Only Light  (As you might have guessed, this is the most important part of the whole thing.  You don't want any of the diffuse color from the surfaces included, just the lighting.)
• Normal direction: Surface front
• Prefix: lightMap_  (Of course, you can give it whatever prefix name you want.  I recommend "lighMap_" simply because that will make the file easy to identify.  If something else makes more sense to you, that's fine.)
• X resolution: whatever size you want
• Y resolution: again, whatever size you want
• File format: TGA  (Technically, you can use whatever format you want, as long as your destination program can understand it.  I always recommend TGA, because it's the granddaddy of them all, and as such remains the most commonly used image format in the industry. Whatever you do, just don't choose a lossy format like JPG.)
• Bits per channel: 8 bits
• Number of samples: 1
• Bake to one map: Unchecked (This seeting is largely irrelevant if you're not using bake sets.  Still, for your described purposes, it's safest to leave it unchecked.)
• Bake alpha: Unchecked
• UV range: Normal (0 to 1)
• Fill texture seams: 3  (This is the bleed area around your UV shells.  You can use whatever number you want.  Just make sure it's enough to avoid jagged edges along your seems, but not enough to cause any shell's lighting to bleed into any other)
• Override mesh UV assignments: Unchecked   (unless you're using more than one UV set, in which case you'll want to use the name of whichever UV set you want the lightmap to follow)

3.  When everything is set properly, click Convert.  You'll find the newly created baked image under /renderData/mentalray/lightMap inside your project directory.

Note, if you want further explanation of what all the options do, click Help -> Help on Mental Ray Baking Options, at the top of the Mental Ray Baking Optons dialog.  Maya's options box help is absolutely fantastic.  It's so easy to learn what all the options do in every single options box in the whole program, and because the help is always right there, you don't even have to expend any conscious effort to memorize them!  Use the help as long as you need, until you the day comes that just naturally discover you've got it.

 

If you're using Turtle, it's super easy.  Just go into Turtle's output settings from within the Render Settings dialog, check Illumination, and uncheck everything else. From there, simply render the frame, and the illumination map will render in the standard render window.  The file will be located in /turtle/bakedTextures inside your project directory.

 

Bring the lightmap into Photoshop, and use whatever blending method you want, to apply it over the diffuse texture.  If you want the lighting to be additive, without the shadows being subractive, use Lighten or Screen.  If you want the shadows to be subtractive, without the lighting being additive, use Darken or Multiply.  If you want both, use any of the options on the overlay section (Overlay, Hard Light, Soft Light, etc.).  Use layer opacity to control the intensity of the effect.

[Herpes Melodie]

 How does that work exactly?

The overlay blending mode usually (over) saturates, when using pure white, on textures. It highly depends on the actual texture itself.

Screen adds white values, but doesn't brighten/lighten the actual color itself. Same goes for mulitply and hardlight. Hardlight is basically a combination of screen and multiply.

To see what I mean, try using a brightness adjustment layer on an image and play with the settings. If you darken/lighten it, it gets darker/lighter in a different way than how it would be if you'd use white on screen, or black on multiply.

When you bake out a texture map with light effect baked in, you see this effect as well, just like how you would brighten an image like I described above with a brightness adjustment layer, and I want to reproduce this effect in Photoshop.

 

I could be wrong though, but it just doesn't look right when I play with the blending modes on lightmaps in Photoshop.

 

 

[Azriel Drumheller]

Multiplying the occlusion layer over your color layer should not lighten it as all the white areas will appear transparent. You want to use multiply. If it comes out too dark just adjust the fill/opacity until it suits you.

 

-Az

[Chosen Few]

---

Herpes Melodie wrote:

 How does that work exactly?

---

The blending mode to use will depend on several factors.  As you know, the white areas of the lightmap represent the parts of the surface that are to be fully lit, while the black areas represent the parts of the surface that are to receive no light at all.  What you may not have considered yet is that the definitions of "fully lit" and "not lit at all" are not fixed.  They will vary in accordance with what exactly you did to create your pre-lightmap texture, and what kinds of effects you're trying to add to it.

Very commonly, "fully lit" will mean a display of unimpeded diffuse color, and "not lit at all" will mean a display of unimpeded ambient color.  With that in mind, if the diffuse coloring in your texture is already at the maximum brightness you want it to be, you won't want to let the lightmap make it any brighter. In that case, your lightmap is effectively a shadow map, and you'll want to use one of the darkening blend modes, such as Multiply.   Conversely, if you created your unlit texture with only ambient color values, in anticipation of the fact that you'll be using a lightmap to brighten it up to full diffuse color in places, then you'll want to use one of the lightening blend modes, like Screen.

Another common way to think of "fully lit" is that the colors get supersaturated (pure white), as if under a super high intensity spotlight.  If that's what you're going for, then you might want to use diffuse colors that are already bright in the unlit texture, and use Screen to blast them up to near whiteness, or even full whiteness, in the most lit areas.

Along those same lines, if "not lit at all" means complete darkness (pure black), then you might want to use Multiply, even if the unlit texture is already quite dim.

If you're going for a combination of those last two, then you'll probably want to keep the unlit texture values in the medium brightness range (around 50%), and set the lightmap layer to Hard Light.  The white areas of the lightmap will then super-brighten (whiten) the colors, while the black areas will super-darken (blacken) them.

There are all kinds of possibilities.  The above are just the tip of the iceberg, of course.  Hopefully, they'll provide enough of a push-start to get your mental gears turning in the right direction.

There's a very good explanation of all the various blend modes, along with some suggestions of when to use what, at http://www.cgtextures.com/content.php?action=tutorial&name=blendmodes.  I'd recommend reading through it.

 

---

Herpes Melodie wrote:

The overlay blending mode usually (over) saturates, when using pure white, on textures.

---

White on Overlay doubles the values of the colors below. If you start with 50% red (128, 0, 0) you'll end up with pure red (255, 0, 0).  If you start with 25% purple (64, 0, 64),  you'll get 50% purple (128, 0, 128). 

This is the equivalent of shining a 2x white light in Maya onto any surface that has a default (black) ambient color, with any diffuse color you like.  It's also equivalent to shining a 1x white light onto a surface whose ambient and diffuse colors are equivalent to each other.  (Of course, there are plenty of other combinations of light and ambiance, not to mention other material properties, that will also add up to two times the diffuse color.)

So, whenever you apply a lightmap in Overlay mode, the question you should be asking yourself is, "Do I really want to double the amount of light that falls on the white-mapped areas?"  If the answer is yes, then Overlay is the mode to use.  If not, use something else, or lower the overlay's opacity.

 

---

Herpes Melodie wrote:

 It highly depends on the actual texture itself.

---

Yes, absolutely.  The effect that any lightmap under any blending mode will have will depend directly upon what's going on in the unlit texture, every bit as much as it will on what's happening with the lightmap itself.  The end result is a combination of all elements in the image, always.

With that in mind, you may want to start to rethink the way you create your unlit textures.  Bright diffuse color is not always the best strategy.  Sometimes it's better to tone things down in the unlit texture, in anticipation of what the lighting effects will add.

Also, don't forget you're not limited to just one lightmap layer.  You can apply one in a lightening mode, and a second in a darkening mode, and then independently adjust the opacity of each.  And it doesn't have to stop there.  You can combine dozens of copies of the lightmap in various ways, to acheive the exact effect you want.  You can also render multiple lightmaps, under different lighting schemes in your 3D scene, and composite them all in Photoshop any way you want.  The possibilities are endless.

 

---

Herpes Melodie wrote:

 

Screen adds white values, but doesn't brighten/lighten the actual color itself.

---

At full opacity, pure white in a screen will always yield pure white as a result, no matter what's below it.  Lower opacities limit the maximum whiteness to shades of gray.  So, if you don't want to super-saturate the whites, keep the opacity well below 100%.

 

---

Herpes Melodie wrote:

Same goes for mulitply

---

Black in a multiply will always yield black as a result, since anything times zero is zero.  So, the answer here is the same as with the screen.  If you don't want pure black from the black areas of the lightmap, keep the opacity down.

 

---

Herpes Melodie wrote:

Hardlight is basically a combination of screen and multiply.

---

It's often described that way, but it's not quite the same.  I don't pretend to understand all the mathematics, but there are some observable differences.  It's pretty close, though.

For whatever it's worth, there are lots of places online that purport to explain the mathematics of the various blending modes, but none that I've ever seen actually jive with the real observable results.  Many give the same math to Overlay as to Hard Light, for example, but we can clearly see that they're not the same.  Most also say that Overlay has the same mathematics as Screen, whenever the values are above 128.  But again, the observable results just don't seem to support this.

In any case, less than full opacity will often yield the best results.

 

---

Herpes Melodie wrote:

 

To see what I mean, try using a brightness adjustment layer on an image and play with the settings. If you darken/lighten it, it gets darker/lighter in a different way than how it would be if you'd use white on screen, or black on multiply.

---

Absolutely true.  And the good news is you can apply your lightmap to adjustment layers, too.  You'll find that a lightmap on a levels adjustment layer, for example, can do wonders.

 

---

Herpes Melodie wrote:

When you bake out a texture map with light effect baked in, you see this effect as well, just like how you would brighten an image like I described above with a brightness adjustment layer, and I want to reproduce this effect in Photoshop.

---

 

I've got three answers on this one.

1.  If all you want to do is produce the same effect you'd get with a full shading bake, you might as well just do the bake.  The point of rendering separate maps is to be able to produce variants in Photoshop that you wouldn't necessarily be able to produce very easily or time-effectively with just the bake.

2.  If what you're concerned about is brightness/contrast in particular, then as I said above, you can certainly apply your lightmap as a mask on a brightness/contrast adjustment layer (or several), and produce the effect you've described.

3.  As I said toward the beginning, you may want to rethink your base texture strategy.  If what you're doing is painting textures the same way you would if lightmaps weren't to be part of the equation, then you're going to be barking up the wrong tree in many cases.  Often the best strategy is to paint the texture far more plainly, and let the various render maps (light, AO, bump, etc.) do the heavy lifting for you.

 

---

Herpes Melodie wrote:

I could be wrong though, but it just doesn't look right when I play with the blending modes on lightmaps in Photoshop.

---

Keep at it.  Whatever you've done so far, there are millions more options.

 

 

[Herpes Melodie]

Thanks both.

The reason I asked about it, was because I want to sell stuff with light maps included. Full shading bakes don't work for this particular scenerio. The trick you mentoned Chosen (applying a lightmap in an adjustment layer) might do the job as well, haven't thougt about this one. I'll check it out.

 

Again, thank you all.