Strange white line in Ambient Occlusion

[Rolig Loon]

I've seen this same thing several times in the past and have always written it off as just one of those things that happen with the edges of things like stair steps.  It's starting to annoy me, though, and I am sure there must be a way to solve it.  I don't see white lines like these on other people's AO textures.

I have tried changing the sampling method and size, varying the distance, and can't find anything that affects it.  Ideas?

EDIT:    Oooo!  I can make it go away if I use flat shading on those surfaces instead of smooth shading when I make the AO.  But is that the best solution?

Edited February 21, 2018 by Rolig Loon

[OptimoMaximo]

That is the correct way, Rolig. Ambient Occlusion calculations rely on surface AND vertex normals to determine how exposed a surface is and therefore the chance to be lit by the environment's diffuse light. Smooth shading is meant to fake roundness, so over there it's trying to average the shading as if those pieces were cylinders

Edited February 22, 2018 by OptimoMaximo

[Rolig Loon]

Thanks, Optimo. I wasn't sure whether there was a better solution, once I stumbled onto that one. (I don't know why I didn't think of doing this ages ago.)  I was hoping to retain the gentler appearance of smooth shading on this edge, which is the nose of a curved step.  If that's not possible, it's best to avoid the white line. 

[OptimoMaximo]

You're welcome.

You shouldn't feel limited by the model you consider as the final SL model, though. If you still need a gentler look on the shading, you can perform your bake on a copy of your model, where you add another subdivision near the edge you would like to smooth shade. Then, don't do a smooth shade on the faces, do it on the edges. The way FBX (inherited from Collada, its parent) encodes things is by vertex normals. Doing the smoothing by selecting the edges ensures they are being averaged using a common tangent (along the multiple edge's selection components' direction) delivering the bevel effect on the shading. Basically, you're baking on a higher polygon model made from and made to work on your game asset, it doesn't really need to be sculpted to the billion polygons level  

Edited February 22, 2018 by OptimoMaximo

[Rolig Loon]

That's a good idea. My high LOD model doesn't have those extra edge loops, as you can see, because I was trying to keep the vert count low.  It didn't occur to me, though, to put the edge loops there just for the purpose of baking the AO. 

[OptimoMaximo]

30 minutes ago, Rolig Loon said:

It didn't occur to me, though, to put the edge loops there just for the purpose of baking the AO. 

Well you may want to duplicate the pieces and do an actual bevel on those corners to extract a normal map instead, that other method is just quick and dirty shading management, since i assume you weren't going for map extraction in the first place.

However, if i recall correctly, in Blender you have to add your edgeloops making sure that the "Correct UV" checkbox is ticked in the operator panel (bottom of the Tools panel "T") in order to get a new UV correct placement, otherwise your texture would stretch.

[Rolig Loon]

That gives me more to think about for future projects. This one is pretty simple, a set of four benches around a fountain on my own property, so the hard edges on the seat aren't objectionable. Next time I need to do something that calls for a softer edge, though, I'll give that a try.

Here's a quick snapshot of the way this one (or a quarter or it) looks now.

[OptimoMaximo]

Here's something else to remember for the next project then:

Usually, that is an issue due to non-uniform scaling, occurring when the object scale was manipulated, not frozen and then unwrap UV was called. The unwrapping script takes the scale into account and uses the original 1,1,1 scale to size the UV islands. You should always make sure that scale is properly applied before unwrapping (CTRL+A -> Apply Scale). I guess that is what happened also on the  bricks section.

My advice is to create a material, tile your base texture in there, check the stretches, address them where possible and bake the texture look resulted from material textures tiling. Then all the other maps you may need (like the AO)

[Rolig Loon]

Thanks again.  In fact, all texturing was in Photoshop, with the AO applied as a multiply layer, so there was no baking in Blender other than the AO.  I always re-scale things in the UV map to make best use of space.  In this case, I was not as careful as I should have been to scale the edges of the bench to the top.  I realized that this morning as I applying the local texture to test, so I will need to go back in and mess with the UV map again and re-bake the AO and retexture if I want to perfect this bench.  I'd certainly do that for something I intend to release into the wild.  Not sure that I will take the time for benches in my own back yard. 

[OptimoMaximo]

14 hours ago, Rolig Loon said:

I always re-scale things in the UV map to make best use of space.

That is non-uniform scaling, so watch out for that. Once averaged and packed, UV islands shouldn't be touched (too much) in regard to their scale so that each surface gets its proportional number of pixels to keep the whole map resolution/proportion-consistent.