Tangent spaced Normal Map seam visible in uv border area

[Jake Koronikov]

I tried that too and it did not help, the value of 22 px was the largest I tested. Also the seam is independent of normal map background color. The maps I have tested are generated using Blender, XNormal and ZBrush. All of them produce the same effect.

The irritating issue is that the normal map seam really breaks the normals - the vertex normals on each side of the seam are clearly pointing to different directions. Inworld SL the issue is much more noticable than in static images above. Technically it looks just like the verteces having two tangent info (from two different locations in uv-map) are not "averaged" using those two values. They pick the tangent info from one side only. When looking more close the normals stabilize again after going one polygon away from the seam line.

 

 

[RipleyVonD]

---

Jake Koronikov wrote:

 

 

 

---

After a second look, it seems like the UV Island that represents the back isn't sharing the same pixel texture space as the UV seams around the waist and the pectoral. The only way to remedy this is to use a larger size texture or stretch that part of the UV island to match the size of the other UV Island. 

Testing your UV Island with a checkerboard texture is good to unsure some parts of a UV Island is not overly reduced or enlarged. Before baking out your Normal Map its good practice to check and ensure your UV Islands are uniform as possible. 

This should be the reason why it works with your other UV Layout, because the other UV Layout looks reasonable sized to the other UV Island. This can also be achieved with irregular sized UV Islands you just need to ensure some parts aren't overly reduced or zoomed out. You can use a checkerboard texture and adjusting the square sizing to spot the reduced and zoomed parts.

[Jake Koronikov]

 

I wish that would be the explanation. But the uv layout is very consistent, the stretching is about zero and size difference is not even noticeable in render. And sure I have used checker grids for this. Also, the back side is "larger" measured as an area. So it is natural that the uv island is wider too when unwrapped.

(ETA: Added following text and checker photo): Also, in the SL render the vertex normals both side of the seam are clearly pointing to different directions. The direction of the normals differ so much on seam area that it cannot be explained by minor uv-island size difference.

[RipleyVonD]

Can you double the amount of your blocks? Those blocks might be too big to notice small differences. 

Actualy, no they are reasonably closely sized.

[Jake Koronikov]

Sure, feel free to edit. I will post you here a close up of the most problematic area (in the previous inworld render the area where the seam is most visible).

I also tested a moment ago an UV map that is exaclty consistent with size. I can post you later that image too...but I am quite sure the size of uv-islands is not the issue here. The issue is somewhere in the tangent space, vertex normals across the uv border, averaging of those vertex normals, shader vs. baker difference in handling the tangent space...dont know.

What makes me wonder most is the fact that other real time shaders I tested (well..umh.. you cant really call them engines but..) work fine. Those are Blender GLSL viewport and XNormal 3D view. Only this SL gives me crazy renders. With both LL viewer and Firestorm viewer.

 

 

[RipleyVonD]

The only thing I can think of at the moment is there is not enough edge padding. Can you show your current Normal Map? Or Is it the same one I saw above? If its the same one I saw above, it still needs more edge padding. Dont worry about pixel amount in edge padding focus on how much is actualy extending away from the UV Islands. Strech the UV padding atleast as much as i did, or a little more if you need to.

It seems to me the Back UV island around the waist isnt using the same amount of texture pixels producing less detail definition. Below is the edited Normal Map showing how I would ensure the waist ant the pectoral area are reasonably closely sized along the UV seam.

Of course the UV Island would need to be edited and then rebake the Normal Map.

 

 

[RipleyVonD]

---

 

---

With this Normal Map you are getting close detail definition and no uv seam, yes because of the angle of the UV Layout matching square pixels, but also because they are using very close pixel utilization.

Along the seam is one of the most important area to match your pixel utilization and ensure the UV Island are closely sized.

If these two issues don't fix your UV seam then the last thing I can think of is Triangulating your mesh before you bake your Normal Map, and make sure you upload that exact mesh you Triangulated and use only Normal Maps baked from that same Triangulated mesh.

Triangulating before baking your Normal Map is something that is supposed to be done also. One more thing I use significantly smaller checkerboard squares to to test my distortion amount on my meshes, about 1/4 the size of the squares you are using on your checkerboard.

I've nearly exhausted myself from sitting infront of my screen. lol I'm gonna call it quits for awhile good luck.

[RipleyVonD]

I forgot to mention this last thing. Select all your faces and align your normals before you bake. That's all I can think of at the moment.

[Jake Koronikov]

I have to disagree, because in this square layout everything is wrong. The stretch is enormous, the size that uv islands occupy is wrong compared to their real measured area. Still, this square layout produces prefect normal map render inworld.

I would go back to much more simple example. Here is a half cylinder, with sculpted details normal mapped from subdivided hi-poly version. UV islands are perfectly same size, proportions are perfectly same in uv islands. For testing purposes, I triangulated the mesh before normal bake. The result is same with or without triangulation. I increased the edge padding to 25 px, whitch is insane...

All I did was that I rotated the islands to different angle, whitch mimics the situation in any complex uv-map.

 

Again, the issue comes visible when

- islands are not aligned according to uv-axeses.

- UV map contains uv-borders that have natural organic edge flow (with angles, eg uv-border is not a straight line)

 

 

And finally, inworld render, where the face normals point to wrong directions across the border line:

[RipleyVonD]

I forgot to also ask a key question. Which viewer are using? I'm using the latest Linden Lab viewer. Last thing I can offer is baking the Normal Map myself for that cylinder example and take a inworld screen shot.

[Jake Koronikov]

I tested this with both LL viewer and Firestorm. Same result.

[RipleyVonD]

This is about the most resilient Normal Map seam problem I have ever encountered. lol I'm gonna need to take sometime and really think about what the cause is. That last simple example clearly has nothing wrong with it. 

This is almost like a seam that actualy exists with unmerged geometry. Just weird.

[Jake Koronikov]

Yeah, this seam issue drives me crazy . The angle between border edges (edge loop, vertex loop) is the key element. If you straighten this simple cylinder uv border -> the seam disappears. To re-produce it, the uv map has to have angular edge flow in the border and the uv island counterpart flow must be in different angle. A weird setup but very very normal setup in organic shape models.

[RipleyVonD]

Yesterday I spent some more time thinking about what was happening with this UV seam, until I decided to import the mesh and Normal Map Jake sent me into Marmoset Toolbag, to see if the Normal Map would be interpreted correctly. Marmoset Toolbag is a great program for testing your 3D Models and Textures in a realtime renderer and its really affordable. Part of Its business model is accurately presenting Artists work (IMO) so I expected to see accurate results. The Normal Map is working correctly like I would expect it to work with its edge padding. It seems Second Life's tangent space interpretation of Normal Maps is different. 

Yesterday I didn't spend too much time wondering why the UV seam on this mesh was so resistant to getting fixed because I then realized I myself would never create such complicated UV islands on such a simple surface. All this mesh needs is a flat projection and one UV island and that's all. The UV seam here was created from improper UV mapping, and as Jake stated rotating the Island to match the other Island fixes the UV seam. This example isn't a realistic example of a Normal Map seam. 

The UV seam on your male character is a real world example of a UV seam that can be fixed. When I said UV islands weren't dependent on angle that's assuming you choose your UV seams in places where you find hard edges, places where your UV islands split your vertices's, and also keeping your UV islands right side up as possible to closely match what you see on your model. When I look back at my castle's UV mapping (my first SL project) I mostly have my UV seams running along hard edges away from large flat surfaces. If I do have a seam splitting a flat area its usually away from view and its part of the same UV island usually keeping both ends of the split right side up because its from the same island, thus not showing a seam. 

Normal Maps work good enough at the moment from what I can see with my castle, but good practice with UV mapping is essential for a good result to avoid UV seams. 

So that brings me to Jakes male UV seam I still wonder whats going on there.

 

 

 

[RipleyVonD]

---

RipleyVonD wrote:

---

 

---

With this Normal Map you are getting close detail definition and no uv seam, yes because of the angle of the UV Layout matching square pixels, but also because they are using very close pixel utilization.

Along the seam is one of the most important area to match your pixel utilization and ensure the UV Island are closely sized.

If these two issues don't fix your UV seam then the last thing I can think of is Triangulating your mesh before you bake your Normal Map, and make sure you upload that exact mesh you Triangulated and use only Normal Maps baked from that same Triangulated mesh.

Triangulating before baking your Normal Map is something that is supposed to be done also. One more thing I use significantly smaller checkerboard squares to to test my distortion amount on my meshes, about 1/4 the size of the squares you are using on your checkerboard.

I've nearly exhausted myself from sitting infront of my screen. lol I'm gonna call it quits for awhile good luck.

---

In this post I wasn't supporting this type of UV mapping for an organic shape like the body, I was just pointing out the few things it was doing right that can be applied to the proper mapping.

[Jake Koronikov]

I think this issue will not be solved without LL comments. As far as I could find out the whole thing is related to tangent basis that is used by baker and its suitability for corresponding shader tanget basis. It is possible that baker calculated tangent basis is different that what shader (SL) expects.

Normal map uses a vertex data that is called tangent basis. The normal map has three channels: blue as Normal vector, red as tangent vector and green as bi-tangent vector. Tagent follows usually U-coordinate and Bi-tangent follows V-coordinate in uv-texture-space. During render, the tangent space is converted to actual world space and rendered using final face normal that is received from normal map vector calculation. (or world space is converted into a tangent space, but anyway).

Seam verteces are a special case, because they receive two different tangent info from two points in the normal map. Technically the seam area has two verteces, but in smooth mesh only one is rendered tho. But: the face-normals adjacent to seam verteces are affected by the vertex-normals of the seam...

Those two points have different tangent space in this cylinder example...because the uv:s are rotated...and do not share same tangent basis anymore. A good guess might be, that SL shader system handles this situation in a different way than baker does. This causes problems when seam line tangent basis is very much different on each side. The smaller difference there is between tangent basis the less visible the seam is.

A couple of images:

 

In above example the tangent basis is different across the seam.

Many many years ago 3ds Max had same kind of a issue when rendering normal maps. At that time there was a lot of talking and someone even suggested that the shader has a bug.  They suspected that tangent space information is genarated after breaking verteces that have more than one uv point. This way the seam verteces do not have same normal anymore. The generation should have been done before the break and use an averaged normal of two verteces, instead of only one.

Anyway. I will give up trying to fix this in bake phase. Instead I try to organize the uv-islands another way and use some manual painting in Photoshop green and red channel to make the remaining small seam to disappear.

 

 

[RipleyVonD]

Manually painting Normal Maps to fix seam issues, seems like extreme measures. I remember manual editing back in 2005 when I would edit FarCry Normal Maps (I was so green then lol) back then I would use Nvidias DDS Plugins for Photoshop. This plugin had an option to Re-normalize your Normal Map after editing it. 

http://www.bencloward.com/tutorials_normal_maps12.shtml

I do some manual editing to my normal maps and they turn out just fine without re-normalizing, its probably because of the type of editing I choose to do, I don't dare touch the colors on a Normal Map in any way, flipping channels or anything.

The only manual editing I do is clean erase in pencil mode to combine details from two different bakes of the same UV Layout, or paint in a Normal Map's base color to shave off some height (with the brush tool) or to completely remove height on a Normal Map. I usually bake out two different bakes to try and get correctly baked details in different areas and then use the erase tool in pencil mode to combine correctly baked details into one map. Thats all the type of editing I dare try in Photoshop, but I rarely try and paint in a Normal Maps base color to adjust height, I would rather fix surface details on my high resolution model.

I haven't had the need to fix Normal Map seam issues by manually painting a Normal Map. I usually add enough edge padding, keep my seams away from flat surfaces, and try to keep my islands right side up. In some cases you will need to add a seam on a flat surface, but like you saw from your example don't rotate your Islands unnecessarily, try to keep them in close relation to how they actually look on your model. 

Getting a perfect bake on complicated models in one shot can be difficult sometimes, so i adjust my settings and bakeout at least two maps, then in Photoshop I combine correctly baked portions into one map. Another thing, never resize your Normal Maps with a filter that adds a sharpening effect, it can be done by accident. Normal Maps don't need a sharpening effect applied to them they are not treated like regular diffuse textures. 

I hope that helps, goodluck.

[Drongle McMahon]

I haven't been able to make a simple reproduction of this. So this reply is not directly about the seam problem. However, it is about artefacts based on mismatched tangent base calculation in programs generating and rendering a normal map. There is a freeware thingy called Handplane that can take an object-space normal map and a low-poly obj file and generate output tangent-space normal maps for different rendering engines. SL isn't among the target rendering engines, but, for me so far, using the Unity engine output seems to give maps with much less distortion than the maps pruduced in Blender. It's just possible that this might also mitigate the seam problems decribed here.

Here is a picture of the same scene with a model using the Blender tangent-space map (top) and the one generated by Handplane (below). The workflow was...  Low poly is a simple cube, triangulated before doing anything, all smooth shaded; Make traditional UV unwrap after specifying seams; Export collada and obj (latter including normals - I don't know if that matters); Bake normals from high poly to tangent space and save as Blender-generated normal map. Bake normals from high poly to object space ans save for input to Handplane; Use Handplane with low poly being the exported obj file, object space normal map from Blender, using xNormal as input map generator (Blender and xNormal use the same tangent basis calculation, so, maybe same for object space too?), using Unity as output rendrer. Other settings (channel flips) left at default; use output as Handplane generated map. Upload lo-poly collada and both  normal maps to Aditi; set up scene with two objects to maximize shading artefacts with Blender normal map; screencapture; replace Blender map with Handplane map; screencapture again.

Here are the two maps (reduced), with the differences (value x 5) in between.

[Jake Koronikov]

Hey Drongle!. That Handplane you mentioned looked like a very intersting little tool. I dowloaded it too and tested a bit. However it did not solve this seam issue ;/. The seam looks excalty same with or without using Handplane. But I found and easy way for you to reproduce this all (if you wish to join this neurotic obsession team of seam chasers LOL)

You propably have a standard SL avatar file. Here is a simple workflow:

- Use SL standard avatar as low-poly mesh (ETA: Delete the left side of the SL avatar to be exact in this test. There are overlapping uvs in hand and arms, whitch could in theory affect the result  -> images below has both sides still, but the result is same anyway)

- Duplicate it and make a Hi-poly by just subdividing a couple of times.

- Sculpt some random lines across the UV island seam areas

- Bake a normal map to this standard SL avatar uv layout, maybe using XNormal,Blender or anything you wish

- Import, rezz, set the NM, and Move the sun around to get a shading where light comes from sides, not direclty from up

What I get on the shoulder area is a really ugly seam....those damn normals point to €¤%%##¤¤ (sensored).

I have made some tests with hard edges, and it is sort of a solution. The reason is that in hard edge there are now two separate verteces on each side of the seam. So vertex on the other side gets it normal from its tangent space, and the other vertex from its own tangent space. And they render well. But of course organic model can not have hard edges thrown here and there. It would ruin the smooth shading totally. So in principle, this is an issue related to single vertex in smooth shading. And that vertex receiving some weird normal from ...somehwere. That normal will destroy the adjasent face normal too, because in smooth shading the face normal is interpolated between vertex normals on each corner of the face.

Here are the photos:

 

And the uv layout with the problematic area highlighted:

[arton Rotaru]

Handplane3D is nice. However, I get even slightly better results with xNormal, and using an "Unity3D Tangent Basis Calculator for XNormal" (google it), and baking a tangent space normal map as usual. Though, I have the impression the Unity3D tangent space still isn't exactly the same as the one SL is using.

 

@Jake,

you should file a jira with your findings. I wouldn't be too surprised if this seam issue is a bug, or flaw in the SL shader code.

 

 

[Jake Koronikov]

---

arton Rotaru wrote:

Handplane3D is nice. However, I get even slightly better results with xNormal, and using an "Unity3D Tangent Basis Calculator for XNormal" (google it), and baking a tangent space normal map as usual. Though, I have the impression the Unity3D tangent space still isn't exactly the same as the one SL is using.

 

@Jake,

you should file a jira with your findings. I wouldn't be too surprised if this seam issue is a bug, or flaw in the SL shader code.

 

 

---

Thanks for the hint. I will check that Unity Tangent Basis calculator. I made a final test with a quite old software (propably seen it's best before date by now hehe), Nvidia FX Composer 2.5. It allows you to design shaders and write high level code in them. I I think it will not allow you to go in-depth level of calculating tangent basis tho.

FX Composer rendered all of these tests just fine, no seam visible. I will file a Jira next week. Just waiting a couple of days if someone finds a reasonable non-bug explanation and posts it here.

 

 

[Drongle McMahon]

Using you example, I can now reproduce the seam effect in a simple pipe bend with a suitably strained UV map. The effect is very obvious for the map baked from high poly with sculpted effects (left). However, it's more interesting comparing no normal map (middle) with the blank normal map (right, mine or the inbuilt blank). With no map, there is no seam. With the blank map, there is a seam. So it's something happening with any UV map. The effect of the blank map can't have anything to do with tangent basis etc. Since there is no seam with no normal map, it's not to do with the basic rendering in ALM. So it must be happening in the code that applies the normal  map. My blank is <128,128,255>. It might be interesting to try slightly different colours. The pictures use blank spec map with the default shiny settings, g=51, e=0.

ETA let us know if/when you submit a jira, and I will add this example (I think we can now before triage, no?).

ETA2 Here's the model and uv map - affected seam highlighted.

[Jake Koronikov]

Jira is now filed: https://jira.secondlife.com/browse/BUG-5975

 

[Drongle McMahon]

Comment added, but it seems that I can't add attachments, which makes it practically useless. If you can add the two pictures from my message above (46), that will help. I could add the uv baked from high poly here (although it is 1024x1024 !), but I don't know how to get you the dae file, which is the crucial thing :matte-motes-confused:

 

[Masami Kuramoto]

---

Drongle McMahon wrote:

Here are the two maps (reduced), with the differences (value x 5) in between.

 

---

That kind of "waviness" in normal-mapped surfaces is not just due to differing tangent spaces. It's also a result of baking to the wrong kind of geometry. Your normal maps show that your bake target was a smooth-shaded cube, i.e. a low-poly sphere. That's why there are those large color gradients in areas that should be completely flat (and therefore use nothing but RGB color 127,127,255). If you had used a cube with sharp edges, your normal map would look more like this:

This map comes straight out of Blender, without postprocessing in Handplane or similar tools. Color gradients occur only in areas where the high-poly cube had additional detail such as ridges and bevels. All the coplanar regions are uniform 127,127,255 and therefore maintain their flat appearance in-world when mapped to a basic 12-triangle cube with split edges: