Q on LoD texturing and texture memory use

[Kwakkelde Kwak]

I know how to make an "efficient" mesh, but I'm still pretty clueless about one thing that might affect performance in a big way.

If I use a 1024 texture my high LoD, is it best to use that same texture on all the other LoDs when possible, or is it better to use a duplicate, smaller texture on the lower LoDs? In other words, do people who are far away and stay far away so they never get to see the higher LoDs have to load all textures on the model or just the ones they get to see? The server load would be around 30% higher as far as loading the textures goes, with (for example purposes) 1024 on high, 512 on med, 256 on low and 128 on lowest, but the load on the graphics card and the streaming load would be far far less.

[Drongle McMahon]

My pretty ignorant guess would be that jpeg2000 allows progressive download of increasing reolutions, so that the full 1024 resolution would not be requested/downloaded until it was required. There must be a per texture overhead to be added if you used multiple textures this way. We need someone who knows the texture download pipeline code and the details of jpeg libraries to answe this authoritatively. I suggest you table a question on the content mesh UG agenda. Or you could spend a few happy hours staring at the texture console (developer menu), which might or might not make you any wiser!

[Kwakkelde Kwak]

---

Drongle McMahon wrote:

My pretty ignorant guess would be that jpeg2000 allows progressive download of increasing reolutions, so that the full 1024 resolution would not be requested/downloaded until it was required.

---

That should be possible and would be the best way, still we both don't know if this is the case....

---

I suggest you table a question on the content mesh UG agenda.

---

Good idea, where and how do I do that?

 

 

[Kwakkelde Kwak]

Ah well I think I just woke up. If you use 4 textures on the model, all 4 will have to be applied to all LoDs...DUH!

[Fizz Savira]

Second Life, for as long as I've known it, has always done progressive download of the "mip maps", loading a low resolution first and then loading higher resolutions as needed, using something similiar to the LOD mechanic (I'm guessing) to determine which resolution to use.

So using one 1024 texture for your high LOD is the way to go (assuming that one 1024 is the right size for your situation; sometimes I go as low as 256 for some things), and one of the great things about mesh is that you can use a single texture for multiple pieces of your model, which means your overall texture load for that object is less than the sculpted equivalent. So yay!

 

[leliel Mirihi]

The GPU already does that for you, when told to, You don't even need to give it the mip levels as it can auto generate them, when told to. The GPU can also do some fancy linear interpolation between mip levels, when told to. And even anisotropic filtering when the texture is viewed at oblique angles, when told to.

In short the GPU can do all of that for you and the viewer does tell it to do that stuff.

[Kwakkelde Kwak]

---

Fizz Savira wrote:

Second Life, for as long as I've known it, has always done progressive download of the "mip maps", loading a low resolution first and then loading higher resolutions as needed, using something similiar to the LOD mechanic (I'm guessing) to determine which resolution to use.

---

Yes I'm guessing aswell, that's the entire issue..well it was until as I said woke up and came to the conclusion using duplicate textures is useless since all 4 have to be on all 4 models. So even if the download doesn't progress with closing in on the object, one 1024 would still be the best choice.

---

and one of the great things about mesh is that you can use a single texture for multiple pieces of your model, which means your overall texture load for that object is less than the sculpted equivalent. So yay!

---

I don't see how that is an advantage over a sculpt. A sculpt has one texture.

UV mapping and using several smaller textures or one smaller texture, now that is a great advantage.

 

[Kwakkelde Kwak]

Ofcourse the GPU can do that...IF the viewer tells it to. What else can a GPU do than what's it told to do? It has no brain, it's not creative, it's not rebellious..

Anyway, if  the viewer does send the textures in different resolutions as you say, that mystery is solved, thanks. Not that it would have made any difference in how to texture the mesh, since adding is just adding with the "all LODs need all materials" way of uploading.

[leliel Mirihi]

---

Kwakkelde Kwak wrote:

Anyway, if  the viewer does send the textures in different resolutions as you say, that mystery is solved, thanks.

---

I don't know if the viewer just gives the GPU the full res texture and tells it to auto generate the mipmap or if it fills in the mipmap with the discard levels from jpeg2k, but in the long run it doesn't really matter.

---

Not that it would have made any difference in how to texture the mesh, since adding is just adding with the "all LODs need all materials" way of uploading.

---

Using separate textures to simulate mipmaps would cause a noticeable performance drop when done sim wide. Mostly because you have no way of telling the viewer not to do mipmaping for that objects. There's also the question of why you'd want to do that by hand when the GPU can figure it out for you and do a better job of it. :smileysurprised:

 

 

[Drongle McMahon]

"all LODs need all materials"

Yes. It wasn't supposed to be that way, only the high LOD should need them all, and the lower ones any subset, but there was a regression that reimposed that rule. Mesh/Conent UG agenda is here.

[Kwakkelde Kwak]

Yes hence my DUH and all I posted after that:) Now if the lower LoDs didn't have to include the high LoD model materials, you could use textures far far smaller, in a lot of cases probably the good ole 32x32 blank texture with some color. Now that would mean some performance gain no doubt. Maybe LL will roll back that bad decision, but I'm sure they had their reasons...

Btw I'm pretty sure I have a better brain than a GPU... if the GPU renders a low LoD model a couple of pixels wide, I'm sure something a lot smaller and cost effective than what the GPU can spit out will do just fine. Again, for now it makes no difference.

My question was a bit academic in the first place with 1024 textures used in unjustifyable ways all over the grid, whether I use a 64 or 128 instead of a GPU genereated 256 equivalent won't have a very big impact.

[Kwakkelde Kwak]

---

Drongle McMahon wrote:

Mesh/Conent UG agenda is

here

.

---

Thanks, I'll see if I can put together something less all over the place than this thread...

[leliel Mirihi]

---

Kwakkelde Kwak wrote:

Btw I'm pretty sure I have a better brain than a GPU... if the GPU renders a low LoD model a couple of pixels wide, I'm sure something a lot smaller and cost effective than what the GPU can spit out will do just fine. Again, for now it makes no difference.

---

I'm shure you are a lot smarter and more creative than the GPU but it has a lot more information than you do. The GPU knows exactly how many pixels each triangle will take up and what angle it is relative to the camera so it can sample and filter the textures to perfectly match every triangle on the screen. You on the other hand have no such luck, all you can to is make a best guess as to how large a texture should be based on some rough estimates that are guarantied to not be right in all cases.

There is also the fact that even if you did do the mipmaps yourself you'd still have to turn on some texture filtering just so they'd look good. 

[Kwakkelde Kwak]

I'm pretty sure the GPU does its job better than I would and certainly a lot faster. The thing is, as you said so yourself, it ofcourse has to work with what it's given. I as a human being on the other hand can decide whether something looks good enough at a certain distance, so I can reduce and reduce the texture the GPU has to work with to a point where it still looks good enough. Then the GPU can go from there with a much simpler task.

Again, the gain would be overshadowed by the inefficient texture use in SL, so it's a bit academic and probably not worth the efford in this environment. A 5% gain for one texture (just guessing something) won't be very noticable when someone teleports in with 20 unique 1024 textures on their avatar.

I do wonder about the streaming. Does the server send the full resolution texture or the reduced (mipmapped) one to the viewer? On a busy sim that can make a lot of difference I think. (no idea how well those jpg2000's compress, normal jpg's don't compress very well) Next to the streaming there's the cache. Does it store a load of full res textures or just the reduced ones that have actually been used? And another thing, how does texture load on the server-viewer system compare to the script and geometry updates and chat and voice and all the other things?

[leliel Mirihi]

---

Kwakkelde Kwak wrote:

I'm pretty sure the GPU does its job better than I would and certainly a lot faster. The thing is, as you said so yourself, it ofcourse has to work with what it's given. I as a human being on the other hand can decide whether something looks good enough at a certain distance, so I can reduce and reduce the texture the GPU has to work with to a point where it still looks good enough. Then the GPU can go from there with a much simpler task.

Again, the gain would be overshadowed by the inefficient texture use in SL, so it's a bit academic and probably not worth the efford in this environment. A 5% gain for one texture (just guessing something) won't be very noticable when someone teleports in with 20 unique 1024 textures on their avatar.

---

I think we drifted into talking about different things. At first you were talking about simulating mipmaps using separate materials, which honestly, is a horrible idea (except for tricks like using a billboard for the lowest LOD of an object). Where as now you seem to be talking about tweaking how the GPU does texture filtering, which is a better idea.

I don't agree with your example tho. If a texture has such low detail that you can get away with using a significantly smaller image than what a given mip level would call for then wouldn't that mean the full res texture is too large?

---

I do wonder about the streaming. Does the server send the full resolution texture or the reduced (mipmapped) one to the viewer? On a busy sim that can make a lot of difference I think. (no idea how well those jpg2000's compress, normal jpg's don't compress very well) Next to the streaming there's the cache. Does it store a load of full res textures or just the reduced ones that have actually been used? And another thing, how does texture load on the server-viewer system compare to the script and geometry updates and chat and voice and all the other things?

---

JPEG2000 inherently creates mipmaps due to how it works (wavelet transforms). The spec takes advantage of this to enable some cool features, the main one being partial downloads. You can download part of the file and decode it to a lower res image than the main one. This is in fact the main reason LL chose jpeg2k over other formats despite its high processing requirements at the time. The viewer uses partial downloads so it only has to get textures up to the resolution it needs, instead of having to download the full res version of every texture in sight. The viewer caches however much of the texture its downloaded so far.

As for quality, jpeg2k is a significant improvement over jpeg1994, and 12 years later is still considered one of the better image codecs. With one caveat however, they were both made for compressing images of the real world, they don't do as well with artificial images.

I don't know how much of a load sending textures is on the servers, you'd have to ask the lindens about that. For the viewer texture loading and decompressing is the second hardest thing after rendering.

[Kwakkelde Kwak]

---

leliel Mirihi wrote:

I think we drifted into talking about different things. At first you were talking about simulating mipmaps using separate materials, which honestly, is a horrible idea (except for tricks like using a billboard for the lowest LOD of an object).

---

Yes since the first wasn't usable for various reasons, I took a next possible step. I'm just thinking out loud and any feedback will help speeding up that process a lot or fill in some blanks...

Using a billboard or imposter isn't anything like mipmapping, but it is a fine example of how you can reduce data load on the servers without losing any looks, in fact they can improve them quite a bit. I am just thinking out loud about other possible ways to reduce load.

btw, imposters can be used in far higher LoDs than just lowest. I've made a hanging chain with imposters for all LoDs except highest. The result was far far better in looks than any geometry would have been. You can imagine the landimpact was pretty good aswell. At a size of 5x5 or so meters and close to 50 links, it has a landimpact of 2 and LoD changes are as good as invisible. (The pic is the highest LoD)

 

---

I don't agree with your example tho. If a texture has such low detail that you can get away with using a significantly smaller image than what a given mip level would call for then wouldn't that mean the full res texture is too large?

---

That's where the difference between a GPU and a human brain is important. The GPU only has numbers to work with. The endresult of an object on screen isn't determined by numbers though, but how it looks. A human can see that so I'm pretty sure that human will make the better decision. If all the GPU does is mipmapping, it means it uses the high res texture as a base, with all its colours and details. Use a smaller texture and less colour variation, possibly to a point where there is no variation at all, and the GPU can mipmap with that.

I don't see how this technique has got anything to do with the highest LoD texture being too large.

---

JPEG2000 inherently creates mipmaps due to how it works (wavelet transforms). The spec takes advantage of this to enable some cool features, the main one being partial downloads. You can download part of the file and decode it to a lower res image than the main one. This is in fact the main reason LL chose jpeg2k over other formats despite its high processing requirements at the time. The viewer uses partial downloads so it only has to get textures up to the resolution it needs, instead of having to download the full res version of every texture in sight. The viewer caches however much of the texture its downloaded so far.

---

Yes that's what I was asking. Pretty smart and it means the original idea I had, isn't useful, even if the lower LoDs didn't need all materials.

 

 

[leliel Mirihi]

---

Kwakkelde Kwak wrote:

That's where the difference between a GPU and a human brain is important. The GPU only has numbers to work with. The endresult of an object on screen isn't determined by numbers though, but how it looks. A human can see that so I'm pretty sure that human will make the better decision. If all the GPU does is mipmapping, it means it uses the high res texture as a base, with all its colours and details. Use a smaller texture and less colour variation, possibly to a point where there is no variation at all, and the GPU can mipmap with that.

I don't see how this technique has got anything to do with the highest LoD texture being too large.

---

 I'm still not seeing it. Can you give an example of how this would work and not just turn into a blurry mess?

I think you're focusing so much on optimizing mipmaps without realizing that mipmaps are the optimization. Back in the old days GPUs would use the full texture for everything, every single frame it would down scale the texture to the needed size. But that used up a lot of memory bandwidth. So some one came up with the bright idea of storing pre scaled versions of the texture along with it that way the GPU would only have to fetch the closest one and scale from it saving a large amount of bandwidth.

[Kwakkelde Kwak]

You're mixing two things up now. According to what you said earlier the GPU doesn't have to do any mipmapping, or hardly, since the viewer sends the reduced texture in the first place. Still a question what happens if you walk away from an object instead of walking towards it, since then your viewer has received the bigger texture, maybe there's some mipmapping then.

Anyway, the example. You have a necklace made out of silver chainlinks and you use one full unwrap on it, no repeats (which would in most cases be stupid, but it's an example). As soon as people are 10 meters away from you(probably a lot closer even), they won't see any detail in the texture, but the entire texture area is still pretty big. In that case I would say it's better to make the necklace a plain color than a reduced version of the high res one.

Again..again..again... it's all very academic, since the performance gain would be so small it won't be noticable next to the other textures.

[leliel Mirihi]

---

Kwakkelde Kwak wrote:

You're mixing two things up now. According to what you said earlier the GPU doesn't have to do any mipmapping, or hardly, since the viewer sends the reduced texture in the first place. Still a question what happens if you walk away from an object instead of walking towards it, since then your viewer has received the bigger texture, maybe there's some mipmapping then.

---

 

What? Either we're having a major miss communication or you don't understand what mipmaping is.

Here's an OpenGL tutorial that explans how all the texture filtering options work and shows you why it's a good idea to use them.

---

Anyway, the example. You have a necklace made out of silver chainlinks and you use one full unwrap on it, no repeats (which would in most cases be stupid, but it's an example). As soon as people are 10 meters away from you(probably a lot closer even), they won't see any detail in the texture, but the entire texture area is still pretty big. In that case I would say it's better to make the necklace a plain color than a reduced version of the high res one.

Again..again..again... it's all very academic, since the performance gain would be so small it won't be noticable next to the other textures.

---

 

Why not just use a low res texture to begin with?

[Kwakkelde Kwak]

---

leliel Mirihi wrote:

What? Either we're having a major miss communication or you don't understand what mipmaping is.

---

I know what mipmapping is. You said the viewer sends a lower res texture when not close to an object. That gives essentially the same result as mipmapping then. Or at least a first step in that process.

 

---

Why not just use a low res texture to begin with?

---

Did you read my post at all? the example? On for example small objects, where all detail is lost at a distance not that big, you can use a 1x1 pixel image (well if SL would support such small ones, but 32x32 with only one colour will do, the "blank" texture) The small object will be viewed up close though, then a high res texture can be preferred. This is exactly why I say a human being will make a better choice than some piece of machinery. I expect the GPU to adjust the texture to the size of the object on screen. Object twice as small, next step in mipmap. But some objects escape our attention when not in our face, so the texture size could regress faster than that. I also don't think mipmapping takes LoD changes into the calculation.

 

 

[leliel Mirihi]

---

Kwakkelde Kwak wrote:

---

leliel Mirihi wrote:

What? Either we're having a major miss communication or you don't understand what mipmaping is.

---

I know what mipmapping is. You said the viewer sends a lower res texture when not close to an object. That gives essentially the same result as mipmapping then. Or at least a first step in that process.

---

I think we've completely lost each other on this one.

---

Did you read my post at all? the example? On for example small objects, where all detail is lost at a distance not that big, you can use a 1x1 pixel image (well if SL would support such small ones, but 32x32 with only one colour will do, the "blank" texture) The small object will be viewed up close though, then a high res texture can be preferred. This is exactly why I say a human being will make a better choice than some piece of machinery. I expect the GPU to adjust the texture to the size of the object on screen. Object twice as small, next step in mipmap. But some objects escape our attention when not in our face, so the texture size could regress faster than that. I also don't think mipmapping takes LoD changes into the calculation.

---

 

In a real game a human would decide it's a waste to use such a high res texture for something you'd have to zoom into in order to see. One of the problems in sl is that the residents often decide the exact opposite and plaster the smallest of trinkets with the biggest texture they can just on the off chance people will cam in on it all day long.

ETA: In your example above of the necklace covered with one texture, no repeats. By the time your camera is 10m away the necklace is only taking up 100 pixels at most, the GPU would have long since dropped to the 32x32 mip level or lower.

[Kwakkelde Kwak]

---

leliel Mirihi wrote:

In a real game a human would decide it's a waste to use such a high res texture for something you'd have to zoom into in order to see. One of the problems in sl is that the residents often decide the exact opposite and plaster the smallest of trinkets with the biggest texture they can just on the off chance people will cam in on it all day long.

---

Yes and isn't that exactly what confuses so many people on both the forums and on the grid? SL is indeed realtime, but it's not a game. People dress up, can zoom in on things, care about small details you will never find in a game. So that is exactly why you can make big differences between the LoDs. A necklace with nice detail on the highest LoD won't bother anyone. (A nice necklace with high detail on lower LoDs will.) Not until they zoom in real close and that is when the rest from their screen disappears and they can focus on that one single item. I know perfectly well how to build low poly on all LoDs (and I am very careful with textures), I just don't think that's a wise decision in a lot of cases.

A "big" texture on the highest LoD won't be visible on lower LoDs/bigger distances/smaller sized objects as you keep explaining. You say both the viewer and graphics card make sure of this.

---

ETA: In your example above of the necklace covered with one texture, no repeats. By the time your camera is 10m away the necklace is only taking up 100 pixels at most, the GPU would have long since dropped to the 32x32 mip level or lower.

---

100 pixels on your screen is most definately not 100 pixels of texture. on something like a necklace you only get to see maybe 20-30%. I gave the 10 meters just as "a"number...I didn't test it. Use 2 meters instead then if it makes a difference or 3 or 6, any distance where the GPU would pick a multicoloured 64x64 mipmapped texture.

 

 

[leliel Mirihi]

---

Kwakkelde Kwak wrote:

A "big" texture on the highest LoD won't be visible on lower LoDs/bigger distances/smaller sized objects as you keep explaining. You say both the viewer and graphics card make sure of this.

---

 The problem is you now have more date you want to store in vram then what you can fit. My video card has 1GB of ram and I routinely see the viewer eat all of it up and still want more. That's because people use high res textures way more often than they need to. They don't use texture atlases as much as they should (mesh now makes this easier). 

---

100 pixels on your screen is most definately not 100 pixels of texture. on something like a necklace you only get to see maybe 20-30%. I gave the 10 meters just as "a"number...I didn't test it. Use 2 meters instead then if it makes a difference or 3 or 6, any distance where the GPU would pick a multicoloured 64x64 mipmapped texture.

 

 

---

The point is you're really not going to save much in the way of memory bandwidth doing this, and that's not what's in short supply (in this case). These big textures take up more vram and that's what we're short on.

Geometry and texture LOD may work similarly but they're intended to solve very different performance problems. Applying the theory of one to the other does not work so well.

ETA: I suppose I should expand on this. The way LOD is done for Geometry and textures is similar but they are handled very differently. Texture LOD is done by the GPU where as geometry LOD is done by the program.

When you switch between geometry LODs the viewer is actually unloading one list of vertices and loading the next, both don't stay in vram at the same time. With texture LOD it's all done by the GPU and the whole texture mipmap stays in vram the whole time, no matter how far the object is from the camera. SL is a bit of a special case since the viewer only loads the texture up to what is currently needed, but once you zoom in on the object the whole texture is in vram and stays there for as long as the object is in range. The GPU can not unload part of a texture and the viewer never does.

[Kwakkelde Kwak]

---

leliel Mirihi wrote:

 The problem is you now have more date you want to store in vram then what you can fit. My video card has 1GB of ram and I routinely see the viewer eat all of it up and still want more. That's because people use high res textures way more often than they need to. They don't use texture atlases as much as they should (mesh now makes this easier).

---

Ok, now you have me confused. In two ways.

Why would I have more memory than will fit the vid memory?

And why would using atlasses be better for memory use?

One 1024x1024 texture uses as much memory as two 512x1024s to my best knowledge. I thought the big advantage was the textures on an object loading all at once rather than piece by piece. If there was no need for the entire set of materials on all LoDs, you could use two 512x1024s, on a lower LoD only one of them. I am tempted to say that is a reduction of memory use by  50%.

---

The point is you're really not going to save much in the way of memory bandwidth doing this, and that's not what's in short supply (in this case). These big textures take up more vram and that's what we're short on.

---

Yes very clear, I thought I said the savings were very small a couple of posts up. And how is a 32x32 oncoloured texture big?

---

Geometry and texture LOD may work similarly but they're intended to solve very different performance problems. Applying the theory of one to the other does not work so well.

---

I don't know where I said they are the same and can be treated the same way.

---

When you switch between geometry LODs the viewer is actually unloading one list of vertices and loading the next, both don't stay in vram at the same time. With texture LOD it's all done by the GPU and the whole texture mipmap stays in vram the whole time, no matter how far the object is from the camera. SL is a bit of a special case since the viewer only loads the texture up to what is currently needed, but once you zoom in on the object the whole texture is in vram and stays there for as long as the object is in range. The GPU can not unload part of a texture and the viewer never does.

---

Now this is interesting. And it more or less brings me back to the "fake" mipmapping. Again in order for it to work, the lower LoDs should have a subset of the higher LoD materials rather than the entire set. (As Drongle said the way is was supposed to be in the first place)

Also as I understand you, the entire texture is loaded into vram, this is not what you said earlier when you said the viewer uses a lower res version of that texture when far away before sending it to the GPU.

If the entire texture is always loaded into vram, that indeed means the GPU will pick its mipmap choice from that loaded memory depending on the size on screen. If you fake the mipmapping by using 1024 for LoD0, 512 for LoD1 etc...  and you walk away from an object, the biggest offender can be dumped from memory. The GPU will do its mipmapping with a smaller version, giving the exact same results visually, but using a fraction of the memory. When closing in on an object, the same will happen.

I can see problems with textures not loading fast enough, but memory wise it sounds a lot more efficient.

 

[leliel Mirihi]

---

Kwakkelde Kwak wrote:

Ok, now you have me confused. In two ways.

Why would I have more memory than will fit the vid memory? 

---

 Because you're plastering everything with high res textures! See the part below about the whole texture staying in vram. That's what's happening in sl right now, seriously go log in and turn on the texture console and render info console and you're see what I mean.

( Your whole example sounds to me like "How can I use a high res textures for tiny objects without it increasing memory use". Maybe I'm reading more into it than there really is but that's the way it looks to me.)

---

And why would using atlasses be better for memory use?

One 1024x1024 texture uses as much memory as two 512x1024s to my best knowledge. I thought the big advantage was the textures on an object loading all at once rather than piece by piece. If there was no need for the entire set of materials on all LoDs, you could use two 512x1024s, on a lower LoD only one of them. I am tempted to say that is a reduction of memory use by  50%.

---

One atlas for more than one object, i.e. the table and chair use the same texture. This works best for non square objects that would otherwise have a large amount of wasted texture space.

---

I don't know where I said they are the same and can be treated the same way.

---

You didn't say it but your example is based on the same idea (i.e. nonlinear drops).

---

Also as I understand you, the entire texture is loaded into vram, this is not what you said earlier when you said the viewer uses a lower res version of that texture when far away before sending it to the GPU.

---

The viewer only loads the low res texture when the object starts out far away, once you zoom in on the object it has to load the full texture which stays in vram even if you zoom back out

---

If you fake the mipmapping by using 1024 for LoD0, 512 for LoD1 etc...  and you walk away from an object, the biggest offender can be dumped from memory. The GPU will do its mipmapping with a smaller version, giving the exact same results visually, but using a fraction of the memory. When closing in on an object, the same will happen.

---

What makes you so shure the viewer will dump the large texture? The object is still in view after all. And this whole trick will only work if you have one copy of the object in view, if you have multiple copies at different LODs than you're now using way more vram then a normally made object.

.