Notes on big textures.

[animats]

I've been experimenting with an JPEG 2000 decoder. This is the format used for texture storage for Second Life. JPEG 2000 has the strange and useful property that if you just truncate the file to a shorter length, you still get a picture; it's just lower resolution.

Full scale, 650678 bytes.

 

This is what you get when you only read the first 65736 bytes.

 

Using only the first 4K bytes.

Below this level, the next drop is to greyscale at 2K bytes (not 2K x 2K image), and the decoder gives up around 1K bytes and reports an error. So reading 4K bytes is about the minimum.

The viewers use this. They can read only part of the file. I get the impression that they mostly download either the low-rez example above, or full size, not much in between. That's what you seem to see as textures come into focus.

That was an appropriate strategy when the sim servers served the assets. Their time is a scarce resource. But it's less of an issue now that assets come from a completely different set of servers. SL's asset system is on Akamai's content delivery network, with caches spread around the world. Making more, but smaller requests may be worth it. Worth a rethink.

Optimal strategy here is an interesting problem. If you make the smallest request possible for each texture, as you approach the object you have to make more requests, one at each scale. The optimal request is a function of distance, size, relative velocity, and look direction, plus how recently some other size of this texture was requested.

If the viewer is clever about this, oversized texture files shouldn't hurt anything. Most of the file shouldn't get fetched over the network most of the time. Unless you get really close; then the viewer has to get all the detail.

[Love Zhaoying]

So..is your proposal / conclusion that perhaps disabling the "load and display lower-rez portion of JPEG2000 textures" is no longer needed, and may actually be reducing viewer performance?

ETA..disabling / removing it in some scenarios, if not altogether..?

Removing the JPEG2000 steps would certainly have SOME kind of impact - could use newer / different image types, resolutions, etc. etc. etc.

Edited February 28, 2023 by Love Zhaoying

[Fluffy Sharkfin]

18 minutes ago, Love Zhaoying said:

So..is your proposal / conclusion that perhaps disabling the "load and display lower-rez portion of JPEG2000 textures" is no longer needed, and may actually be reducing viewer performance?

I believe what @animats is suggesting is that rather than sending a request for the entire JPEG2000 file as soon as the object in question is within draw distance it may be preferable to calculate how much of the file is required in order to display the texture at an appropriate texel density for the objects current size on-screen based on the objects distance from the camera, it's relative velocity (in order to ascertain if the object is approaching the camera and is therefore likely to be viewed at a larger size or if its traveling away from the camera and won't require additional texture information) and the direction in which the camera is facing (since objects within the cameras field of view should obviously have higher priority than objects that aren't currently being rendered).

As long as the assumptions are correct in regards to the potential performance improvements that could be gained from this approach as a result of the difference in the way in which SL assets are hosted compared to the previous approach it seems like a really good idea (but then I'm not a network engineer nor am I particularly familiar with SLs network architecture so I could be missing some critically important factor).

It certainly sounds plausible though and, if it works, could potentially even go some way to solving some of SLs texture loading issues.

[animats]

1 hour ago, Love Zhaoying said:

So..is your proposal / conclusion that perhaps disabling the "load and display lower-rez portion of JPEG2000 textures" is no longer needed, and may actually be reducing viewer performance?

Fetching lower rez images is essential. I'm saying that the strategy for picking which sizes to fetch might be improved. You can request full size, 1/4 size, 1/16th size, etc. all the way down to a 1x1 texture. Which ones to fetch and how often to update are interesting technical problems. Something I'm working on for my own viewer.

Incidentally, 1x1 textures, which are just colors, can be useful for distant objects.

Babbage Palisade, with all textures reduced to 1x1. Nearby objects look rather blah, but out past 100m or so, you can barely tell that there are no textures.

The goal is to allow longer draw distances without dragging down performance or running out of GPU memory.

(That's the Polish Embassy on the left, and the picture is from the roof of Ying Industries. Those places are, sadly, gone.)

Edited February 28, 2023 by animats

[Henri Beauchamp]

Unlike what you are suggesting, the viewer does use the full range of LODs of JPEG2000 textures: this is thanks to this mechanism that it is able to use so little memory, when compared with what would happen if it had to display all the textures at 0 discard level (i.e. max LOD/full resolution).

And yes, it makes a HUGE difference at both the RAM and VRAM consumption level, because it basically means that for each increased discard level, the displayed texture is four times smaller in size (pixels height and width are both divided by two at each level). E.g. a 1024x1024 pixels texture (as defined by the builder of the object using it) can therefore be finally used at 512x512, 256x256, 128x128, etc, depending on how important to the camera it is, i.e. how large is its ”display area” in the currently rendered scene; at 1024x1024, the (decoded) texture uses 4MB of RAM and 4MB of VRAM, while at 256x256, it only uses 256KB...

If you want to see what would happen if the viewer were to load the full res textures, enable the ”TextureLoadFullRes” debug setting, and get ready for a crash (unless you are in a place where there are not too many textures), because the memory consumption will soon skyrocket, till your RAM and VRAM are exhausted.

Also, the viewer is ”clever enough” to use HTTP range requests so to avoid having to re-download the low LOD part of the texture files (i.e. the already downloaded part) when it needs a higher LOD for it, and its textures cache is used intensively to avoid and re-download a texture when increasing its discard level to make room in memory for another, more prominent texture in your FOV; instead, the already cached raw texture is fetched from the disk cache and only re-decoded at a lower LOD.

Edited March 1, 2023 by Henri Beauchamp

[Qie Niangao]

My SL experience is very much texture-loading-limited (despite a beefy fiber connection). As a result, I run my viewers with very short draw distances. If some magic improved texture downloading, I'd crank up those draw distances—and probably hit a texture-loading constraint again, only this time with many more requests. So this would be a net improvement in my SL experience, which is great, but I don't think it would reduce the total bandwidth devoted to downloading textures, neither on my network nor at Akamai's servers (not that anybody said it should).

Also, regarding "velocity" as a predictor for pre-warming the texture cache: Personally, my camera pans around much more than it zooms (or rides a vehicle, etc.) into proximity of new textures. I doubt the cam's angular velocity would be an efficient predictor of future scene content because any pre-loading would need to be very fast to even matter, but for me that motion overwhelmingly drives change in what needs to be rendered, enough so that I doubt normal velocity would have much chance of making a difference in (my) actual use.

[animats]

15 hours ago, Henri Beauchamp said:

Unlike what you are suggesting, the viewer does use the full range of LODs of JPEG2000 textures:

Some of those LODs come from fetching part of the file, and some are made locally in the viewer by cutting down a higher resolution. The machinery is in there for both (I've looked at the code in Firestorm) but the strategy on when to do what lacks something. This is part of why you keep seeing objects that don't go to high texture resolution even when you're close.

[Love Zhaoying]

1 minute ago, animats said:

This is part of why you keep seeing objects that don't go to high texture resolution even when you're close.

Wasn't there some trick, like click on an object to get it to fully load the texture?

[Henri Beauchamp]

9 hours ago, animats said:

This is part of why you keep seeing objects that don't go to high texture resolution even when you're close.

This problem is rather most likely due to bad/insufficient virtual size re-evaluation as the camera FOV changes (only a small part of the texture virtual sizes are reevaluated at each frame).

Try the Cool VL Viewer (I reworked the texture fetcher and cache quite a bit): you won't see this issue, unless a texture got somehow corrupted (it happens, sometimes, due to network issues), at which point you still can force-reload it (right click on object and press CTRL SHIFT U).

I also implemented a feature to load textures at proper resolution faster when moving around: ”Advanced” -> ”Rendering” -> ”Textures” -> ”Boost proportional to active fetches”. And you may also ”Boost textures fetches now” manually (CTRL B), which also happens automatically on login and far TPs, to rez the world hyper-fast.

Edited March 2, 2023 by Henri Beauchamp

[Kathrine Jansma]

1 hour ago, Henri Beauchamp said:

to rez the world hyper-fast.

Try it. Especially with a good network pipe and enough cores for the texture decoding it makes visiting a some shops with lots of textures awesome. Only all the shredded meshes of avatars that need time to assemble themselves properly are a bit annoying, when all the rest of the scene appears rapidly.