Too Many 1024 Textures and the NEW (please hurry) Land Impact rules

[ChinRey]

5 hours ago, animats said:

Please lay off the text effects. It's become tiresome and pathetic.

 

5 hours ago, Klytyna said:

That was my exact thought, when I recently read a patronising suggestion...

Rey's recipe for a cure against discussing form rather than content:

1. Switch off computer.
2. Run three rounds around the block (or equivalent physical exercise - depending on physical conditon and RL environment).
3. Make cup/mug/glass/jar/barrel/truckload of favorite brew.
4. Focus 100% on savouring said brew until empty.
5. Switch on computer.

If that doesn't work, it's time for a mud wrestling match. Do we have that in SL?

Edited July 11, 2018 by ChinRey

[ChinRey]

7 hours ago, Klytyna said:

Maybe, just maybe, our time would be better spent actually trying to teach creators HOW TO BLOODY UV MAP PROPERLY.

I think that sums it up.

One of the most basic principles of programming, so basic it's all but forgotten, is

GIGO

"Garbage In, Garbage Out".

For every byte Linden Lab manages to shave off the data stream, content creators will add ten bytes of waste. For every ms they manage to shave off the frame time, content creators will add 2 ms of waste. Unless the content creators start to make more efficient content that is.

The processing needs to be as streamlined as possible of course but even so, it all starts and ends with the data. If the data isn't good enough, any improvements in the software is for nothing.

Edited July 11, 2018 by ChinRey

[Chic Aeon]

Unfortunately a lot of creators don't care to learn new more efficient (game asset) ways. 

I was in a large event a few months ago. One of the sims was completely unusable, so much so that they moved some items over to "my" sim since I had actually made a lag free sim (even ChinRey would have been happy with it except for some very heavy "flora" that was added - don't ask).  

I went over and inspected the problematic sim and saw the the issues. WAY too heavy mesh in many many pieces, high LIs and lots and lots of textures (mostly ones supplied with full perm building models and there were tons of little pieces making up the buildings). Definitely not something for an EVENT VENUE.   I explained WHY the sim wasn't working in a long detailed notecard with examples. (Honestly even I couldn't move WHEN OUTSIDE with  no one else in the sim. When inside a building it was fine, so it wasn't the event patrons that were at fault.  Anyway, no one cared. They just wanted The Lab to fix the "bad" sim (sigh). 

 

So even when information IS AVAILABLE, it often does no good. I can't really see things changing. 

[animats]

On mipmap generation, the viewer definitely has the capability. The code is in llimagegl.cpp starting around line 652. The code even supports mipmap generation on either the main CPU and inside OpenGL. It's not clear for what textures it's turned on, or how many levels of mipmap are generated. That's all conditional and the policy control for that comes from upstream.

If mipmapping isn't being used on machines with reasonably good graphics hardware, something is broken. Does anyone have a test for this in-world? Callan Merriman said she'd seen slow graphics with 1024^2 images on tiny objects. If that's happening, it's a bug. How can others reproduce that? Can you set up the test in some empty space on the beta grid?

[ChinRey]

1 hour ago, animats said:

Callan Merriman said she'd seen slow graphics with 1024^2 images on tiny objects. If that's happening, it's a bug. How can others reproduce that?

Nope. Testing with a 1024 and a 512 on the beta, zooming in and out. As long as the surface covers less than 512x512 pixel on the screen, fps is the same. The moment it covers more, fps drops significantly for the 1024 while there's no noticeable changes with the 512. I think that's clear evidence of mipmapping.

That doesn't mean Callum's experience was wrong though, mipmappig may well be a recent addition. A test I did a while ago showed quite a bit of fps difference between 512s and 1024s. I can't find the report right now though and I'm not absolutely sure the surfaces were smaller than 512x512 that time.

And of course, it's still aquestion of VRAM and download time.

Edited July 11, 2018 by ChinRey

[Klytyna]

2 hours ago, animats said:

On mipmap generation, the viewer definitely has the capability. The code is in llimagegl.cpp starting around line 652. The code even supports mipmap generation on either the main CPU and inside OpenGL.

The problem isn't MIPMAPPING.

Try and keep up.

The problem is TOO MANY textures being loaded into VRAM, resulting in texture dropping and instant reloading, this is known in SL as "Texture Thrashing".

Using auto-mipmapping to load mipmapped versions of the textures into VRAM, increases VRAM usage PER TEXTURE by 50%, making thew texture thrashing problem significantly worse.

As for it's effect on thrashing, rather than the current problem of load a sharp image, then drop it and reload a blury preview, then load the sharp image, then drop it and reload the blurry preview etc., etc., etc.

With "Animats Auto-Fubar-SL (tm)" technology, you load the FULL SIZED fuzzy preview, then load the FULL SIZED sharp image, then load a HALF SIZED mipmap with 75% less pixels for a substantial drop in quality, THEN drop the texture FASTER than before, because VRAM is full even EARLIER than before, because 50% more VRAM use per texture, then start the full size preview, full size sharp, half sized mip, drop, cycle all over again.

...

Simply thinking "Derp UE4 has mips, force SL to use them, it must improve everything" just makes the ACTUAL problem WORSE.

2 hours ago, animats said:

If mipmapping isn't being used on machines with reasonably good graphics hardware, something is broken.

Yes, your basic problem comprehension skills...

Using 50% more VRAM per texture does NOTHING to reduce the NUMBER of tetures being loaded...

NOTHING. TOTALLY BLOODY USELESS.

"Our roads are choked with too many cars, and our air is choked with pollution, we need fewer cars"

"I St. Animats of Kama City have the solution, replace all the V8 engines with V12's and add a ton of steel armour plate so the cars are bigger, heavier, cause more polution and harder to get rid of!    Hey.... Why are you all looking at me funny, it's a kewl idea, something is broken if we don't have armoured  V12's to drive around in right?

*rolls eyes*"

 

Edited July 11, 2018 by Klytyna

[animats]

49 minutes ago, ChinRey said:

Nope. Testing with a 1024 and a 512 on the beta, zooming in and out. As long as the surface covers less than 512x512 pixel on the screen, fps is the same. The moment it covers more, fps drops significantly for the 1024 while there's no noticeable changes with the 512. I think that's clear evidence of mipmapping.

That doesn't mean Callum's experience was wrong though, mipmappig may well be a recent addition. A test I did a while ago showed quite a bit of fps difference between 512s and 1024s. I can't find the report right now though and I'm not absolutely sure the surfaces were smaller than 512x512 that time.

So mip-mapping works, at least some of the time. Thanks. That should prevent frame rate decrease due to overly large textures on small or distant objects.

If it's turned on. Mip-mapping behavior may change with platform and graphics card. There are tests in the low level code for the core version of OpenGL. Tests for some Mac problem. Those look like legacy code for old problems. There  are parameters which determine how many levels of resolution to store. Somewhere upstream, there's something that decides when to use mip-mapping; the code has a "usemipmap" variable passed down through many levels, but I haven't found the policy control code yet that decides when to turn it on. It might be disabled when VRAM is close to full, but I don't know.

[ChinRey]

1 hour ago, animats said:

So mip-mapping works, at least some of the time. Thanks. That should prevent frame rate decrease due to overly large textures on small or distant objects.

Yes, but it doesn't reduce the memory or bandwidth requriements.

Chalice Yao and Vaelissa Cortes did some tests a while ago and the worst case they found was a house that used 3 GB of VRAM all on its own. Even a scene in an environment as carefully optimized as Coniston will always contain hundreds of textures, just imagine how many there wil be in an overcrowded, unoptimized scene. We're surely talking a two digit number of gigabytes and that itself is a serious problem, regardless of how well the mipmapping works.

With good clientside generated mipmapping, texture load has limited significance until we reach a certain limit. But once we're past that limit, it skyrockets. Unfortunately it seems quite common in SL for even the most powerful computers to overstep the limit so it doesn't really change anything: efficient texture management is still very important for reducing lag.

The mipmap function would explain an anomaly in a test I did earlier to determien the effect of the fitmesh LoD bug. There was a significant performance improvement as I moved the camera away from my avatar and the change was in steps, not gradual. I was wondering how that could be with the LoD models effectively disabled but I suppose the answer is that it was all a reduction in texture load due to mipmapping and not about the mesh at all.

[Coffee Pancake]

5 minutes ago, ChinRey said:

Chalice Yao and Vaelissa Cortes did some tests a while ago and the worst case they found was a house that used 3 GB of VRAM all on its own. Even a scene in an environment as carefully optimized as Coniston will always contain hundreds of textures, just imagine how many there wil be in an overcrowded, unoptimized scene.

But good luck even being able to assign 3 GB of VRAM on any viewer.. so as a worst case example it's rather irreverent. I find it more probable that this example demonstrates a bug in their VRAM accounting code. 

Edited July 11, 2018 by CoffeeDujour

[ChinRey]

40 minutes ago, CoffeeDujour said:

But good luck even being able to assign 3 GB of VRAM on any viewer.. so as a worst case example it's rather irreverent.

It's "only" 750 1024x1024 textures though.

It is a very extreme example of course but objects with 50 or even 100 MB of texture data aren't at all uncommon and you don't need many of those before yuo get to 3 GB and more.

You do have a very good point but not the apparent one. Yes, I really hope all that data isn't actually stored in VRAM but if it isn't, it'll have to be stored somewhere else on the client computer and that's bad enough.

 

57 minutes ago, CoffeeDujour said:

I find it more probable that this example demonstrates a bug in their VRAM accounting code. 

Hopefully we'll get an answer to that soon. I asked Beq about Fs' new VRAM readout yesterday. She couldn't give me any answers right away since it wasn't her code but she promised to check it out for me. Hopefully we'll hear from her soon.

Anyway, I seem to argue against myself here since I've usually been far more concerned about geometry than textures. But both are important and textures happen to be the topic for this particular thread.

[Coffee Pancake]

That situation seems more likely to end up in a crash than anything else. The viewer hacks we have in TPVs right now to get a little more VRAM are just that, hacks. They work to a point and past that tend to crash horribly (typically crashing the drivers)

With the work Kitty has done on Catznip we're able to allocate about 50% of the total reported by the card (mainly nvidia), however the windows task manager and other GPU/VRAM tools will report that significantly more is actually being used. Trying to allocate more that 50% results in terrible performance followed by a crash. This is even the case with GPUs that aren't being used by Windows as the primary display.

(I have a second nvidia card, non SLI with its own screen, Windows loads everything it can onto GPU0 and ignores GPU1's VRAM. It's possible to render opengl on the second card giving the SL client dedicated access to almost all of it's VRAM. Allocating 50% of this to SL results in 90% of it being used. Worth noting that this does allow me to run separate GPU intensive applications side by side on the same machine with no impact on performance to either .. so typically SL on GPU1 side by side with WoW or blender on GPU0. The window can be placed on either of the screens, but there is a performance impact when rending on one card and displaying it on the other .. worth also mentioning that making this setup work on Linux has so far defeated me)

An official boost to VRAM is coming from LL, but it's taking it's sweet time as LL have found it to be rather involved, made worse by many GPU's outright fibbing about the amount of VRAM present / usable.

[Chic Aeon]

3 hours ago, ChinRey said:

Anyway, I seem to argue against myself here since I've usually been far more concerned about geometry than textures. But both are important and textures happen to be the topic for this particular thread.

If I have to pick between faulty LODs where the mesh disappears OHTOOSOON  or heavy textures, I will take heavy textures every time. Still though -- it would be REALLY GREAT to know how things actually work and how we can optimize our mesh and textures accordingly. Wandering around in the dark and guessing isn't the best plan :D. 

[ChinRey]

16 hours ago, Chic Aeon said:

If I have to pick between faulty LODs where the mesh disappears OHTOOSOON  or heavy textures, I will take heavy textures every time.

Oh yes, LoD butchery is by far the worst offense a builder can commit. And since I've been so critical to the Mole's lack of understanding of LoD, they are erring on the right side, I have to give them that at least. If you don't know what you're doing, add some extra struts and braces. Excessive weight is bad but not nearly as bad as collapsing builds.

 

16 hours ago, Chic Aeon said:

Still though -- it would be REALLY GREAT to know how things actually work and how we can optimize our mesh and textures accordingly.

The two fundamental principles for optimisation are very simple and easy for anybody to understand if they want to:

1. TANSTAAFL: There Ain't No Such Thing As A Free Lunch. Every single byte of data adds to the overall load. Anything you can leave out without reducing the overall effect, is a plus.
2. Content may be King but without Queen Context he is nothning at all. That's Chin Rey's addendum to an old magazine publishers' slogan but it's really just the basic law of art as old as art itself. We never see details isolated, it's always how they relate to their surroundings. That's how the human brain works - it's all about patterns and connections. The moment we understand that and try to create scenes from components that support each other rather than fight each other, we can simplify a lot and still improve the overall look immensely.

Sadly both these principles are far too often ignored in Second Life, by the Lindens, by the content creators and by the users. But once you aknowledge them and try you best to follow them, you're doing what you can do and anything bad that happens is not your fault. And if the resource management system fails to recognise your effort or even - as is too often the case - punishes you for it, it's the system that is wrong, not you.

Edited July 12, 2018 by ChinRey

[Penny Patton]

On 7/9/2018 at 10:40 PM, CoffeeDujour said:

because it does nothing. You see a jelly doll .. but the viewer is still going to fetch all the assets, textures and stuff them into VRAM. You get a fps boost by not having to render them.

 

On 7/9/2018 at 11:33 PM, CoffeeDujour said:

I did verify this was the case before posting ... 

 

On 7/10/2018 at 12:15 AM, Callum Meriman said:

I confirm.

I did follow up on this and the texture console is backing up my observation. The texture jelly doll feature does stop grabbing assets from an avatar once it hits the cap.

Here's how you check it. First, get a viewer with the texture jelly doll feature. I'm not certain where you can do that right now but here's the Firestorm jira with the feature submission.

Once you've got that sorted, log in to SL and press cltr+shift+3 to open the texture console. This shows you information on all of the textures SL is loading.

With the texture cap on, go to any sim with people. Watch the texture console and wait for it to more or less level out. This might take a minute or two, SL throws a LOT of textures at you. But once it finally does level out, turn the cap off so all of the avatars begin popping in. Now look at the texture console, you'll see it full of activity again, fetching all those textures it did not grab before.

 That's pretty solid evidence that it does, in fact, work as advertised.

[Coffee Pancake]

Having the texture console open has a major impact on how the texture processing pipeline operates.

[Penny Patton]

Are you suggesting SL only stops downloading textures with this jelly doll feature enabled if the texture console is open? My VRAM use also shot up as all those textures were loaded in, which happens regardless of whether or not I have the texture console open.

 

The feature works.

[Penny Patton]

On 7/11/2018 at 7:17 PM, CoffeeDujour said:

But good luck even being able to assign 3 GB of VRAM on any viewer.. so as a worst case example it's rather irreverent. I find it more probable that this example demonstrates a bug in their VRAM accounting code. 

You don't need to load it all into VRAM just to know how much memory it would be. You can inspect the number of textures an object in SL uses, the size of those textures, and from there calculate how much VRAM it uses. It's basic arithmetic. 

Of course it's not going to fit it all in VRAM, but it's going to try and that's going to result in the fps hit, viewer freeze-ups, lengthy rez times, and texture thrashing you'll experience while in that house. So it's good to know how much memory things are trying to use, and keep memory use down in your own content.

[Kyrah Abattoir]

Slightly off topic, but IMHO avatars should have a fixed budget, it works for land after all.

On 7/11/2018 at 5:17 PM, Chic Aeon said:

Unfortunately a lot of creators don't care to learn new more efficient (game asset) ways. 

I was in a large event a few months ago. One of the sims was completely unusable, so much so that they moved some items over to "my" sim since I had actually made a lag free sim (even ChinRey would have been happy with it except for some very heavy "flora" that was added - don't ask).  

I went over and inspected the problematic sim and saw the the issues. WAY too heavy mesh in many many pieces, high LIs and lots and lots of textures (mostly ones supplied with full perm building models and there were tons of little pieces making up the buildings). Definitely not something for an EVENT VENUE.   I explained WHY the sim wasn't working in a long detailed notecard with examples. (Honestly even I couldn't move WHEN OUTSIDE with  no one else in the sim. When inside a building it was fine, so it wasn't the event patrons that were at fault.  Anyway, no one cared. They just wanted The Lab to fix the "bad" sim (sigh). 

 

So even when information IS AVAILABLE, it often does no good. I can't really see things changing. 

And they don't care because it doesn't affect them in any way. Creators generally don't eat their own dog food, if they even spend time in SL at all.

I'm not even sure there is a way to convince them to be more mindful because the way they do things right now is the most efficient possible way to release a product with the least amount of time spent on it.

Doing any kind of optimization costs time, and that time is better spent working on a new product (or simply not working at all).

People who see SL strictly as a business are not interested in spending time on things that doesn't translate into more income. I wish i knew of a compelling argument that would justify "hey you should probably spend an extra week building a low poly model and packing your texture UVs", but I haven't found any way so far.

People usually agree with me on this because they genuinly want to make quality game grade assets, or they just laugh at me, calling me a dreamer because I literally suggested to spend more time making less money.

Edited July 18, 2018 by Kyrah Abattoir

[Fionalein]

38 minutes ago, Kyrah Abattoir said:

Slightly off topic, but IMHO avatars should have a fixed budget, it works for land after all.

They kinda have - unless part those special communities who have prim attachements cheaper the dozen - there is always peer pressure. Who wants to be a jelly doll after all?

[Kyrah Abattoir]

I recall when the feature was implemented what people wanted to know the most was "how do i disable that notification that I might not be rendered by everyone cause I don't really care".

[Fionalein]

They will have to care - there are security orbs now which kick you for a too high avatar load...

Edited July 18, 2018 by Fionalein

[Coffee Pancake]

There is no way from LSL to guage an avatar render cost, those security orbs are based on script memory usage and responsible for hammering regions into the floor. Thankfully their use isn't fashionable anymore.

 

[ChinRey]

2 hours ago, CoffeeDujour said:

There is no way from LSL to guage an avatar render cost

That's true but the OBJECT_RENDER_WEIGHT parameter of the llGetObjectDetails() function pretends to do it.

I was the first to write a commercial render cost based avatar lag meter script and Hattie Panacek made a really cool atompunk styled container for it. It sold quite well but we withdrew it from the market after a week or two because it turned that the numbers it came up with had no relation whatsoever to any known or theoretically possible reality.

That hasn't stopped others from launching similar products based on the same (dys)function though and they are occasionally used to harrass innocent visitors the same way as the old style script based lag meters.

Edited July 18, 2018 by ChinRey

[Coffee Pancake]

Well, if anywhere I frequent deploys such a thing, I will simply move on somewhere else.

Scanners that eject avatars based on arbitrary and poorly understood numbers should be against the ToS, they cause more problems than they solve.

[ChinRey]

11 minutes ago, CoffeeDujour said:

Scanners that eject avatars based on arbitrary and poorly understood numbers should be against the ToS, they cause more problems than they solve.

You can say that again but if you expect anybody to listen, you'll be sorry.

As for Linden Lab, they couldn't care less. I did actually file a JRIA about it but LL showed no itnerest in the matter whatsoever.

But since I had to waste time doing LL's job before I filed that JIRA, I might as well post a little about it here for the few who are interested. In addition to all the flaws of the ARC system itself we are all familiar with (and the ones that aren't quite as well known yet) the lsl flavour has an additional quirk all of its own. The values are not calculated serverside. What happens is that the server requests data from all the clients in the same region and the neighbor regions (presumably to find some use for all that spare bandnwidth we have) and then it presents the average of them all. That means avatars more than 4,000 m away from you may influence the number - and influence it a lot. This is not jsut a theoretical possibiity, it's exactly what happened during my final test before I filed the JIRA. I was all alone in a corner of a premium sandbox trying to figure out why the ARC meter gav the correct reading there but not at my home in Coniston. Suddenly somebody tp'ed into the neighbor sandbox in the corner furthest away from where I was and high in the air, and the ARC meter's reading dropped to a half. Case solved. i wrapped it up, filed a JIRA and waited for LL to do something. When they chose to do nothing, we withdrew the meter from the market since neither me nor Hattie are the kind of people who like to fool out customers. I think it took almost a year before the kind of people who do like to fool their customers caught on and started foisting dysfunctional ARC lag meters onto unsuspecting buyers.

The experience taught me two things: Honesty doesn't sell in Second Life and never trust a Linden's competence - always check and double check the facts yourself.