How LL Can Achieve Higher/More Stable FPS in Second Life

[Penny Patton]

A few people have asked about this in the "A Conversation with Linden Lab CEO Ebbe Altberg" thread so rather than derailing there, I'm making a new thread here to give people some insight into this issue.

To get a higher and more stable framerate in SL would require LL to really discourage poorly optimized content in SL while also providing tools to encourage better optimized content, and make it easier for content creators to optimize their work. As well as tools for consumers to make more educated purchasing decisions, and disincentives for purchasing content that uses excessive amounts of resources.

For example, excessive texture use is far and away the biggest framerate killer in Second Life right now. Based on months of observation using a viewer used to demonstrate some VRAM management tools, I'd say that your average avatar is walking around with over 200MB worth of textures. Avatars exceeding 500MB of textures are not as uncommon as you'd think. I've regularly seen avatars pushing 800MB to a GB of textures.That's one full gigabyte of data that you have to download and hold in your graphics memory, for one single avatar.

 There are sims out there using over 10GB of memory just on textures alone. Why do people do this? Because there's absolutely nothing to discourage them from doing it except the performance hit, and too few people know how this issue affects them so they blame it on bad viewer code or some other strawman.

 How much VRAM does your videocard have? Even if you have an expensive, high end gaming card you probably only have 8GB of memory. Chances are, most of us have much less than that. Also, VRAM isn't just for textures. Shadows, DoF effects, light effects and more all need VRAM. This makes the "just buy a better videocard" argument unhelpful, to put it politely.

 And there are people who will argue "but SL doesn't load all those textures at full resolution all at once!" ignoring that as you move and cam around SL you're hardware is constantly sifting textures in and out of data. Constantly moving that much data in and out of memory is a huge burden on both your videocard and your bandwidth.

 Go to a sim built by someone who understands VRAM issues and you could easily find yourself experiencing double or triple the FPS you typically experience in Second Life. At least until one of those 1GB avatars wanders into view. In some cases, after optimizing an environment I've found myself enjoying FIVE TIMES the fps compared to what I saw before optimizing texture use. That fact alone should be enough to prove the point.

You want higher and more stable FPS? LL needs to step up and start discouraging excessive resource use in content. Giving users a tool to derender avatars with excessive VRAM use. Tools that let users see the VRAM use of all of their attachments and rezzed objects. LL needs to start including texture use in Land Impact calculations, so that objects with excessive VRAM use cost far too much LI for anyone to want to use. LL needs to add texture memory to the information marketplace sellers can display in their listings, so people can make informed choices and start avoiding content with high memory use, or content which fails to list memory use (just like people tend to avoid products that fail to list land impact cost). Give people tools and all of this can be super easy, for both content creators and the people who buy that content.

 There's good, easy to learn tricks to reducing memory use but LL doesn't make it available through any official channels. There's no official SL building tutorials, and the tutorials out there are made by other residents, typically those who do not have any experience or education regarding game design.

LL needs to improve the "jelly doll" feature so the resources used by avatars get released when they're jelly-dolled. Right now their textures, at least those that loaded before the avatar gets derendered, remain in VRAM.

 And that's just textures. Avatar attachments, not limited by LI costs, tend to be far too high-poly. Some of the mesh bodies out there are absolutely ridiculous. There's tricks people use to get around reasonable LI and draw weight costs (avoid any content where the creator recommends you increase your LOD threshold), LL needs to close those loopholes and rework LI and draw weight to be more useful. To better encourage good content creation habits, and better discourage the bad habits.

It's very important to note that discouraging bad habits does not mean reigning in creative freedom, or making content creation more difficult. A lot of people hear what I've posted above and are terrified that if LL did these things it would drive them out of SL. That's simply not the case. Optimizing is easy! It's just a matter of learning where you need extra texture detail, where you can get away with less, and learning not to create textures that are filled with unused space. Tell people new limits are coming, give them the tools to work within those limits, the time to learn to use those tools, and it will not be a problem.

According to LL some of these tools are coming! That's a very good sign that in the future we may very well experience a Second Life where higher framerates are the norm. If LL handles this right they could even increase the maximum texture resolution cap in SL.

[AnimalChi]

10 hours ago, Penny Patton said:

 

As well as tools for consumers to make more educated purchasing decisions, and disincentives for purchasing content that uses excessive amounts of resources.

 

You know how many times I've bought a hair and then realized it was WAY too much render cost due to high complexity and texture misuse!? 

I hope my snapshots and bad reviews of their SINGLE item (the hair) adding 100,000 render cost to my avatar is discouraging enough, but you know it's not. lolol.

A way to see how complex an item is BEFORE we buy it would be nice. (for stuff that doesn't include demos as well.)

[Fionalein]

10 minutes ago, AnimalChi said:

You know how many times I've bought a hair and then realized it was WAY too much render cost due to high complexity and texture misuse!? 

I hope my snapshots and bad reviews of their SINGLE item (the hair) adding 100,000 render cost to my avatar is discouraging enough, but you know it's not. lolol.

A way to see how complex an item is BEFORE we buy it would be nice. (for stuff that doesn't include demos as well.)

Some hair shops write that in the product info...

[Bree Giffen]

Yeah when you look at how game studios work with the modelers and artists they devote a huge amount of time in optimizing the models and textures. In SL the norm is high poly this and that with ipad retina levels of pixel density. Hopefully there can be improvements in optimizing creators output. Doesn't LL already do this in Sansar? Maybe move that workflow into Second Life? 

[Callum Meriman]

15 hours ago, Penny Patton said:

For example, excessive texture use is far and away the biggest framerate killer in Second Life right now.

As per usual I really expect the coming Fantasy Faire to be chock-full of 1024x1024 vendor textures as it is every year, with the resulting crashes, 1fps, and lag.

On another related point, I was debugging someone's laggy home the other day and came across a small sideboard with 24 x 1024x1024 textures. 24!

[Penny Patton]

All true. What's more, you go to any game modding community or talk to any game artist and they will tell you the same things. Memory management is important.

I always say that the best thing about Second Life is that anyone can come in and start making content, without needing years of art school or any sort of professional background. You can just pick it up and start creating. I love that about SL. However, with that in mind, LL needed to develop tools that would naturally guide people towards better content creation habits and they failed to do that.

 The result is that many content creators don't believe excessive use of texture memory has any effect on performance, or that it only effects people on severely underpowered hardware. Even as they complain about the lag and performance issues they experience but chalk their experiences up to the SL viewer code being awful and leave it at that.

Edited April 15, 2018 by Penny Patton

[cykarushb]

SL's performance problems have nothing to do with textures, believe me, i run this game on a GPU with 768mb of video memory, and ive run it on way less than that even. Textures dont work that way with a video card. Video memory is a frame buffer, the GPU renders the scene in frames, sends the frame to the frame buffer, before its displayed on your screen. The larger and more detailed the frame, the more memory it uses. So a 1080p, high color, highly detailed image (such as a giant rainbow gradient or something) will take up a lot of video memory, a flat color screen will take up almost none.

Textures are not stored in vram, they are cached elsewhere and the GPU takes them directly from the cache.

The reason you get low performance in SL has very little to do with unoptimized user created content, thats definitely a problem, but modern hardware is up to the task of displaying the absolute mess that is the average sandbox. The real performance killer with SL is the single threaded rendering of lighting, shadows, and particles. They are CPU bound tasks, and they are limited to one thread on a processor. Its sort of a legacy thing that has stuck around since a single core processor was the norm, those graphical features got more advanced but the rendering programming behind them did not.

The better single threaded performance you have, the better your framemate will be, because even a current mid tier processor can be a bottleneck to a GPU if the single thread/core its using is being maxed out, so while the GPU can easily do its job, the CPU cannot.

My best performance on SL was on my old gaming PC with an i7 4770k. Even after i sold the GPU and it had a mere 750ti (a woefully underpowered card to pair with a 4770k), i was netting nearly the same performance as i was with my GTX Titan black. The Titan was on a whole other level compared to the 750ti, but i was still getting my usual 70-80fps on higher settings with people around in 1080p. Maybe a bit less, but less than 5fps was lost. And i remember at the time i was just mind blown, every other game out there when i upgraded FROM the 750ti to the Titan originally i was seeing 2-3x the framerate i had before, SL was a weird outlier.

And its simply because of the processor, if SL was able to utilize more cores and threads properly, im willing to bet that 4770k+ Titan Black would've gotten me triple digit framerates, the card was hardly being used.

And the same thing is going on now with my poorlad machine (had to sell off my nicer PC and hardware after i lost my old job), with its Core2Quad Q6600 and various GPU's ive tested, the CPU is a bottleneck, i get almost the same performance whether using a Quadro from 2006, a GTX 465, Radeon HD 4870x2, GT 720, even borrowed a GTX 970 from a friend and saw maybe a 20fps increase. The Q6600 is old, and while its still fairly usable in 2018, not with just one core being utilized fully. Im pretty much relying on multicore usage for games because thats the only thing saving the Q6600 now after 12 years, if a game can use all four cores, it can still hold its own, but SL wont use more than 1 so im seeing a CPU bottleneck.

Whats hilarious is i previously tested out SL on the legacy viewer on XP using a 2.2ghz Pentium 4 and a GeForce 6200 and got it to run, and besides being kind of laggy on my normal settings, if i dropped it down low, it was still playable. And not one of those 775 p4's, a proper northwood one from 2002 was still capable of playing SL.

 

tl;dr, no. problem is a poorly utilized CPU

[Lillith Hapmouche]

... and my two cents: it's not a question of either and or. It is rather a combination of both, poor utilization and poorly created content.

[Penny Patton]

10 hours ago, cykarushb said:

SL's performance problems have nothing to do with textures, believe me

I can go point by point explaining why almost everything you've posted is incorrect but you don't have to take my word for it. So instead I'll just point you at some videos that break it down in an easy to understand way. Videos like this one. And this one.

And look, here's the simple plain truth of the matter: reducing the texture load and otherwise optimizing content in SL increases framerates substantially and consistently. This isn't an untested theory, this is well established fact. Optimizing content gets you a huge performance boost every time. And not just framerates. Things load a lot faster, too. Downloading a gig of textures doesn't take nearly as long as downloading 10 GB of textures.

Like Lillith says, there are multiple issues with SL which, if addressed, could lead to better performance in SL. Yes, SL could utilize the CPU better. But denying the impact of unoptimized content is indefensible.

[So Whimsy]

11 hours ago, cykarushb said:

SL's performance problems have nothing to do with textures, believe me

I'd rather not as it has been proven time and time again that textures do indeed affect performance, and that's a fact.
When you try to get people to believe you, you should be 100% sure with facts backing you up 100%. Because now? I won't believe a future word you say.

[Solar Legion]

49 minutes ago, Syn Anatine said:

I'd rather not as it has been proven time and time again that textures do indeed affect performance, and that's a fact.
When you try to get people to believe you, you should be 100% sure with facts backing you up 100%. Because now? I won't believe a future word you say.

How about you try reading elsewhere on the forum before jumping to conclusions on such a complex issue?

Seriously though, texture memory/optimization is only part of the problem with Second Life.

Edited April 15, 2018 by Solar Legion

[Penny Patton]

5 minutes ago, Solar Legion said:

How about you try reading elsewhere on the forum before jumping to conclusions on such a complex issue?

Seriously though, texture memory/optimization is only part of the problem with Second Life.

Yes, optimization is only part of the problem. A major part, and one where you can see substantial performance gains by addressing, but only part.  Syn did not say otherwise.

You know who said otherwise? cykarushb, when they claimed the lack of texture optimization is not a factor at all. And that is what Syn was responding to.

[So Whimsy]

18 minutes ago, Solar Legion said:

How about you try reading elsewhere on the forum before jumping to conclusions on such a complex issue?

Seriously though, texture memory/optimization is only part of the problem with Second Life.

Thank you for verifying my post as correct!

What I was responding to was the complete denial of the poster I quoted.

 

Perhaps how about you try to read my actual post and the one that was quoted before responding?

[Solar Legion]

3 minutes ago, Penny Patton said:

Yes, optimization is only part of the problem. A major part, and one where you can see substantial performance gains by addressing, but only part.  Syn did not say otherwise.

You know who said otherwise? cykarushb, when they claimed the lack of texture optimization is not a factor at all. And that is what Syn was responding to.

That's nice Penny. I don't really care what wording was used.

The entire point was that there are far more factors to this - period.

 

[Solar Legion]

Just now, Syn Anatine said:

Thank you for verifying my post as correct!

What I was responding to was the complete denial of the poster I quoted.

 

Perhaps how about you try to read my actual post and the one that was quoted before responding?

I have verified nothing.

Perhaps you should read the entire post, then the entire linked thread, then focus on the data presented instead of choice of wording.

You know, what someone actually interested in facts would do?

[Solar Legion]

Before either of you respond further, do remember which section of the forum this is.

If you're looking to debate the technical side of things, there's a section just for that. Use it.

Edited April 15, 2018 by Solar Legion
Added a link to the Technical Discussion section

[So Whimsy]

Just now, Solar Legion said:

I have verified nothing.

Okay.

 

12 hours ago, cykarushb said:

SL's performance problems have nothing to do with textures

 

1 hour ago, Syn Anatine said:

textures do indeed affect performance

 

23 minutes ago, Solar Legion said:

texture memory/optimization is only part of the problem with Second Life.

What exactly is your problem here or are you just being confrontational for the sake of it?

Let me rehash it for you so you can get off your high horse. I was solemnly responding to the quoted post above that completely denies textures having any impact on performance. I told him they do. You proceeded to say they do, too.

This isn't about intrinsic details of how much or how little they impact performance. It's about that they do impact the performance which the poster I quoted denied. Are we done yet?

[Solar Legion]

1 minute ago, Syn Anatine said:

Okay.

 

 

 

What exactly is your problem here or are you just being confrontational for the sake of it?

Let me rehash it for you so you can get off your high horse. I was solemnly responding to the quoted post above that completely denies textures having any impact on performance. I told him they do. You proceeded to say they do, too.

This isn't about intrinsic details of how much or how little they impact performance. It's about that they do impact the performance which the poster I quoted denied. Are we done yet?

You dismissed an entire post based on one part you disagreed with.

You've shown yourself to be too fixated on the single section you disagreed with to consider any of the information presented. You did this in your reply to Cykar.

You threw the baby out with the bath water and seem quite proud of it.

Yep, we're done.

[animats]

Large textures alone shouldn't choke SL. The viewer reduces the resolution of textures to fit them into the available texture space of the graphics card. It also reduces them when the object is not taking enough screen space to need that much texture. It's the viewer's job to manage this, not the object creator's. The viewer knows how close the camera is to the object; the creator does not.

SL's big rendering problem is weak level of detail processing. All the viewer really does is pick which level of detail to display based on distance, and clip the world at the draw distance. That's simple but way below optimal.

SL needs automatic low level of detail model generation. Expecting people to do a good job by hand isn't working. It's too much work, and it's better done by a program anyway. There are reasonably good automatic systems for that now. (Better than the "decimate" feature in Blender.)

SL also needs automatic imposter generation. Distant objects need to turn into a bounding box or a cross with a render of the object painted on it. Most game engines have that today. Higher level imposters are needed. Buildings and all their contents should be replaced with a big imposter when far away. There's no way to do this in SL, even by hand. Entire sims off in the distance should be replaced with big impostors, so you can see the mountains and buildings in the distance. SL's frame rate problems are mostly due to too much rendering effort going into distant objects.

Almost all AAA video games today have all of this. SL is behind.

It's the job of the viewer to manage frame rate. LL is looking at the rendering pipeline as part of Project Arctan. Maybe things will improve. There's certainly lots of headroom for improvement viewer side.

[Penny Patton]

7 minutes ago, animats said:

Large textures alone shouldn't choke SL. The viewer reduces the resolution of textures to fit them into the available texture space of the graphics card. It also reduces them when the object is not taking enough screen space to need that much texture. It's the viewer's job to manage this, not the object creator's.

Partially correct. Yes, the viewer does stream lower resolution versions of textures that are some distance from the camera, and when your VRAM is overburdened SL will often flush textures and attempt to reload them at a lower resolution. However, this is not a magic wand. If you try to rely on this as an alternative to proper texture optimization, SL will constantly be flushing and reloading textures as there is simply not enough memory to hold them all. This is what is known as "texture thrashing", a regular problem in Second Life.

A smart creator is going to try and minimize the memory use of their content as much as they can without sacrificing quality. A smart landowner is going to try and use the most optimized content they can find to prevent texture thrashing and the framerate hit you'll take  from having too much texture data in a scene.

13 minutes ago, animats said:

SL needs automatic low level of detail model generation. Expecting people to do a good job by hand isn't working. It's too much work, and it's better done by a program anyway. There are reasonably good automatic systems for that now. (Better than the "decimate" feature in Blender.)

SL also needs automatic imposter generation. Distant objects need to turn into a bounding box or a cross with a render of the object painted on it. Most game engines have that today. Higher level imposters are needed. Buildings and all their contents should be replaced with a big imposter when far away. There's no way to do this in SL, even by hand. Entire sims off in the distance should be replaced with big impostors, so you can see the mountains and buildings in the distance. SL's frame rate problems are mostly due to too much rendering effort going into distant objects.

Yes, these are things SL needs and they would help performance. However, having these features would not negate the need for proper optimization. You're still going to have all those textures clogging VRAM, resulting in texture thrashing, low framerates, and substantial bandwidth use.

You can't say "well, I optimized this one aspect of my art assets, so I can go totally bonkers with this othe aspect!" It doesn't work that way.

16 minutes ago, animats said:

Almost all AAA video games today have all of this. SL is behind.

All AAA videogames today put a lot of effort into optmizing their art assets. They don't use unoptmized Zbrush models with billions of triangles. They use cleaner, lower-poly models that are easier to render. They don't go insane with wastefully bloated textures. They do their best to waste as little of a texture map as they can, and to fit as much detail into as little memory as they can manage. Because it's important for performance.

And again, and I really cannot stress this enough, I have been walking the walk on this for years. I put hours into optimizing the sims I build, reducing the texture load and everything else as much as I can. The result is always much higher framerates. I get messages from strangers, gushing about how high their framerates are in the sims I build. I recently began using a viewer a friend of mine put out to showcase a tool that lets you derender avatars with high texture use. With this tool active I again see a massive FPS boost. Sometimes just by derendering a single problem avatar with an excessive amount of textures.

 So I put to you: If crazy amounts of texture data have no impact on framerates, how is it that so many people see a substantial fps increase when I optimize texture use in my builds? Because you seem to be arguing that I am a wizard, using mystical powers beyond the understanding of mortals, to increase the framerates of everyone around me. And even those not around me, but who have decided to optimize content on their own. And my powers are not limited to SL, I've managed to trick the entire professional game design industry with my sorceress powers. The fools! They waste so much time and effort optimizing their art assets and finding new ways to manage VRAM use, little do they know I alone control the framerates!

[cykarushb]

Your GPU does not cache textures, no modern video game does this. They are taken by the GPU from the games cache, sometimes that's held in RAM or from a drive. Your GPU is not displaying to you a full high resolution texture at any time. If there's a poster in the distance using a 1280x1280 picture, it's not magically going to cram 1.6 million pixels into something the size of 2 square inches on your screen. It's taking that image, compressing it down to size to be relative to the distance you see it at, and then rendering the frame.

Every frame rendered in SL is going to be pretty much the same size unless you're looking at lots of flat colors or gradients, like the sky.

If a texture is too big and is taking a while to load, the GPU doesn't care, it just won't load that texture until it's properly cached. You just get gray/pink objects. And that can be mitigated by having any decent modern internet connection and anything faster than a 4200rpm hard drive.

Stick this game on an SSD and set the cache there, stuff loads near instantly on my humble 10mbps download speeds.

I think I'm gonna make that one of my next tests, go down in graphics card video memory until SL can't be played.

though, as I posted a while back in general tech discussion, a 256mb GeForce 6200 was still capable of graphically displaying this game 

i wonder how it would pair with the core2quad, maybe it was the 16 year old pentium 4 holding it back 

[Penny Patton]

1 minute ago, cykarushb said:

They are taken by the GPU from the games cache, sometimes that's held in RAM or from a drive.

Yes, it's read from the drive and held in RAM. Video RAM. Also known as VRAM.

4 minutes ago, cykarushb said:

though, as I posted a while back in general tech discussion, a 256mb GeForce 6200 was still capable of graphically displaying this game 

Yes, and you had to drop the graphics considerably to make that work. SL will run on an impressive broad range of hardware, but when you get to the lower end you really need to sacrifice the fidelity to make it work.

Meanwhile, people come to my optimized builds and find they can run SL at higher graphics settings than they can manage anywhere else. Explain that to me. Explain why people experience higher framerates in SL when textures are optimized if memory load and performance are unrelated. Why do they see performance take a hit when an avatar wearing a gigabyte of textures lumbers into the scene. These are things that happen. Explain why, if texture optimization and performance have no correlation.

Explain why high-res texture packs for games come with a performance hit if the amount of textures being rendered is wholly divorced from performance like you claim.

Explain why game devs and modding communities talk about optimizing textures for performance if there's no correlation. 

Why does this article from Tom's Hardware describe what VRAM for like this:

• Loading textures
• Holding the frame buffer
• Holding the depth buffer ("Z Buffer")
• Holding other assets that are required to render a frame (shadow maps, etc.)

Why is it that I can point to all these sources, why is it that I have been successfully demonstrating my claims in action in SL for years now, and why is Linden Lab themselves so keen on introducing tools to discourage unoptimized content, if there's truly no correlation?

Speaking of, you have sources to back up your claim I assume? Give me some reading material. Show me a video. If I'm wrong, educate me with your sources. But then I still want an explanation as to why optimizing content guarantees a performance boost, if content and performance are unrelated. That's a huge coincidence that framerates go up whenever content is optimized. Kind of scary, really, if they're unrelated as you say. It really cries out for an explanation.