Automatic adaptive LOD

[Selc]

Wondering if it's possible for devs to implement an adaptive LOD feature that automatically and seamlessly reduces the polycount of avatars and static objects at different zoom levels or different graphics settings. Something that enables the user to override the current four static levels of manual LOD for mesh. This should also be enabled by default.

The complexity update did nothing to fix user framerates so that VR would be viable with SL, because currently every mesh creator exploits the unfixed complexity loophole that allows for low complexity but super highpoly avatars (everyone is guilty, including me and my hair team). Even the highest LOD for the majority of rigged mesh in SL will be rendered at the lowest graphics setting when zoomed out — it's everyone's fault.

We really want to see Second Life thrive again, even if it's 17 years old and long shunned by anyone not banned from Chuck E. Cheese's.

And everyone wants to see VR re-implemented.

My techie friend helped me benchmark some framerates with his RTX 2080 Ti... 200+ fps with three 40k-poly avatars (normal maps generated from highpoly models) in a beautiful mesh house on a quiet private island. More than enough fps for VR. But he gets only 6-15 fps in a crowded club full of highpoly mesh avatars, even with max # of non-impostor avatars set to 1 and maximum complexity set to 20,000. That would probably translate to only 2-7 fps in VR without adaptive LOD. The impostor avatars don't hinder the calculation of polygons. Although 26,000,000+ polygons in the clubroom with 30-40 avatars aren't being rendered at full resolution, they're still being calculated (enough for 433 fully-detailed Playstation 4 characters on your screen).

Automatic adaptive/customizable adaptive LOD is a feature that could retroactively fix all of these issues, make framerates viable again for VR, and significantly boost the fps on your average resident's laptop that currently runs SL like a flipbook in crowded areas.

 

Average SL avatar today at over half a million polys

 

 

 Example:

 

Edited March 18, 2020 by Selc

[NiranV Dean]

Possible, yes. Simplygon from Unreal Engine is a similar solution for instance. The question is not whether its possible but who's going to do it. Besides, it wouldn't fix the obvious issue of overpolgonation, you'd have to set the ranges for when models start to lose details so tiny that basically everything would immediatly become broken. Remember that automatic solutions for LOD are problematic on complex meshes, for simpler stuff this is perfect but more complex stuff like plants and houses, this would cause a lot of false triangles and the items in question seemingly exploding/imploding. We'd need a flag to set which item should use said system and which shouldn't.

Edited March 18, 2020 by NiranV Dean

[animats]

SL has a long series of problems in the LOD area. One big one is all the assets with lower LODs created by the terrible mesh reducer in the uploader. The one that generates meshes with holes if pushed too hard. Then, the standard LI formula rewards extremely sparse lowest LODs. The combination of these problems is why you see so many things that show some random triangles at lowest LOD. Makes SL look junky.

This spills over into clothing. Much SL clothing partly disappears at the lower LODs. So, as a hack to fix this, all avatar clothing and attachment switch LODs at the same time, regardless of size. A highly detailed bracelet doesn't drop to a lower LOD until the entire avatar does. So avatars tens of meters away are still being rendered at full detail, even though the detail on the bracelet is now smaller than a pixel. This is the cause of much overload in busy sims.

There are better mesh reduction algorithms. They're great for weapons, vehicles, buildings, and terrain objects. Clothing, not so much. I tried using Blender's quadric mesh optimization on a hoodie for SL. It moved the edges of the fabric, rather than flattening out the wrinkles. That's because quadric optimization tries to minimize the volume error between the original and reduced mesh. For a very thin object, that goal leads to pulling the edges inward, not flattening a bump.

This problem is not seen in most games, because they don't have layered clothing that's not part of the character. So SL needs a solution that works for SL clothing. Mesh reduction for clothing probably needs to be done with a tool that knows it is working on clothing, and comprehends thin sheets. I'm not a clothing designer, and the clothing community needs to address that one.

Automatic LODs for buildings, terrain, vehicles, etc. should be do-able with the same technologies used in games. SL clothing, though, is going to be tough.

We do have avatar impostors. Set "number of visible avatars" to 3, and only the 3 nearest avatars are rendered fully. The remaining ones are only rendered four times a second, and shown as a billboard impostor, a flat picture. This is effective, but has display problems with some avatars and elaborate attachments, where the bounds of the object are difficult to determine.

We don't have general object impostors. There really should be a way to represent an object as a billboard impostor, a set of flat pictures where the picture shown is changed depending on the viewpoint. It's possible to fake this in-world, for one avatar at a time, with a flat object that rotates to face the avatar and changes the texture. This looks pretty good beyond 20 meters or so. I used to have an "impostor garden" in-world to demonstrate this, and many creators visited. Your lowest LOD would have only two triangles. Cheap and fast to draw. Downloading the pictures, though, takes network time. SL really ought to support this. Almost every game does. That's why you can see so far in many games.

I'd like to see impostoring taken to the max. Distant sims should be shown as sculpt-like objects with map-like images painted on them. That's what you should see beyond draw range instead of water. This could work like the SL map; have such objects for each sim, each 4 sims, each 16 sims, etc, used as you get further away. See for kilometers. With a tiny bit of blur and fog, you won't see the lack of detail out there. It would make SL feel much bigger. SL is a big world, but it feels cramped, because you can't see very far.

So, known technology could fix the drawing overload for general objects, but SL clothing presents unique problems.

[NiranV Dean]

45 minutes ago, animats said:

This spills over into clothing. Much SL clothing partly disappears at the lower LODs. So, as a hack to fix this, all avatar clothing and attachment switch LODs at the same time, regardless of size. A highly detailed bracelet doesn't drop to a lower LOD until the entire avatar does. So avatars tens of meters away are still being rendered at full detail, even though the detail on the bracelet is now smaller than a pixel. This is the cause of much overload in busy sims.

This is not a hack, its a bug and it has been known for a long time. It's partly due to bounding box issues, which is again mostly the creators fault, they create things that have huge bounding boxes because they don't know the technical aspect of SL. Couple bad meshes with rigging and the bounding box goes haywire at times.

The "dick" issue was one of those things, theres a certain brand that has animesh genitals and since they are rigged, they become a full "avatar", which shows in its bounding box being one as well which made them huge. SL has a lot of issues but the biggest isn't the client but the content that isn't using the tools given properly.

[Theresa Tennyson]

13 hours ago, Selc said:

 

A Catwa Jessica is a pre-Bento head that uses face-swapping to animate and therefore has far more polygons than a modern head.

If you're testing a Maitreya body with removable hands and feet it would be version 4 or earlier, meaning it has 3 permanently attached full-body shells that are no longer part of Lara version 5.

I'm not saying you don't have a point, but if you're basing your argument on things that people know aren't current you'll be in the position of a pollution activist who gives examples of the pollution produced by a car by sticking a test probe into the exhaust pipe of a Studebaker. They'll get alarming numbers, all right, but any careful observer will see that those numbers are no longer typical.

[Solar Legion]

Sorry, not "everyone" wants VR in Second Life nor does "everyone" want Second Life to be capable of VR.

You want it - that is the beginning and end of how broad a statement you can make on that and have it still be true.

[Jackson Redstar]

I gotta say I really don't understand alot this from a technical perspective, but I am getting real frustrated lately in SL I have a fairly robust computer, i7 3.8ghz, a Gtx 1080, 16 GIG RAM, I video wedding so de-rendering people isn't an option, and graphics need to be fully on, and it is often times a complete mess. Its getting harder and harder to get quality footage unfortunately. FS uses a checkbox to  'dynamically adjust level of detail' and often when filming a wedding, that does need to be un-checked so stuff wont un-render.

What i would find very useful is a rendering prioritization - that particular avis can be selected full full render, never impostor them, every one else you can impostor. As it is now you can set the max# of non imposer avis, but sometimes a bride or groom or other wedding party member might end up as a imposer even if I select them to always render fully. I guess I'm more of a specialty case so would be pretty unlikely that would ever get implemented

[Selc]

My friend just ran another test:

1. He set max # of non-impostor avatars to 1 and set maximum complexity to 20,000
2. Went to a different club to see his RTX 2080 Ti calculating 18,000,000+ polygons with 36 people -- the result was 11 fps
3. Blocked and derendered the other 35 people in the club
4. 18,000,000+ polygons were still being calculated at 11 fps although no avatars were being rendered on-screen (everyone was invisible)
5. Disabled camera constraints and zoomed all the way out into the sky, looking up
6. His fps instantly shot back up to 150+
7. Pressed Escape to zoom back in
8. 18,000,000+ polygons were still being calculated although no avatars were being rendered on-screen, and he plunged all the way back down to 11 fps

I experience similar results when I run the same tests. We've tried both the default viewer and Firestorm. Proof avatar imposters don't do anything, even for the latest gaming computers.

Edited March 19, 2020 by Selc

[NiranV Dean]

22 minutes ago, Selc said:

My friend just ran another test:

1. He set max # of non-impostor avatars to 1 and set maximum complexity to 20,000
2. Went to a different club to see his RTX 2080 Ti calculating 18,000,000+ polygons with 36 people -- the result was 11 fps
3. Blocked and derendered the other 35 people in the club
4. 18,000,000+ polygons were still being calculated at 11 fps although no avatars were being rendered on-screen (everyone was invisible)
5. Disabled camera constraints and zoomed all the way out into the sky, looking up
6. His fps instantly shot back up to 150+
7. Pressed Escape to zoom back in
8. 18,000,000+ polygons were still being calculated although no avatars were being rendered on-screen, and he plunged all the way back down to 11 fps

I experience similar results when I run the same tests. We've tried both the default viewer and Firestorm. Proof avatar imposters don't do anything, even for the latest gaming computers.

Not a proof for anything really.

The difference between 35 impostors and 0 impostors is barely usable 11 FPS to unusable 3-4 FPS.

Impostors are not meant as an all-glory super solution, avatars are still being rendered but they are being updated less often. If you want performance, push jellydolls ontop.

I see drastic differences, from 10-15 FPS with no impostors and no jellydolls, to ~30-40 FPS with impostors but no jellydolls up to 50-60 FPS with impostors and jellydolls.

Even the best system simply reaches its cap at some point, no delaying updates of avatars is going to fix this, as long as you still render them you are incurring their performance impact in some way or another.

Edited March 19, 2020 by NiranV Dean

[Wulfie Reanimator]

11 hours ago, Jackson Redstar said:

What i would find very useful is a rendering prioritization - that particular avis can be selected full full render, never impostor them, every one else you can impostor. As it is now you can set the max# of non imposer avis, but sometimes a bride or groom or other wedding party member might end up as a imposer even if I select them to always render fully. I guess I'm more of a specialty case so would be pretty unlikely that would ever get implemented

(Assuming you're using the pie menu, I'm old and stuck in my ways.) Right-click avatar > More > Render > Always...

Edit: I didn't read the whole bit, nice. That's the first time I've seen the feature not working and I have no clue why that might be.

Edited March 19, 2020 by Wulfie Reanimator

[NiranV Dean]

1 hour ago, Wulfie Reanimator said:

(Assuming you're using the pie menu, I'm old and stuck in my ways.) Right-click avatar > More > Render > Always...

Edit: I didn't read the whole bit, nice. That's the first time I've seen the feature not working and I have no clue why that might be.

Neither have i seen it not working yet.

But yes. For the normal dropdown menu its Right-Click - Render Avatar - Always/Never/Default

[Jackson Redstar]

7 hours ago, Wulfie Reanimator said:

(Assuming you're using the pie menu, I'm old and stuck in my ways.) Right-click avatar > More > Render > Always...

Edit: I didn't read the whole bit, nice. That's the first time I've seen the feature not working and I have no clue why that might be.

I always set the main subjects to fully render but that is primary should they fall outside the limit of being a jelly doll. ( i do sometimes de render some 'super fat' wedding guests) If everyone is rendering and nobody is becoming jelly dolled, the render fully really don't seem t have much purpose

[Selc]

On 3/18/2020 at 2:15 AM, animats said:

I'd like to see impostoring taken to the max. Distant sims should be shown as sculpt-like objects with map-like images painted on them. That's what you should see beyond draw range instead of water. This could work like the SL map; have such objects for each sim, each 4 sims, each 16 sims, etc, used as you get further away. See for kilometers. With a tiny bit of blur and fog, you won't see the lack of detail out there. It would make SL feel much bigger. SL is a big world, but it feels cramped, because you can't see very far.

So, known technology could fix the drawing overload for general objects, but SL clothing presents unique problems.

It seems these distant matte backgrounds could work for rendering the contents of distant sims. But the clientside calculations of polygons that generate these matte backgrounds need to be hindered/throttled at certain distances. Currently, even the clientside calculations of polygons that generate avatar impostors aren't being shoved in a rocket and launched into the sun.

On 3/18/2020 at 1:26 AM, NiranV Dean said:

We'd need a flag to set which item should use said system and which shouldn't.

Any mesh beyond a certain polygon threshold. Mesh houses with sensitive edges could be unflagged.

Edited March 20, 2020 by Selc

[animats]

2 hours ago, Selc said:

It seems these distant matte backgrounds could work for rendering the contents of distant sims. But the clientside calculation of polygons that generate these matte backgrounds needs to be hindered at certain distances. Currently, even the clientside calculations of polygons that generate avatar impostors aren't being shoved in a rocket and launched into the sun.

The idea is to generate them with some bot-like process that takes pictures and puts them on a public server, like the system that generates map tiles. Update maybe once a day. Skip isolated estate regions; you can't see them from a distance.The viewer would download one image and one elevation map upon which to paste it per distant sim, not all the objects in the distant sims.

[NiranV Dean]

2 hours ago, Selc said:

It seems these distant matte backgrounds could work for rendering the contents of distant sims. But the clientside calculations of polygons that generate these matte backgrounds need to be hindered/throttled at certain distances. Currently, even the clientside calculations of polygons that generate avatar impostors aren't being shoved in a rocket and launched into the sun.

Any mesh beyond a certain polygon threshold. Mesh houses with sensitive edges could be unflagged.

And then everyone will be unflagging their stuff. 

[Flea Yatsenko]

On 3/20/2020 at 12:53 PM, NiranV Dean said:

And then everyone will be unflagging their stuff. 

That could easily be solved by doing something like increasing land impact of things that are unflagged.

[NiranV Dean]

2 hours ago, Flea Yatsenko said:

That could easily be solved by doing something like increasing land impact of things that are unflagged.

Sounds like an idea. It encourages flagging your stuff. All of it.

[restaurant]

no, selc and i made a partial mistake.

this is correct:

1 max non-impostor and 20k max complexity:
35 million tris @ 14 FPS with render info ticked
35 million tris @ 73 FPS with render info unticked

8 max non-impostors and 65k max complexity:
35 million tris @ 9 FPS with render info ticked
35 million tris @ 62 FPS with render info unticked

35 million tris @ 62 FPS can still be @ 180 FPS or higher (90 FPS in VR) with auto L.O.D. optimizations by engineers

Edited March 23, 2020 by restaurant

[restaurant]

many house meshes/walls are not too high poly. biggest problem is layered bodies, layered heads, invisible mesh layers

52 max non impostors, 350k max complexity: 49 million tris @ 23 FPS  (render info unticked)

Edited March 23, 2020 by restaurant

[chesse Vyceratops]

heavy agree we need optimization For free

[Kyrah Abattoir]

On 3/23/2020 at 9:05 AM, restaurant said:

many house meshes/walls are not too high poly. biggest problem is layered bodies, layered heads, invisible mesh layers

52 max non impostors, 350k max complexity: 49 million tris @ 23 FPS  (render info unticked)

The big killer with things like houses and other rezzed objects is their bloated texture usage because SL creators don't understand how to create objects without baking unique textures for every single one of them.

[animats]

For non-avatar objects in SL, the problem is that the LOD creation pipeline is terrible. The display side in the viewer is OK.

For avatars, LODs barely work at all. The mesh uploader's mesh reducer makes meshes with holes in them if you ask for too much reduction. Creators overuse that feature. So, if you show clothing at a lower LOD, some of it disappears. Not in an attractive way.

So, to keep that from happening, avatar LODs for all worn objects are set to the same LOD as that for the highest LOD on the avatar. Wear a bracelet with fine detail, and the viewer draws all those little triangles on an avatar 50m away.

This comes up occasionally at Creator User Group. Nobody has a simple answer that won't make users scream about their expensive wardrobe being broken.

 

[Extrude Ragu]

I think that, for creators of bodies like Maitreya etc the idea that the issue is the uploader is null and void.

These people make obscene amounts of money spinning out gacha items constantly they can afford to spend time to do some re-topology, they simply don't because most users don't know better and thus don't demand better.

Personally I think the antidote to this problem is to get more people creating. The more people are making things themselves, the more awareness there is of what makes quality items, and takes people out of this 'just take what is given' mindset as they can do better themselves.

[Selc]

On 4/23/2020 at 6:01 AM, Kyrah Abattoir said:

The big killer with things like houses and other rezzed objects is their bloated texture usage because SL creators don't understand how to create objects without baking unique textures for every single one of them.

 

• Auto-generated texture atlases could be used for avatars, objects with more than one texture UUID, and complex linked objects — to drastically lower the draw calls. Sometimes, this can mean the difference between 5 fps and 60 fps for any given scene.

 

• If an avatar/object/linked object is manually modified in any way by the resident, notify the server; rate-limiting for the automatic updating of any given avatar/object/linked object's (multiple) texture atlas cache could be defaulted to once every 15 seconds. If possible, the server should rewrite only the delta output to any updated texture atlases. And nearby residents should only re-download the delta.

 

• An algorithm could be written so that any face/set of faces updating its texture UUID(s) more than once every 15 seconds due to an auto-updating script is automatically excluded from its respective avatar/object/linked object's (multiple) static texture atlas cache.

 

• Animat says the mipmapping code may be sub-optimal at ZerahMordly's thread here; maybe future updates can work in tandem with texture atlases.

 

On 4/23/2020 at 2:58 PM, animats said:

For non-avatar objects in SL, the problem is that the LOD creation pipeline is terrible. The display side in the viewer is OK.

For avatars, LODs barely work at all. The mesh uploader's mesh reducer makes meshes with holes in them if you ask for too much reduction. Creators overuse that feature. So, if you show clothing at a lower LOD, some of it disappears. Not in an attractive way.

So, to keep that from happening, avatar LODs for all worn objects are set to the same LOD as that for the highest LOD on the avatar. Wear a bracelet with fine detail, and the viewer draws all those little triangles on an avatar 50m away.

This comes up occasionally at Creator User Group. Nobody has a simple answer that won't make users scream about their expensive wardrobe being broken.

 

 

This could easily be fixed by giving residents an edge-masking allowance for each article of clothing/any mesh in general.

Edited May 15, 2020 by Selc

[Kyrah Abattoir]

@SelcTexture count can be an issue but it's almost never the issue in itself, but the total pixel area of those objects, whether they use 64 1024x1024 textures or just one 8K textures doesn't change the amount of vram it represents.

I've been working on an extensive pipe kit that uses only about 4-5 textures for more than 30 different pieces. Those textures are for the most part generic and the objects are built to use generics metal surfaces rather than uniquely baked textures. It was completely normal to do that back in the prim days, but during the transition to mesh and then materials we got "stuck" into this idea that any object, no matter how small, how large, or how important it is in the scene has to be baked into unique textures.

 

I do agree that an upper limit for poly/textures on avatars would dramatically improve the problem, (at least on the avatar side of the equation) but Linden Lab will have to be very careful how they approach any perceived limit beyond what we already have, after more than 15 years of existence, the complete freedom to customize one's avatar is probably the most entrenched expectation.

Texture and mesh batching when not being edited is an interesting thought but it would cause an ungodly amount of server strain, because ANY SCRIPTED ATTACHMENT can potentially change itself at a moment's notice. Batching only has a net benefit when the memory/time you saved with batching is superior to the time you spent doing the batching.