Proposal - FAST MESH

[Coffee Pancake]

This is a cunning plan to make manual creation of LOD's for SL models the norm rather than the exception.

The mesh uploader auto LOD makes junk, especially as the person uploading is often more concerned with hitting an arbitrary Li target. These LOD models are ugly and everyone nukes their client performance, maxing out object detail so they don't have to see them. It makes junk, that looks like junk that causes people to degrade performance to junk.

Remove LOD generation from the uploader entirely. Uploader creates a LOD tris count curve based on model and will allow the optional addition of models that meet the tris count target it specifies for each of the LOD steps **

Models with manual LOD's added during upload are named FAST MESH.

Most shoppers wont know or care what FAST MESH is or how it works, but it's called FAST MESH so it's obviously better.

Meshes without a manually upload model at a specific LOD will have that model generated at rez time. This will be horrible for performance and most likely look like junk, SLOW MESH.

SLOW MESH BAD. 

FAST MESH GOOD.

In all seriousness, this places control over auto generated LOD models in the viewer via an on-the-fly decomposition engine. This engine can be easily tuned to a users preference and keyed into their object detail settings. It can also be improved and updated over time for better visual output, performance, etc (maybe even an AI eventually). It's always going to be slower.

The rest is marketing trickery to apply consumer pressure on mesh creators for FAST MESH with manually created LOD models.

** the curve offered by the uploader for manual meshes is far more aggressive than the one used in the viewer real time,  it also costs more to upload multiple meshes to create a FAST MESH and there are no changes to Li or ARC. Uploading a single model will always be the simplest easiest cheapest option. This is to dissuade mesh creators adding junk auto decomposed meshes just so they can make the claim that it's FAST MESH.

Edited March 10, 2021 by Coffee Pancake

[animats]

It's an interesting idea.  How do you keep creators from creating FAST MESH where the lower LODs are blanked out?

I looked into mesh reduction algorithms once. The one SL uses is not very good, of course. There are better ones that would work for buildings and furniture. But really, most of the trouble comes from pushing way too hard on mesh reduction. Pushing the number of triangles into single digits seldom works. Also, there's a minimum below which there's no LI reduction, and it's around 25 triangles. So don't overdo.

SL clothing, has special problems. SL clothing often has actual fabric thickness. There's an outside and an inside, and they join at the edges. Common mesh reduction algorithms are terrible on thin sheets. To visualize what goes wrong, imagine a sheet with a wrinkle or "dent" in it. Mesh reduction should remove the wrinkle. Most modern mesh reduction algorithms try to minimize the volume difference between the un-reduced and reduced forms. So, removing the wrinkle from the side where it goes inward generates a big volume error. So does removing the wrinkle from the outside. This leads mesh reduction to trim the edges of the fabric!

You can try this in Blender. Make a cube, make it into a thin sheet, subdivide to about 10x10 squares, and edit the mesh by grabbing some central squares and pulling them out to make a "dent". Now apply Blender's mesh reduction tools. The result will be very strange.

Mesh reduction for clothing needs to be done in something that knows that it's a thin sheet, and can work on the inside and outside simultaneously. Marvelous Designer?

[Coffee Pancake]

5 minutes ago, animats said:

It's an interesting idea.  How do you keep creators from creating FAST MESH where the lower LODs are blanked out?

The target tris count calculated by the uploader for specific lower LOD models would only allow ±5% (or there abouts). So zeroing out LODs during creation would be impossible.

Zeroing out a model is useful for certain conditions (such has small indoor items or large buildings) where LOD's below a certain point serve no practical purpose, When to use zeroed out models would be up to the client/user not the creator.

5 minutes ago, animats said:

I looked into mesh reduction algorithms once. The one SL uses is not very good, of course. There are better ones that would work for buildings and furniture. But really, most of the trouble comes from pushing way too hard on mesh reduction. Pushing the number of triangles into single digits seldom works. Also, there's a minimum below which there's no LI reduction, and it's around 25 triangles. So don't overdo.

Real time in the client would mean that improvements to the algorithm would affect all content, rather than only content created after a change to the uploader.

5 minutes ago, animats said:

SL clothing, has special problems. SL clothing often has actual fabric thickness. There's an outside and an inside, and they join at the edges. Common mesh reduction algorithms are terrible on thin sheets. To visualize what goes wrong, imagine a sheet with a wrinkle or "dent" in it. Mesh reduction should remove the wrinkle. Most modern mesh reduction algorithms try to minimize the volume difference between the un-reduced and reduced forms. So, removing the wrinkle from the side where it goes inward generates a big volume error. So does removing the wrinkle from the outside. This leads mesh reduction to trim the edges of the fabric!

Which would make hand crafted LOD models for clothing mandatory.

Rather than the current situation were all clothing uses the full detail mesh at all LODs.

 

[FinnfinnLost]

6 minutes ago, animats said:

It's an interesting idea.  How do you keep creators from creating FAST MESH where the lower LODs are blanked out?

I looked into mesh reduction algorithms once. The one SL uses is not very good, of course. There are better ones that would work for buildings and furniture. But really, most of the trouble comes from pushing way too hard on mesh reduction. Pushing the number of triangles into single digits seldom works. Also, there's a minimum below which there's no LI reduction, and it's around 25 triangles. So don't overdo.

SL clothing, has special problems. SL clothing often has actual fabric thickness. There's an outside and an inside, and they join at the edges. Common mesh reduction algorithms are terrible on thin sheets. To visualize what goes wrong, imagine a sheet with a wrinkle or "dent" in it. Mesh reduction should remove the wrinkle. Most modern mesh reduction algorithms try to minimize the volume difference between the un-reduced and reduced forms. So, removing the wrinkle from the side where it goes inward generates a big volume error. So does removing the wrinkle from the outside. This leads mesh reduction to trim the edges of the fabric!

You can try this in Blender. Make a cube, make it into a thin sheet, subdivide to about 10x10 squares, and edit the mesh by grabbing some central squares and pulling them out to make a "dent". Now apply Blender's mesh reduction tools. The result will be very strange.

Mesh reduction for clothing needs to be done in something that knows that it's a thin sheet, and can work on the inside and outside simultaneously. Marvelous Designer?

When talking about clothing though, most (not all) clothes shouldn't have a true thickness on the inside anyway. "Tucking in" that edge works for the illusion and has way less tris either way.

Back to topic, automatically generating LODs is something that is very much situational. And I've yet to see a detailed model for which a simple decimation works for lower levels, I'm told UE4 generates great LODs most of the time, but that's just a sidenote. Encouraging manual creation of LODs is something I support, but removing generation altogether might take it too far if you ask me.

[Coffee Pancake]

1 minute ago, FinnfinnLost said:

but removing generation altogether might take it too far if you ask me.

That's the point.

Hand made LOD models are easy to make requiring only basic skills, the bulk of which comes down to deleting faces, removing loops, merging verts, zooming out and squinting. Anyone lacking those skills and uploading a mesh probably didn't make the mesh, also something that should be dissuaded. 

The only hard part is knowing what tris count to target for certain LODs, which is why the proposed uploader would request models with a certain tris count ±5% (or there abouts), If a user can't provide them, then on-the-fly LODs is the easy option.

[Kyrah Abattoir]

I do like the idea of the uploader essentially "telling" you what your lod tricount should be.

That's how custom models used to work on there.com, you had to match a target.

In a way it would ensure a minimum quality standard. A creator COULD still create garbage, but at that point it would start to look like intentional sabotage.

And while this might be a bit gatekeepy I do agree that there is no benefit in having lod generation built-in if it produce bad results.

[Kyrah Abattoir]

Now how do we convince LL (but more importantly, Firestorm) to ditch the lod generator?

[Coffee Pancake]

10 hours ago, Kyrah Abattoir said:

Now how do we convince LL (but more importantly, Firestorm) to ditch the lod generator?

TPV's with mesh upload have physically signed and submitted the havok sublicence to LL. No convincing is required.

LL .. maybe some kind of cunning plan 🤔

[Mollymews]

if there was a minimum for each level and not exceeding the maximum for the next level up and there was a scale that went 8 4 2 1 for example, were 8 can be any number and each level down is half.  So if 8 equals 80 then halve it down the levels. 80, 40, 20, 10. If 8 = 100 then 100,50,25,12.5. And so

in the 8=80 case then the cost is 80+40+20+10 = 150.

if we only create the top level. 100 = 8 then the penalty is 100=8 for each level. Cost = 400

i can see how this will work pretty good for rezzed objects and I am supportive of this

I think tho that in the worn attachments market then nobody will create the lower levels as there is no penalty imposed on the wearer, unlike there is for parcel owners. And to be fair to the creators of such attachments, I think there are plenty of customers who would prefer full LOD attachments at every level when wearing them

Edited March 18, 2021 by Mollymews
400

[Wulfie Reanimator]

49 minutes ago, Mollymews said:

if there was a minimum for each level and not exceeding the maximum for the next level up and there was a scale that went 8 4 2 1 for example, were 8 can be any number and each level down is half.  So if 8 equals 80 then halve it down the levels. 80, 40, 20, 10. If 8 = 100 then 100,50,25,12.5. And so

in the 8=80 case then the cost is 80+40+20+10 = 150.

if we only create the top level. 100 = 8 then the penalty is 100=8 for each level. Cost = 400

i can see how this will work pretty good for rezzed objects and I am supportive of this

This is similar to how land impact is calculated for Animesh objects.

Quote

There is an additional land impact cost of 1.5 per 1,000 charged triangles in the model. All triangles in the highest LOD are charged, but only triangles exceeding a limit are charged in the coarser LODs.

There is no LI penalty for LODs as long as each does not exceed half the complexity of the next highest level.

For example, if the highest LOD has 10,000 triangles, you could make LODs with no penalty up to 5,000 triangles in the medium LOD, 2,500 in the low LOD, and 1,250 in the lowest LOD. LODs that exceed those limits will be counted against the LI only for any excess above those limits.

 

Edited March 18, 2021 by Wulfie Reanimator

[Gabriele Graves]

Sounds like a good proposal to me though I like the term "Light Mesh" as opposed to "Heavy Mesh" better.

Edited March 18, 2021 by Gabriele Graves

[Mollymews]

56 minutes ago, Wulfie Reanimator said:

This is almost identical to how land impact is calculated for Animesh objects.

 

i think in the halving method and we do (or do not) create the LODs ourselves then the penalty should be incurred in all non-optimal cases

so if we create a 8 cost object and we then create a single element for level 4 then that one element is going to cost 8. The only time it will cost 4 is when we create a 4 element LOD. Same at levels 2 and 1. Two elements and 1 element respectively

which is what I understand Kyrah was getting at

the idea being that the optimal cost is also the minimum cost: 8+4+2+1 = 15. And the maximum cost (least optimal) is 32

if we overload the LODs we pay for it. 8+5+3+1 = 17.  And if we underload then we pay more. Example: 8+3+1+1 costs 8+8+8+1=25.  8+5+1+1 costs 8+5+8+1 = 22. 8+0+0+0 costs 8+8+8+8 = 32. And so on

edit add: for completeness

in a runtime situation then there would be some small tolerance at each level. Maybe +/- 2 percent or about. So 800,400,200,100 is 800,(408-392),(204-196),(102-98). Idea being that if we create to fit within the allowable tolerances then we get the optimal cost

 

 

Edited March 18, 2021 by Mollymews
element

[VirtualKitten]

it certainly is a nice idea FASTMESH , LIGHTMESH what ever you want to call it, but how is this going to cater for different visual requirements of items you build i.e. some mesh items do not require visibility at distance but some do.  This methods outlined would not cater for these models . Currently am having to pay for all mesh upload as the beta sites don't work for testing . This is expensive  and has been down more than six months to a year.

 

 

[Coffee Pancake]

6 hours ago, VirtualKitten said:

but how is this going to cater for different visual requirements of items you build i.e. some mesh items do not require visibility at distance but some do.

That would be up to the viewer as unless provided, LOD models would be generated on demand. The viewer does have the ability to render different models based on screen real-estate distance etc. There is a lot of room to make the viewer smarter about what it renders and when, but ONLY if the LOD models are of a predictable complexity. 

The current situation allows creators to override that with what looks right to them, on their setup, with firestorm (and it's broken LOD handling left over from when creators jacked sculpts so hard it needed a viewer hack to make them render).

Right now we creators second guessing the viewer, and actively targeting firestorms bricked LOD handling. Garbage in. Garbage out.

[Wulfie Reanimator]

42 minutes ago, Coffee Pancake said:

The current situation allows creators to override that with what looks right to them, on their setup, with firestorm (and it's broken LOD handling left over from when creators jacked sculpts so hard it needed a viewer hack to make them render).

Right now we creators second guessing the viewer, and actively targeting firestorms bricked LOD handling. Garbage in. Garbage out.

What is it about Firestorm specifically?

[ChinRey]

On 3/10/2021 at 5:10 PM, Coffee Pancake said:

Remove LOD generation from the uploader entirely. Uploader creates a LOD tris count curve based on model and will allow the optional addition of models that meet the tris count target it specifies for each of the LOD steps **

Or at least make it optional, not the default, and replace GLOD, the Lod model generator the uploader uses now wth one more sutiabøe for the purpose.

 

On 3/10/2021 at 7:14 PM, animats said:

I looked into mesh reduction algorithms once. The one SL uses is not very good, of course.

It's probably more correct to say it's unsuitable. GLOD is intended for dynamic creation of LoD models so it's bult for speed, not precision. But of course, speed is totally irrelevant for the uploader and it raises one interesting question: What was the Linden who decided it was a good choice smoking at that time? Must have been some seriously weird substance.

 

7 hours ago, Coffee Pancake said:

The current situation allows creators to override that with what looks right to them, on their setup, with firestorm (and it's broken LOD handling left over from when creators jacked sculpts so hard it needed a viewer hack to make them render).

That is the crucial point. The viewers' RenderVolumeLODFactor is by far the biggest obstacle for promoting technically good mesh and control the render cost of objects. There is no solution to either of those issues as long as draw distance can be changed on viewer level. The correct and only sensible way to do it, would be to include a LOD factor for individual objects. That way the creator would know the LoD swap distances of their builds and it wouldn't be possible to "cheat" on the land impact sicne increasing the LOD factor would also increase the LI accordingly.

Any other possible changes to the way mesh is handled and uploaded are trivial by comparasion.

 

On 3/11/2021 at 11:47 AM, Kyrah Abattoir said:

I do like the idea of the uploader essentially "telling" you what your lod tricount should be.

I don't. I hate it. Well, actually I love it as an idea but I've yet to see anybody suggest a way to actually make it work so it's just a fluffy dream.

 

On 3/17/2021 at 1:39 PM, Kyrah Abattoir said:

Now how do we convince LL (but more importantly, Firestorm) to ditch the lod generator?

That's easy. We wait for Hell to freeze over and then we propose it to them.

Edited March 18, 2021 by ChinRey

[Coffee Pancake]

6 hours ago, Wulfie Reanimator said:

What is it about Firestorm specifically?

Back when sculpts were the bees knees, various tricks were developed to cram as much geometry / complex shapes into them as possible for a single prim/Li. These sculpts looked like screwed up hot garbage unless you were right on top of them -even with the object detail in the viewer maxed out. Refusing to admit they were pushing the format way past it's design intentions, creators started telling people to edit LOD related debug settings. In response, FS changed their object detail slider making it go to double what was included in the Linden client.

Medium in FS is max in everything else.

The actual values are displayed in FS preferences, but unless you already know what the numbers mean or what those values are in the stock client they don't inform the user of anything. Meaningless numbers are meaningless.

Mesh creators, using and testing their product on Firestorm can and do publish mesh with overly aggressive decomposed junk / missing LOD models.

This has in effect created an LOD arms race to the bottom and applies pressure on consumers to ignorantly bump their own detail settings to compensate for purchased items popping in and out of existence.

It also kneecaps the viewers built in LOD handling reducing performance for everyone on every viewer in every situation.

5 minutes ago, ChinRey said:

Or at least make it optional, not the default, and replace GLOD, the Lod model generator the uploader uses now wth one more sutiabøe for the purpose.

It is optional in the sense that uploaders will have a choice between going the extra mile to make and append models to a specific triangle budget, or they can choose to YOLO and let the users viewer  do decomposition on the fly -this will be no worse than the creator fiddling with the sliders during upload to achieve a desired Li goal or in the case of clothing setting them to the max.

(important note - hyper detailed rigged avatar attachments is why the viewer tanks the moment people use this social platform to gather, socially. Mesh as it stands has created an ecosystem where avatars impose a linear performance impact that has only increased over time. Appealing to creators to do the right thing has failed YET AGAIN. The current trajectory is unsustainable and needs to die - there is no more CPU headroom to make SL more performant on the horizon - bigger GPU's do nothing - it can only get worse)

Fixing the LOD generator on the uploader still sets all content in stone. Manual budgeted models OR on the fly decomposition allows for algorithmic improvements to affect everything created after the change.

(as an aside, the forced change will give LL a huge dataset of manually optimized models, this can be used in turn to train an AI and further improve the on-the-fly decomposition at some point in the future - an AI being only as good as the training data is a big reason for manual LOD model upload to be punitive, only creators who really care will go to the extra effort & expense required to make a full set of budgeted manual LODS. -simply asking the uploader "is this hand made and awesome" isn't sufficient as people will lie polluting the dataset --yes an AI model could be developed without manually created mesh LODs, but that's a far more ambitious and expensive undertaking) 

 

[Ardy Lay]

43 minutes ago, Coffee Pancake said:

FS changed their object detail slider making it go to double what was included in the Linden client.

Medium in FS is max in everything else.

I screamed about this for a while when they did it.  My screaming about it made me very unpopular and I was attacked several times by the self-proclaimed gods of third-party-viewer-excellence.  I had to ban some people from all the properties I spend time on and establish a "no third-party viewer evangelism" policy on those properties to eject any additional extremists to the swamps.

Good luck getting the third-party viewer(s) fixed.  I have found that lots of really beautiful content is essentially unusable because seeing it requires, as you explain, extremely close proximity or bashing the LoD processing in the viewer resulting in EVERYTHING ELSE rendering in excessive detail for the amount of screen space it occupies.

[Gabriele Graves]

I have found many circumstances where you just cannot set the LOD slider to max and everything works anyway.  Items disappear when you do that as well.  They don't seem to collapse so much as disappear, as in no triangles, leaving just the baked shadow on the floor (in the case of a bed I have).  So if it is a collapse then it collapsed completely.  I am talking within a single room too.  When you cam around, the items that disappear reappear and some others disappear.

So we are pretty much at the point where there is no LOD slider setting in which FS will render all items in say a randomly (with no expert thought put into what items, etc.) fully furnished room in a home.  At least that has been my experience for a while.

Edited March 19, 2021 by Gabriele Graves

[Mollymews]

some info

i use the Linden Viewer (auto-updating to the latest release)

i have a NVidia 1050TI GTX and a 2560 x 1440 screen. i7CPU. 16GB RAM. Windows 10 Home

i use SL using the Recommended settings

EEP Use Shared Environment = On

RenderVolumeLODFactor = 1.125.  The default Linden setting

i go to places like Bellissaria and this is what I see

i can see the light over the bridge (which is about 120meters away)

and this view renders at about 20 FPS and I get zero texture thrashing

and this is what I want to see everywhere

sometimes people have it in for the Moles, saying they could be more efficient. But my ask of people is try making stuff at 1.125 that I can see the same as I can here at 20FPS on this resolution screen with Shadows and EEP

 

i also say about stuff breaking stuff if Linden changed the LI costings

from my pov, stuff that doesn't render like it does in the pic is already broken. Breaking broken stuff doesn't make any difference, is still broken . And if breaking already broken stuff eventually results in the view I can get above, everywhere I go, then I vote for breaking already broken stuff

[ChinRey]

7 hours ago, Mollymews said:

i have a NVidia 1050TI GTX and a 2560 x 1440 screen. i7CPU. 16GB RAM. Windows 10 Home
...

sometimes people have it in for the Moles, saying they could be more efficient. But my ask of people is try making stuff at 1.125 that I can see the same as I can here at 20FPS on this resolution screen with Shadows and EEP

😮

With that setup you ought to get at least 100 fps!!!

I'm one of the people who criticize the Moles for making inefficient mesh but they're not that bad. There's something else going on here.

Edit: I keep going back to this old thread of mine but it should give you some prespective on this.

With a battered old not-at-all-a-gamer Lenovo desktop, tiny little Radeon R7 240 gpu, 8 GB ram, 1920x1080 screen, graphics set to ultra with draw distance cranked up to 1024 m I could still easily get more than 20 fps. Yes, it's a smaller screen but with much weaker hardware and a draw distance as close to decent as the viewer will allow.

Edited March 19, 2021 by ChinRey

[ChinRey]

10 hours ago, Coffee Pancake said:

as an aside, the forced change will give LL a huge dataset of manually optimized models, this can be used in turn to train an AI and further improve the on-the-fly decomposition at some point in the future - an AI being only as good as the training data...

You answered yourself in the sentence I underlined. The huge dataset of optimized models will be buried in a much huger dataset of not very well optimized ones. How is the AI supposed to tell the difference?

If it's a huge training set they want, I'll be happy to give them all my mesh files for that purpose and I'm sure some fo the other lag concious mesh makers will be too. But even so, how is the AI supposed to make sense of it? Sometimes you can get a 90+% reduction between LoD levels, sometimes you can't reduce anything at all.

[ChinRey]

9 hours ago, Gabriele Graves said:

I have found many circumstances where you just cannot set the LOD slider to max and everything works anyway.  Items disappear when you do that as well.  They don't seem to collapse so much as disappear

Yes, in addition to all the other negative effects, setting the LoD factor too high may well reduce, not increase the visual quality. There are two reasons for this. One is that you risk pushing the gpu so ahrd it can't manage to handle it all and has to skip some draw calls. The other is that you risk ending up with tris so small they're close to - or even less than - a screen pixel. Sorting that out is hard work for the poor gpu. @Beq Janus explained this in a blog post once. I can't find the link at the moment but maybe she can?

 

Edited March 19, 2021 by ChinRey

[ChinRey]

On 3/18/2021 at 9:57 AM, VirtualKitten said:

Currently am having to pay for all mesh upload as the beta sites don't work for testing . This is expensive  and has been down more than six months to a year.

This is a digression but I asked them about it earlier this week. Apparently they are working on it but couldn't give any ETA and I don't have the impression fixing the beta grid is high priority for them.

In the meantime there's always opensim to do LoD testing. You can't test physics there of course but keep an eye on the upload cost before you hit the upload button. That's often a good indicator of how well the physics model works.

[Mollymews]

1 hour ago, ChinRey said:

😮

With that setup you ought to get at least 100 fps!!!

 

in full screen mode 2560 x 1440 up in the sky sitting on a box on Evard region up at 1000meters (the closest rez zone to where I showed before)

nothing in the view except me in same outfit and the box. average about 64-65 FPS

derender everything in the view (including me and the box) using Linden Viewer menus: Advanced \ Rendering Types & Rendering Features leaving only render UI

 

average about 81-82 FPS rendering nothing other than the UI

i have done this on a standalone region absolutely empty and the best it can go is 144 FPS. Can see this as well in the above, spiking to 144 FPS

144 FPS is the fastest my computer setup can go in SL, when I don't render anything

the difference between 144 and 81 (63 FPS) is pretty much the cost of my CPU in processing everything else on this Evard region other than sending stuff to the graphics card

the big killer in SL is how much work the CPU has to do even when nothing is being rendered. Other people who know much more about the innards of the viewer have talked about this before. My pics just confirm what is said, at least from my own experience using the program 

just add that I am bit of a bling bunny avatar-wise, about 17-18 FPS just for me on average. Some outfits are a bit less and others a bit more. Like about 22 for my blingyest outfit

so I don't get out much and pretty much stick to myself so I don't mess other people up