Why is there so much high-poly mesh in SL?

[Cube Republic]

Doesn't unity handle it this way Animats? 

[animats]

27 minutes ago, Cube Republic said:

Doesn't unity handle it this way Animats? 

All the major game engines have some impostor system. There are several billboard impostor systems for Unity. The impostors are usually pre-generated, not generated at run time.

Dynamic impostors were used in Tycoon City. Here's the Gamasutra article.

Tycoon City's New York. 2005 technology. Looks pretty good. If it's in the distance, it's an impostor. But you can't tell, mostly. Even when you're moving around fast. Nice in-world building, too.

Edited November 9, 2018 by animats

[Macrocosm Draegonne]

5 hours ago, animats said:

Sinespace now has good in-world building. Sansar and High Fidelity don't. What SineSpace really has is prim building with user-defined prims. You can create a parameterized couch with the out of world tools. In world, users can stretch it, and it acquires more cushions, not wider ones. The arms don't get fatter. Wall sections can be stretched, and they acquire more windows, not wider ones. The bricks stay the same size. So Sinespace has  builder parts the average user can use effectively. The parts have to be optimized for level of detail once. The part user doesn't have to worry about that. User level building is very fast.

Thats something LL could allow creators to offer, its only some extra object properties to account for stretching, I've asked some similar questions about that in regard to slice scaling, so we can set the scaling points of objects to keep certain parts from deforming, then having tileing ability in the inner non scaling areas.  This would be basically a new object type or additional properties that could be created in an object.  Doesn't make sense for all objects to do so but its neat that game does that, they can I suppose because its a limited asset base.

5 hours ago, animats said:

On the LOD front, there are some things that could be done in the viewer. Run-time impostors for objects, like the ones for avatars. Some of the Firestorm people are interested in that, but they have to integrate animesh and EEP first.

WooT!

 

Edited November 10, 2018 by Macrocosm Draegonne

[animats]

Mesh reduction test.  From right to left: 35,000 triangles original, 2500 triangles, 200 triangles, 34 triangle impostor, 34 triangle impostor base model.

Here's what's going on. On the right, we have a high-poly model downloaded from a 3D site, with about 35,000 triangles. That's far too many for a chair in SL.

So I ran Meshlab 2016.12 on it and applied quartic mesh reduction to bring it down to 2500 triangles. I've written before about quartic mesh reduction being much better than any of the options in Blender or in the SL uploader. So here it is.This was entirely automatic - just set the number of triangles desired. This could be your highest level of detail.

Next we push quartic mesh reduction harder and take this down to 200 triangles. The casters disappear and the cushions are a bit squared off, but nothing bad has happened. This is an adequate medium level of detail.

Now we go to impostors. The black chair, 34 triangles, is from my impostor generator. The colors come from textures in Blender, which were not brought through for the other models, so the colors don't match. If you want this to work, all coloring and texturing must be done in Blender. The brown color was added in SL.   Look at the chair legs. That's an alpha texture, not geometry. This is the low and lowest level of detail.

At left is the hand-created impostor model. The black chair is that model with textures baked on it.

All this testing is for untextured meshes. Next, work on textured ones.

This gives a sense of what modern mesh reduction algorithms can do. This isn't even the best technology available. This is all free software. There are expensive commercial solutions which claim to be better. The point here is to show that automatic level of detail model generation is quite feasible. Everything can look good at any distance without choking SL.

Others must have been down this road. Comments?

Notes:

• Meshlab versions before 2016.12 crash if pushed hard on quartic mesh reduction. So far, Meshlab 2016.12 hasn't crashed on this operation. More testing to follow.
• Meshlab takes Collada format (.dae) in and puts Collada out; it won't work directly on Blend files.
• Workflow for this needs to be worked out.

 

[ChinRey]

1 hour ago, animats said:

Comments?

I'm not familiar with the principles of quartic mesh reduction but yes, I have noticed that MeshLab tends to do a much better simplification job than Blender can do automatically and Blender does a far better job than the uploader.

Even so, if you look at the 200 tris model, it's clearly not good enough for medium LoD and it's too high poly for low.

Edited November 19, 2018 by ChinRey

[Kyrah Abattoir]

Dunno if i mentioned my "semi-automatic" approach but here goes.

I'll put a decimate modifier on my model and dial it to the target polygon count and when i run out of ideas on how to further reduce the mesh, i look at the decimator "suggestion" sometimes what Decimate does is unacceptable for me but some other times it removes some of the guesswork on hand made lod models.

Edited November 19, 2018 by Kyrah Abattoir

[animats]

11 hours ago, ChinRey said:

I'm not familiar with the principles of quartic mesh reduction but yes, I have noticed that MeshLab tends to do a much better simplification job than Blender can do automatically and Blender does a far better job than the uploader.

Even so, if you look at the 200 tris model, it's clearly not good enough for medium LoD and it's too high poly for low.

Yes, 200 tris was not a great choice. I just wanted to see how far I could go and still have it look OK.

For an overview of approaches to mesh reduction, see this slide set from Stanford. Especially slide 19. Quartic mesh reduction tries to minimize the volume between the original and reduced mesh, which is a reasonably good goal. Blender's tools are limited dissolve and decimation. Limited dissolve tries to flatten shallow angles, so it does badly on rough areas with sharp edges like lettering or engraved detail. Decimation just attacks the smallest triangles, so it randomly removes detail. Those are both very local operations.

The real problem with using Meshlab is workflow. Meshlab is a standalone program which can read and write several 3D formats, but .blend is not one of them. So you have to export from blender as .dae or .obj, import into Meshlab, work on the mesh, re-export, and import back into Blender. Now you've lost all the Blender-specific info like nodes, and probably introduced other problems. For example, Blender to .dae to Meshlab to .dae to Blender swaps axes somewhere and the object comes out rotated. So  you can only use Meshlab either before starting work in Blender, or when all done with Blender, just before upload to SL.

Meshlab is really a basic 3D GUI which calls "filters", the little programs that do all the work. The filter interface is well documented. It might be possible to build an add-on for Blender which can call Meshlab filters, allowing Meshlab mesh tools to be used from within Blender. I've posted on that in a Blender group to see if there's any interest. Someone tried something like that in 2012.

[Macrocosm Draegonne]

How has Sansar solved this?  I recently realized we do not have to upload LOD there, so its automatic, and absolutely imperceptible in game too everything looks perfect no matter how far away.

[animats]

2 hours ago, Macrocosm Draegonne said:

How has Sansar solved this?  I recently realized we do not have to upload LOD there, so its automatic, and absolutely imperceptible in game too everything looks perfect no matter how far away.

They just load everything, have huge downloads, and require that you have enough graphics card memory for it. NWN has a writeup.

[Macrocosm Draegonne]

12 hours ago, animats said:

They just load everything, have huge downloads, and require that you have enough graphics card memory for it. NWN has a writeup.

Yeah I cant load harvest at all 24gb! wow thats way overkill!  My internet is too slow, however, that said, even with crappy net, I do not have to wait long for experiences to load, and I always have 100fps which ive capped it at.  Plus, my PC is 6 years old, not exactly a beast of a machine by today's standards.

I would have to say LOD must be in play, otherwise my system would just crap itself with all the millions and millions of prims in any given direction looking around a scene, there is also EXCELLENT occlusion built in too.  You can see in my short one min video here with the Diagnostics enabled (upper left corner) how many prims are around, and not bothering performance one bit, FPS is just above & left of the prims # its called Rate #

I see in that article many things but no mention of LOD, other than the sentiment its mysterious in some areas due to still being in beta.  And I have to laugh at the concurrency guessing, there is NO way to know how many people are in Sansar, unless your on LL staff, those numbers are not published anywhere.  Only numbers available are on who is in a publicly accessible experience, and none of the people in Edit mode, which is most still at this point.

Edited November 21, 2018 by Macrocosm Draegonne

[animats]

That scene has less than a thousand draw calls, and several million "prims". That reflects a big static environment.

SL needs draw call ratios like that. The big bottleneck in SL viewers is computing transform matrices and sending them to the GPU with a draw call. In SL, every object has its own transform, because it can  potentially be moved. The viewer has no idea which objects are going to receive object updates telling them to move. So each individual object has its vertices stored in its local coordinate system, and the CPU sends a transform matrix with a draw call to transform them to world coordinates and draw the individual object.

If you use Blender, you know that you can apply all the transformations (rotation, translation, scaling) to all the vertices in a group of objects. This bakes all the transformations; you can't move child objects separately any more. Game engines do that for background objects. Everything that's been baked like that can be drawn with one draw call. With vertices and textures already loaded into the GPU, the CPU just says "draw all that" and the GPU does the rest.

Potentially, an SL viewer could cache, on the assumption that most objects don't move. (Does it do this already?)  Combine big groups of non-moving objects, bake the vertex locations, and draw with one draw call. If any of those objects move, you have to discard the combined group and rebuild it without the moving objects, which means extra overhead. So there's a tradeoff, and if done badly, it makes things worse. All it takes is an occasional random leaf falling from a tree.

[Macrocosm Draegonne]

17 minutes ago, animats said:

That scene has less than a thousand draw calls, and several million "prims". That reflects a big static environment.

SL needs draw call ratios like that. The big bottleneck in SL viewers is computing transform matrices and sending them to the GPU with a draw call. In SL, every object has its own transform, because it can  potentially be moved. The viewer has no idea which objects are going to receive object updates telling them to move. So each individual object has its vertices stored in its local coordinate system, and the CPU sends a transform matrix with a draw call to transform them to world coordinates and draw the individual object.

If you use Blender, you know that you can apply all the transformations (rotation, translation, scaling) to all the vertices in a group of objects. This bakes all the transformations; you can't move child objects separately any more. Game engines do that for background objects. Everything that's been baked like that can be drawn with one draw call. With vertices and textures already loaded into the GPU, the CPU just says "draw all that" and the GPU does the rest.

Potentially, an SL viewer could cache, on the assumption that most objects don't move. (Does it do this already?)  Combine big groups of non-moving objects, bake the vertex locations, and draw with one draw call. If any of those objects move, you have to discard the combined group and rebuild it without the moving objects, which means extra overhead. So there's a tradeoff, and if done badly, it makes things worse. All it takes is an occasional random leaf falling from a tree.

That makes sense, and I like the SL idea too.

One thing to note though, there were a ton of objects all over that scene that can be picked up and interacted with, and many moving things too.

[ChinRey]

56 minutes ago, Macrocosm Draegonne said:

I would have to say LOD must be in play, otherwise my system would just crap itself with all the millions and millions of prims in any given direction looking around a scene, there is also EXCELLENT occlusion built in too.

Sansar relies heavily on Umbra3D for simplification.

 

17 minutes ago, animats said:

SL needs draw call ratios like that. The big bottleneck in SL viewers is computing transform matrices and sending them to the GPU with a draw call. In SL, every object has its own transform, because it can  potentially be moved.

SL even has separate draw calls for each face on the same prim/mesh when they share the same texture data.

[Macrocosm Draegonne]

6 minutes ago, ChinRey said:

Sansar relies heavily on Umbra3D for simplification.

Very nice! No wonder its so slick and smooth.❤️ wowza https://www.umbra3d.com/

Quote

Umbra Composit is the world's first fully-automated solution for delivering high-resolution, photorealistic 3D to web, mobile, AR and VR devices. Change the way your 3D content is viewed, shared and interacted with.

 

6 minutes ago, ChinRey said:

SL even has separate draw calls for each face on the same prim/mesh when they share the same texture data.

ouch, so many draw calls here, its why I thought some proper use of texture atlases may help in some specific cases.

Edited November 21, 2018 by Macrocosm Draegonne

[Kyrah Abattoir]

Objects that are added to the  navmesh cant be moved  its a shame that its not also used to determinate what is static in the scene.

[animats]

The bigger picture is this. Until a few years ago, Second Life had its technology niche mostly to itself. The Massively Multiplayer Online industry did some specialized big world games, but the mainstream game tools didn't really support persistent big worlds where players could build.

In the last few years, that's changed. There are MMOs with worlds bigger than SL. There are MMOs with user building. There are MMOs with economies. So the tool industry has had to support big persistent worlds. Now there's SpatialOS, and Umbra, and Simplygon, and InstaLOD, and they're integrated with Unreal Engine and Unity.

With all these off the shelf tools, small teams can build a big shared virtual world. One guy did it by himself. He created "Community Garden" which is on Steam. It's a big virtual city, where you can walk around and tend the gardens.  This was mostly a tech demo. Then he came out with "Sominum Space", which has scripting, building, commerce, and VR. The number of users is small, but bigger than Sansar. I haven't tried this, but might. I suspect this guy wants to be acquired by a bigger player.

The barriers to entry are much lower than they used to be. Competition is coming. LL needs to get ready. They're not going to be the only game in town much longer.

[Macrocosm Draegonne]

6 hours ago, animats said:

The bigger picture is this. Until a few years ago, Second Life had its technology niche mostly to itself. The Massively Multiplayer Online industry did some specialized big world games, but the mainstream game tools didn't really support persistent big worlds where players could build.

In the last few years, that's changed. There are MMOs with worlds bigger than SL. There are MMOs with user building. There are MMOs with economies. So the tool industry has had to support big persistent worlds. Now there's SpatialOS, and Umbra, and Simplygon, and InstaLOD, and they're integrated with Unreal Engine and Unity.

With all these off the shelf tools, small teams can build a big shared virtual world. One guy did it by himself. He created "Community Garden" which is on Steam. It's a big virtual city, where you can walk around and tend the gardens.  This was mostly a tech demo. Then he came out with "Sominum Space", which has scripting, building, commerce, and VR. The number of users is small, but bigger than Sansar. I haven't tried this, but might. I suspect this guy wants to be acquired by a bigger player.

The barriers to entry are much lower than they used to be. Competition is coming. LL needs to get ready. They're not going to be the only game in town much longer.

One of the nice things about being the first and/or biggest most successful of a thing  (if you play your cards correctly), Is that no matter what competition may do it only pushes you higher, because you're ahead of them in the game and they cant really ever catch up, they're treading ground you've already thought through and designed way past, ex. Sansar which will rock it out when its full blown.  Besides... competition is excellent for market development, and most especially for consumers.

Years back the community decided they did not want new tech, they did not want c#, or unity, or any such thing. That is when Sansar started IMO, and its not as though SL people want less, its just that they want it in a different way, opensource based, community built, open development, quite different from Sansar, but both are awesome to me.  I dont doubt SL can and will improve, seems the weather has changed around here, and things are in high gear.

To your other point about the ease of one guy building out things, well whats the difference now from any other time in history?  More is more, doesnt matter if less people can accomplish more than prior, because larger more experienced groups can do far more than that, always.  They have all those tools too, and can afford the nextgen stuff those tiny teams cant touch until many years go by, if ever.

Edited November 22, 2018 by Macrocosm Draegonne

[Ana Stubbs]

@chinrey It is out there if you search (and I have, extensively), but you do have to piece it together, and I had the distinct advantage of knowing what I was looking for.

The ones that were most helpful were aimed at making game assets. 

I've taught myself mesh making in the course of about three months, and I'm confident I could produce multiple variations on that third tree with ease. It wouldn't be the best tree SL has ever seen, but it would be low poly. 

It would also be an incredibly ugly tree, because a texture like that should be hand-drawn, and I can't draw for toffee - which is why I buy my trees from other people and stick to building non-organic forms. 

[Ana Stubbs]

Ooops, I got the first and last page of the thread mixed up. Sorry about that, serves me right for trying to read the forum on my phone. 

[Kyrah Abattoir]

On 11/22/2018 at 12:17 AM, animats said:

The bigger picture is this. Until a few years ago, Second Life had its technology niche mostly to itself. The Massively Multiplayer Online industry did some specialized big world games, but the mainstream game tools didn't really support persistent big worlds where players could build.

In the last few years, that's changed. There are MMOs with worlds bigger than SL. There are MMOs with user building. There are MMOs with economies. So the tool industry has had to support big persistent worlds. Now there's SpatialOS, and Umbra, and Simplygon, and InstaLOD, and they're integrated with Unreal Engine and Unity.

With all these off the shelf tools, small teams can build a big shared virtual world. One guy did it by himself. He created "Community Garden" which is on Steam. It's a big virtual city, where you can walk around and tend the gardens.  This was mostly a tech demo. Then he came out with "Sominum Space", which has scripting, building, commerce, and VR. The number of users is small, but bigger than Sansar. I haven't tried this, but might. I suspect this guy wants to be acquired by a bigger player.

The barriers to entry are much lower than they used to be. Competition is coming. LL needs to get ready. They're not going to be the only game in town much longer.

The size is irrelevant, it's merly a scaling factor, what matters is how much information is packed in ths space. Do remember that SL doesn't (cannot) rely on cooked content while any major unreal/unity game cannot exist without it.

[Klytyna]

1 hour ago, Kyrah Abattoir said:

Do remember that SL doesn't (cannot) rely on cooked content while any major unreal/unity game cannot exist without it.

People have spent best part of a year trying to tell him that, pretty much every time he said something like "i know nothing about SL or 3D, but why isn't SL more like fill-in-the-blanks-with-your-cooked-content Unreal/Unity MMOFPS games?"

He still doesn't seem to get the difference between a pre-cooked 30gb one time download and continually streaming a selection from over a petabyte of uncooked content...
 

[Kyrah Abattoir]

6 minutes ago, Klytyna said:

People have spent best part of a year trying to tell him that, pretty much every time he said something like "i know nothing about SL or 3D, but why isn't SL more like fill-in-the-blanks-with-your-cooked-content Unreal/Unity MMOFPS games?"

He still doesn't seem to get the difference between a pre-cooked 30gb one time download and continually streaming a selection from over a petabyte of uncooked content...
 

Oh no, he make some genuine and valid points, but not on this topic.

[animats]

8 hours ago, Kyrah Abattoir said:

The size is irrelevant, it's merly a scaling factor, what matters is how much information is packed in ths space. Do remember that SL doesn't (cannot) rely on cooked content while any major unreal/unity game cannot exist without it.

The big MMO worlds are becoming more dynamic. Check out "Worlds Adrift".

SL has some "baking". You can't make  or change a mesh in-world. You can't make or change a texture in-world. And, of course, there's "bakes on mesh". That's all pre-built work. What SL doesn't have is a processing phase between upload and rendering where there's object grouping and optimization. Sansar does. For SL, you have to be able to re-run that processing as objects move. It's a caching problem. Most objects don't move and aren't animated, so you could group ones that haven't moved recently, bake their transforms, and draw them with one draw call. Or render them once and and make billboard impostors, a useful trick for distant objects.

Edited November 24, 2018 by animats

[Arduenn Schwartzman]

On 10/6/2018 at 11:20 AM, WingsOfPurity said:

Why is there so much high-poly mesh in SL?

Maybe there's some positive feedback mechanism at play here that causes a runaway effect. Sort of like male peacocks evolving bigger tail feathers that impair their flight.

Person A's skin and clothes and jewelry and furniture and knick-knacks are more detailed than person B's and their Flickr pictures are more crisp, so person B buys more detailed ones too and zooms in more to show how rounded their curves are and how much more intricate all the details. Creators pick up the tendency of buyers to buy more detailed stuff, and the vicious circle is complete.

Last week, I visited a store where they sold chairs with a land impact of 17 each. Also: flat surfaces made up of 20,000 triangles. Smoke was coming out of my 7740X/1080Ti when I walked through the store, but it it was more detailed and wonderful than most 2018-triple A games will ever be.

Edited November 26, 2018 by Arduenn Schwartzman

[Kyrah Abattoir]

The positive feedback loop is that "if it sell, more of it will be made". Most people who use SL don't see the relationship between the content they use and the performances they get (it's not me, it's the others!) or just blames Linden Lab.

The beauty of shared spaces, ultimately we are collectively responsible for it, but we don't really see it because individual impact is not perceptible.