Sculpts vs. Mesh && the Industry

[Anna Nova]

On 10/7/2018 at 3:20 PM, JasonClandestino said:

What do you think?

I think you are a dangerous teenager.

[Love Zhaoying]

On 9/26/2018 at 2:08 AM, JasonClandestino said:

so that they can upload their mesh monkey.

Is this a reference to Joffrey?

[Theresa Tennyson]

14 hours ago, ChinRey said:

There is one more. I mentioned it earlier in the thread and it may actually be significant under certain conditions.

A 1024 vertice sculpt shape requires about 12.5 Kb of raw data to be transferred from servers to client. (That's for the shape itself, not including rotation, scale, location, texturing, and any special prim property effects -these are the same for sculpts, meshes and prims). It actually only needs about 3 Kb - the rest is due to dodgy programming - and it usually uses a bit more than 16 Kb since sculpt maps tend to include alpha channels.

I'm not sure how much raw data the same shape would need as mesh but it's at least 120 Kb - almost ten times as much as the sculpt - probably considerably more.

Any prim shape can be defined by 14 bytes - 0.014 Kb of data although it probably uses a few bytes more.

That's uncopressed data but as I said earleir in the thread, the compression rate should be about the same for sculpts and for mesh.

When I joined SL in 2013, I was told by somebody who seemed to know what they were talking about that SL used very little bandwidth. These days I hear it uses more bandwidth than streamed video. I can't imagine that change is only due to the icnreased use of mesh but it must have been important and in any case, it means bandwidth limits can be a significant limiting factor on the performance. When that happens, more sculpts and less prims may well be the medicine.

Okay, what's that even supposed to mean? Leaving aside the fact that "streaming video" can be anything from low resolution to 4K, you're comparing completely different things. Streaming video sends a comparatively constant amount of data continuously, and Second Life sends a large amount of data quickly to set the scene and then uses very little data to maintain that scene. You can't directly compare them.

I'm sure that sculpts were originally used to save on data transmission, but that was also in a time where data was all sent by UDP and maximum connection speeds were far lower than they are today. Some of Second Life's beta testers were on dial-up. When I started in Second Life in 2010 I was on 768k DSL. Then, until recently I was living in an apartment with cable internet that was capped at about 5 mbps. I would have to cross my fingers to watch a streaming movie at DVD resolution on my television but Second Life ran fine most of the time.

[Love Zhaoying]

30 minutes ago, Theresa Tennyson said:

Okay, what's that even supposed to mean?

Perhaps the poster lives somewhere that streaming video is prohibitively expensive. When I visited some family in Texas, I was shocked to learn their city had 1 monopoly internet provider, who charged per usage (and did not offer an unlimited plan). For someone in such a situation, video is $$$ zomg!!!11!!

[ChinRey]

55 minutes ago, Theresa Tennyson said:

Okay, what's that even supposed to mean?

It means that SL's bandwidth use have skyrocketed the last few years and is much higher than it used to be. At least that's what I've been told - I haven't been able to check myself.

Bandwidth is one of the limiting factors in Second Life and also one that increases the cost both for users and for LL. When used intelligently well made sculpts can reduce the bandwidth requirements compared to mesh and that is still today a significant advantage.

I'm not saying that mesh is bad and sculpts are good or anything like that, most of the time well made optimized mesh does have the advantage over sculpts. But not all the time, and that is the point here. Good content creation is always and will always be about making the most out of limited resources. To do that, we need to consider all tools, techniques and materials available and not exclude any on principle or because they aren't fashionable.

 

20 minutes ago, Love Zhaoying said:

Perhaps the poster lives somewhere that streaming video is prohibitively expensive.

I don't myself actually but I know many who do.

Theresa may have preferred a Second Life reserved for herself and her close friends and would only be happy to get rid of all the riff-raff filling up the place. I have a slightly different opinion there and I suppose we just have to agree to disagree.

Edited October 9, 2018 by ChinRey

[Macrocosm Draegonne]

Transfer will always be a concern even when speeds are much higher than now, ive wondered why online games didnt take a cue from the scientific community and use a better compressible mesh like a Fibonacci hexagrid for example, it compresses to smaller size and does so faster, and same for decompression. 

Fibonacci vortex grid lattice of the "reality" variety are infinitely compressible/expandable, its how the reality hologram works, at a much higher functioning level than humans can yet accomplish. ? But even a little is useful.  Scientific communities need to be concerned with IO, storage, and running calculations with such vast datasets, this has fueled some interesting innovations in the area.

[Love Zhaoying]

Because not enough tesseract holographic fractal quantum karma cancellation to rise above the samskaras and thus view reality from outside itself, without building the outer fractal quantum holographic external reality before or outside the one the observer believes themselves to be in. 

[Janet Voxel]

What’s a tesseract?

[Theresa Tennyson]

5 hours ago, ChinRey said:

It means that SL's bandwidth use have skyrocketed the last few years and is much higher than it used to be. At least that's what I've been told - I haven't been able to check myself.

Bandwidth is one of the limiting factors in Second Life and also one that increases the cost both for users and for LL. When used intelligently well made sculpts can reduce the bandwidth requirements compared to mesh and that is still today a significant advantage.

I'm not saying that mesh is bad and sculpts are good or anything like that, most of the time well made optimized mesh does have the advantage over sculpts. But not all the time, and that is the point here. Good content creation is always and will always be about making the most out of limited resources. To do that, we need to consider all tools, techniques and materials available and not exclude any on principle or because they aren't fashionable.

 

I don't myself actually but I know many who do.

Theresa may have preferred a Second Life reserved for herself and her close friends and would only be happy to get rid of all the riff-raff filling up the place. I have a slightly different opinion there and I suppose we just have to agree to disagree.

My "what's that supposed to mean" was specifically a reference to "These days I hear [SL] uses more bandwidth than streamed video" - which is why I made it that line bold and turned it red. I also explained exactly why I had an issue with that statement.

Sculpts remind me of various interesting but dead-end engineering solutions in other fields. For instance, in the first third of the 20th century a number of car makers used Knight sleeve valve engines which were superior to the conventional poppet valve engines (i.e. the type of valves still being used in cars today) being made at the time as far as smoothness, quietness and torque range (important because transmissions of the time were hard to shift). However, eventually engineers improved the technology of poppet-valve engines and transmissions and the Knight engines had other issues that couldn't fixed so they eventually died out. 

Sculpts do indeed use less bandwidth, which was vital when they were developed, but bandwidth isn't nearly as much as an issue for most people and for those who still have issues with it relying on them would have the same value as the sheriff's badge with a bullet dent that James Garner saw when he was about to take a job as the sheriff in a lawless town in the movie "Support Your Local Sheriff".

James Garner: "It looks like this badge saved a man's life!"

Mayor: "Well, yes, it would have - if it weren't for all those other bullets."

[Macrocosm Draegonne]

1 hour ago, Love Zhaoying said:

Because not enough tesseract holographic fractal quantum karma cancellation to rise above the samskaras and thus view reality from outside itself, without building the outer fractal quantum holographic external reality before or outside the one the observer believes themselves to be in. 

lol, I am talking about a real thing though.

Graphic of Dan Winter, its not related to data compression exactly, but represents the infinite compressibility of the structure, this has many implications, but I was really speaking to the efficiency of compressing things, which the large geometric dataset folks have also found, in their own way, the same principles at play.

Edited October 9, 2018 by Macrocosm Draegonne

[ChinRey]

8 minutes ago, Theresa Tennyson said:

My "what's that supposed to mean" was specifically a reference to "These days I hear [SL] uses more bandwidth than streamed video" - which is why I made it that line bold and turned it red. I also explained exactly why I had an issue with that statement.

Just one practical tip, you can actually edit uotes and remove the irrelevant bits. That saves a bit of space on the pages and makes your replies even easier to follow than the red outline solution.

[Arduenn Schwartzman]

On 10/9/2018 at 6:42 PM, Macrocosm Draegonne said:

I am talking about a real thing though

I don't know what the hell you just said, but it sounds damn saucy.

[Love Zhaoying]

On 10/9/2018 at 12:35 PM, janetosilio said:

What’s a tesseract?

A theoretical 4-dimensional shape..I read science fiction stories (well, at least one) when I was young about them. Space folded upon itself, kinda. 

https://en.wikipedia.org/wiki/Tesseract

[ChinRey]

On 10/9/2018 at 6:42 PM, Macrocosm Draegonne said:

Graphic of Dan Winter, its not related to data compression exactly, but represents the infinite compressibility of the structure, this has many implications, but I was really speaking to the efficiency of compressing things, which the large geometric dataset folks have also found, in their own way, the same principles at play.

The key words here are "structure" and "geometric".

The datasets for polylists meshes and textures aren't exactly random but they are full of unpredictable "chaotic" factors, and I've yet to hear of any compression algorithm that can manage more than about 2:1 compression ratio for that kind of data. Streamed video and audio can do 50:1 and so can lossy image compression but lossy is what it is.

There are basically three ways to compress a set of data (yes I know I don't use the right terminology here. sorry):

Encoding

Replace long recurring strings with shorter predefined ones. Let's use a list of colors as an example:

red red red blue red green blue green blue yellow green green blue red blue

replace "red blue" with "x" and "green blue" with "y" and we get:

red red x y y yellow green y x

Quite a bit shorter (and in this case it could have been even shorter but it's just a quick example).

The problem is, we have to add data to define those replacement strings and there is no guarantee there are that many replaceable strings in the data at all. It's perfectly possible to end up with a "compressed" file that is bigger than the original. In reality, it has proven to be possible to achieve a fairly consistent 2:1 ratio for data files compressed this lossless way.

Simplification

Do we really need that one blue in the middle of all the reds at the beginning of my color list? And how about replacing the blue-green pairs with yellow? This is the principle behind lossy compression. You can get very high compression ratios that way but there is a reason why it's called lossy.

Formalisation

This isn't generally used as a compression method per se for two reasons but it's an important part of the whole picture. Here is a mesh, it's taken me hours of hard work it so I hope you all love it:

If we define this the way it's done with polylist meshes, we start with a list of vertice coordinates:

<1.00000,0.00000,1.00000>,<0.98079,0.19509,1.0>,<0.92388,0.38268,1.0> .... oh, never mind - 29 more vertices to list that way.

Or we can define it as:

circle, flat on z axis, 1 m diameter, center <0,0,1>, 32 vertices.

That's essentially how prims work and as I mentioned earlier, we can easily achieve a 1,000:1 or higher "compression ratio" with prims rather than meshes.

Formalised data is also very suitable for further compression by simplification with a minimum of loss. Change the number of vertices to 8 in my list of paremeters - there's your LoD model. (The actual vertice numbers for the four LoD models in SL would be 24, 18, 12 and 6 btw.)

The reason why this kind of formalisation isn't really used for automatic compression is that "reverse engineering" to identify such underlying structures is a lot of work for a poor computer and data sets that lend themselves well to this method are usually already formalised - they are "pre-compressed" if you like.

Now, prims are of course very simple formalised objects and even though they aren't nearly as limited as many seem to believe these days they are indeed limited. But they don't have to be, in fact they weren't always. The early experimental versions of the prim system included functions that had to be taken out because they were to processor heavy back then (probably correct but they can still be far less heavy than the polylist substitutes we use today). Some other functions are - in principle at least - very easy to add to the existing system.

And that's still only the start. Prims are for the most part based on three principles for making procedural geometric shapes: sweeps, extrution and some very, very basic lofting. Add more advanced lofting, nodes, fractals, quasi-random parameters like perlin noise, even some basic CSG principles like volume subtraction and intersection...

Not all of it of course because even 15 years after prims were launched, we'd soon run out of cpu power. But a lot can be done even with some fairly simple upgrades compared to them old prims. The 3D industry as a whole is busy developing new cool procedural tools both for 2D and 3D graphics. LL is the exception. They did add PBR to Sansar but as far as I can see, that's all. They seem to have missed the train and are left at the station while everybody else are racing ahead.

The conclusion to this long rant: There is a limit to what can be done with compression algorithms and SL does actually seem to use those quite effectively already. If we really want to do something to the data stream, we have to find ways to reduce the amount of data we feed into it in the first place.

Edited October 15, 2018 by ChinRey

[Macrocosm Draegonne]

3 hours ago, ChinRey said:

There is a limit to what can be done with compression algorithms and SL does actually seem to use those quite effectively already. If we really want to do something to the data stream, we have to find ways to reduce the amount of data we feed into it in the first place. 

Wow thanks for that detailed info, I was not aware of those aspects of prims, thats actually very interesting.   I wasnt really speaking against what SL does though, more in general and future tense as literally nobody but high end geometric data crunchers are implementing the sacred geometry structure for compressibility/fidelity benefits.  When your model has 10 billion vertices, and you have to quickly swap it from drive, to ram, to app, and store a hundred thousand years evolutionary algorithm permutations on it, it begs solutions from the primordial ooze... lol 

For games I see this eventually leading to faster delivery of mesh on all fronts, network, local, graphics processing, etc, but even more interestingly an automatic LOD that works more like reality does.  As you get closer to something in scale more of the internal dodeca zones load in, being the golden measure it will all feel quite visually immersive and look real too....  Yes... my Monday coffee is kicking in lol! ?

Sansar does have a nicer feel with the PBR, Id love to have that here some day, but they'd have to support legacy and new stuff, must be possible, no?  Seems like theres new blood in SL or more fire, im always hearing about new features and the tons of projects going, seems they're on the upswing for evolution and modernization.

Edit:  A little more to the topic at hand, reducing bandwidth, why not include a larger default texture set in the viewer? right now theres only some normals, specs, and a two blanks AFAICT ...and with this larger built in set, give an li discount to mesh that use only those textures.  Could curb a little bandwidth usage and cause some interesting creations to sprout up.

Another thing is shaders, give us shaders, many shaders means less textures, at least far far more that can be done with less textures, and even infinite things with zero textures.  Im currently grappling with terrain mesh and how to layer in the details I want, not having a shader to use multiple tiling textures that can be dispersed & blended by height or weight paint for instance, will cause me to increase my mesh layers, textures needed, and time needed to pull off a desired look.  All that could be gone with allowing sets of 1024 images (with individual tileablility) baked into a singular texture atlas, and some creative shaders.  Those atlases could also save some request calls & viewer strain at least, if not bandwidth.

Edited October 15, 2018 by Macrocosm Draegonne

[ChinRey]

2 hours ago, Macrocosm Draegonne said:

A little more to the topic at hand, reducing bandwidth, why not include a larger default texture set in the viewer? right now theres only some normals, specs, and a two blanks AFAICT

I think there are about 50 textures hardcoded into the viewer. That is, I'm not sure how "hardcoded" they are. Their UUIDs are in the viewer software or some of those XML files associated with it but the assets themselves may not be.

Here's one of them:

Apparently this was one of the normal maps they tried for system water but it was rejected in the end. I'm not sure why but it may be that a rectangular rather than quadratic map turned out to be impractical.

These two could actually have been quite useful if people had known about them:

They are 1024s though and that seems a little bit excessive...

UUIDs are 49085953-8c5c-1cc3-a971-bd19c17be6f5 and 2b8d91e4-a106-6b5b-d492-749a1dbf9121 in case somebody want to play with them.

How do I know all of this? The Whitecore developers are brilliant programmers but they are not content creators at all. When they made the on-a-Stick version, they were desperately looking for something - *anything* - to put into the library folder. So they added all those hardcoded textures there.

Edited October 15, 2018 by ChinRey

[Macrocosm Draegonne]

9 minutes ago, ChinRey said:

I think there are about 50 textures hardcoded into the viewer. That is, I'm not sure how "hardcoded" they are. Their UUIDs are in the viewer software or some of those XML files associated with it but the assets themselves may not be.

Ahh, I see, they may not actually be in the viewer itself   Those are some nice normals though!  and wtf on those furry asses lol? ?

Well, all the more reason to add a nominal collection of textures to the viewer, any of those textures would be zero bandwidth to load, and the more abundantly/creatively they're used the least amount of texture calls in the viewer too.  Just an idea, Im not making demands or anything lol.  More shaders, and texture atlas baking would be biggies too of course!

My main point is that there are ample ways to reduce bandwidth, increase performance, and immersion, given how tech has progressed.  Linden is in a sweet spot too, they have a solid established base with staying power, they can pick and choose from what best-of's to benchmark off of in the industry and slide it in, while all the others struggle to gain even a tiny fraction of SL's footprint, they will never catch up so long as LL knows when to eb & flow with innovations and evolutions.

Edited October 15, 2018 by Macrocosm Draegonne

[Janet Voxel]

6 hours ago, Love Zhaoying said:

A theoretical 4-dimensional shape..I read science fiction stories (well, at least one) when I was young about them. Space folded upon itself, kinda. 

https://en.wikipedia.org/wiki/Tesseract

I know Love, it’s from A Wrinkle in Time. 

I thought it was a chapter title or a foreword or something. I remember it being in big letters somewhere like that: WHAT’S A TESSERACT?

Edited October 15, 2018 by janetosilio
Apparently it’s not a chapter title!

[Love Zhaoying]

22 minutes ago, janetosilio said:

I know Love, it’s from A Wrinkle in Time. 

I thought it was a chapter title or a foreword or something. I remember it being in big letters somewhere like that: WHAT’S A TESSERACT?

Yep, I forgot it was in the movie I saw just last year. Never read the book. Also in other stories..was surprised Wikipedia had a “real” entry for it.

[Whirly Fizzle]

On 10/15/2018 at 5:29 PM, ChinRey said:

Here's one of them:

Apparently this was one of the normal maps they tried for system water but it was rejected in the end. I'm not sure why but it may be that a rectangular rather than quadratic map turned out to be impractical.

That's the normal map used for Torley's "[TOR] Arrakissed variation" water windlight.

[animats]

On 10/9/2018 at 9:35 AM, janetosilio said:

What’s a tesseract?

Just a cube in 4 dimensions. Tends to be called a hypercube of order 4 in mathematics; the word "tesseract" shows up mostly in fiction.

[ChinRey]

2 hours ago, Whirly Fizzle said:

That's the normal map used for Torley's "[TOR] Arrakissed variation" water windlight.

?

It actually is!

[Macrocosm Draegonne]

On 11/10/2018 at 4:10 PM, Whirly Fizzle said:

That's the normal map used for Torley's "[TOR] Arrakissed variation" water windlight.

I love Torleys stuff!  I have a bunch of textures he made back in the day, so much wild variety and they can be used for a lot of different effects n such.  I found him over in Sansar is that the same Torley?