Prim Equivalence Explained

[Runitai Linden]

There's been a lot of discussion lately about prim equivalence, so I thought I'd post some of the principles that have been guiding the implementation of the streaming cost metric.  There's a lot of text here, but please read the whole thing before responding, as it starts off with some statements that seem bald face whacko without further explanation.

First and foremost, the current implementation is far from final.  This is evidenced by the fact that the number in the upload floater, the number in the streaming cost debug display, and the number in the build floater all disagree with one another.  The last few months have been spent getting accounting code and asset formats into a place where we can fine tune the cost metrics.  We've only just begun fine tuning the cost equations.  To help, please build as much as you can.  I can't stress this enough.  The more content we have to examine and analyze, the better the final cost formula will serve us all.

Second, meshes are not prims.  They LoD differently, transmit differently, tend to be different sizes, etc.  There is no analog between meshes and prims as far as triangle count or streaming cost.  They are completely different types of content.  The argument of "prim X has this many triangles so mesh Y should cost this many prims" has no merit.  Don't even get me started on sculpts.  The argument that some people will continue to use sculpts and prims instead of meshes specifically for an increase in triangle count at reduced prim cost, while completely true and valid, has no bearing on this discussion.  Please read on and I'll explain why.

So what does "Prim Equivalence" mean?  Well, it means 1/15000th of a region's available resources.  Abandon the idea that "Prim Equivalence" means an "equivalent complexity compared to a prim."  It's an "equivalent parcel cost" compared to one prim.  So what is the "Prim Equivalence" trying to accomplish?  It's about budgeting.  The task now is to define a realistic resource budget for an entire region and adjust the physics, simulation, and mesh cost equations so that a cost of "15000" equals this budget.  

Here, I'll talk about "streaming cost."  "Streaming cost" more or less translates directly into a "visible triangle cost," given that the number of triangles visible from any given point on a sim is directly proportional to the number of bytes that need to be transferred.  The goal of "streaming cost" is to limit the number of visible triangles to a realistic budget at default settings, and that budget is much smaller than you might think compared to overall scene budget.  

Here are some examples:

Abbott's Aerodome: 

- 3.7 million triangles from prims

- 500 thousand triangles visible from any given location

Lusk:

- 2.8 million triangles from prims

- 218 thousand triangles visible from any given location

Gibson:

- 900 thousand triangles from prims

- 150 thousand triangles visible from any given location

Now, there are a lot of factors that play into performance other than triangle counts, but for the sake of simplicity, we're ignoring those for now.  If you explore built out regions in Second Life, you'll find similar numbers (use the "scene statistics" console to check for yourself).  The majority of built out regions seem to fall in the 200-400 thousand triangle range.  While it's true that prims are a completely different content type than meshes, this "visible triangles" number gives us a realistic target for predictable performance.

Given that Second Life has bad framerates and prims aren't exactly stingy about wasting triangles, it's wise to bias meshes towards the low end of that spectrum, say at 250 thousand triangles give or take 50 thousand.  THAT is your visible triangle budget.  In order for Second Life to have a prayer of running well in a mesh build, the meshes MUST display no more than that number of triangles per frame.  

So how does the streaming cost ensure that?  To be blunt, it doesn't.  No single metric can.  Object sizes, placements, and camera position all have a great effect on how many triangles you might see for any given frame.  The streaming cost merely provides an approximation for how much of this 15000 budget will be consumed by a single instance of the mesh in question.  Put another way, it tries to ensure that if a region were filled to capacity with instances of the mesh in question distributed evenly at ground level, the number of visible triangles per frame would closely match the specified triangle budget.

What this means in practice is that triangles added to the high LoD are very cheap, while triangles added to the low LoD are very expensive (except for large meshes where the high lod is displayed over a large area).  This is because the low LoD tends to be displayed over a MUCH larger area than the high LoD, and thus this is the triangle count that will deduct from the 250 thousand triangle budget from most vantage points in the sim.

So, be very aggressive about removing triangles from lower LoDs, build lots of stuff and focus discussion on concrete examples of mesh assets that are being penalized unfairly against the stated triangle budget, resist the temptation to compare meshes to sculpts and prims, and fret not that we're destroying mesh with high costs.  Do this, and two weeks from now we should be in a much better place with respect to prim equivalence.

[Dain Shan]

Hi there

Many thanks for the explanation. Any input is very welcome and needed.

However, i still have a hard time to wrap my brain around the concept.

 

Essentially you say here that the creators of Meshes have to deal with more restrictions in regard to resources then the average prim or sculpt user. For the sake of better overall performance?

 

That sounds .. odd.

Forgive me for being blunt here but that calculation will NOT work.

Essential we already know that the majority of People out there don't give a damn about resources other then prim count.

Its the only measure they have right in front of their eyes. They don't care about KTris drawn or Polygons on screen. They don't care if a sculpt has 10 times the polygoncount .. as long as it looks good and only has one prim against the parcel count. We stated this over and over and it is tiresome to see yet again a comment, that we should forget about the prim equivalent.

 

It simply not true, in the end it will be all that matters.

How many prims will that mesh take from my parcel or sim allowance.

That's all.

 

As you might already have see, most of the mesh creators out here are already building with the resource count in mind.

Means they optimize their stuff to a great extend. But the restrictions are still set in a way that we can't understand, especially if they make no sense whatsoever.

You gave a very nice example here. 1/15000 of a sim, so that IS exactly 1 prim till now. ( We leave that sculpt debacle off the side here )

Ok lets see we have an empty sim, and 1 rezzed standard cube. As you probably already read here:

http://community.secondlife.com/t5/Mesh/Linden-Magic-Math/td-p/921425

It is totally not understandable why a mesh with 1/10 the resources of a simple cube ( Yeah right only 10% of an ordinary cube ) costs twice as much as the the cube in regard to the prim equivalent.

You can expand this. There are even more builds out there who have much less triangles then their prim counterpart, so essentially were helping you already with the performance. We make things that costs far less resources then even the standard original Linden prims.

Why the restrictions that punish the mesh creators ?

Even more so that, if we stop making meshes and flood the world with sculpts instead, you would get a far bigger impact to the negative side of things.

 

There are still a few points in regard to meshes that still need a good overhaul. But thats another story and has nothing to do with ressource costs.

[Runitai Linden]

Please resist the urge to compare meshes to prims in any form.  I realize people will still use prims and sculpts to save on parcel limits, but that has no bearing on this discussion.  Please limit the discussion to triangle budgets.  In this case, would copying that mesh up to the prim limit of the region and distributing it evenly at ground level result in a greater than 300 thousand triangles being visible?  

If yes, then the streaming cost is doing its job and the mesh itself probably needs to reduce the number of triangles used in lower LoDs. 

Will the resulting triangle count be under 200 thousand?  If yes, then that indicates a bug with the streaming cost that needs to be fixed.

There are so many variables that go into that number it gets confusing, so just focus on the result -- a target triangle budget of 250 thousand triangles visible from the center of the region if the region was full to capacity with copies of that mesh distributed evenly at ground level.  Here, "visible" means visible from the center, not all in frame from the center.  Fill a sandbox up with your mesh and stand in the center, do a full rotation to get all the lods to resolve, and look at the scene stastics console (Develop->Consoles->Scene Statistics).

 I should also note that these numbers (200 thousand - 300 thousand triangles) are not final and should not be interpreted as a promise to tune the equations to that specific budget.  It could go higher, it could go lower, but we need more data to tell what's appropriate.

 

[Dain Shan]

Alright ..

Theoretical example..

A cube standard Prim 120 triangles. Copied 14999 times.

That's 15000 x 120 triangles = 1800000 Triangles as a whole

 

Mesh cube 2 triangles per side = 12 triangles

copied 14999 times

That's 15000x12 triangles = 180000 triangles

 

Even if all cubes are in the visibility range, and even if that is a highest LOD, you are far below you 200000 limit.

 

We stretch that out a bit.

We bit a BIG quadrate with this 15000 Cubes side by side

So we have as we can say roughly a side builded from ( i round up here ) 123 cubes

That side would have an lenth of 123x0.5m so 61,5 meters.

Places in the middle of the sim and assuming a view angle of 90 degrees (far Higher then the actual view angle is ) we can assume that you see about 50% of this cubes.

That means also we can see cubes in a distance up to ( since we only see 50% of them ) to 61,5 / 2 = 30,75 meters.

LOD in High is as mentioned before Diameter of the mesh x2 in meters.

Means every cube up to 1 meter away from us is in High LOD. ( If i count that right? ) rest in middle or even low LOD.

For the sake of an Argument lets take the absolute possible worst case .. All of them are in HIGH LOD

( Impossible i know, but just lets assume )

So .. with all this bad and far too high numers we see 15000 / 2 cubes with 12 triangles each.

That 7500 x 12 = 90000 Triangles .. not even half of the limit 200000

 

 

 

[Drongle McMahon]

I will make three points.

1. I understand the rationale for the streaming cost calculation and I think it makes good mathematical sense.

2. I think the effect of the size of mesh objects, which controls the area over which they appear at different LODs, is not immediately obvious. This leads to some of the perceived problem just because you can agglomerate a lot of detail in a big mesh that you would once have done with many smaller prims. That makes the high LOD remain at much greater distance with the mesh than with the collection of prims. That is why we are astonished by some of the costs.

There are two things that you can do in response. The numbers here may change, but for now a 40x40x40 (r=34.66) mesh is going to appear only as the highest LOD wherever you are in the sim. The lower LODs are irrelevant to resource consumption and so the lower LODs will not save you anything in prim cost. 

Between 10x10x10 (r=8.66) and 40x40x40, only the next, medium, LOD will have any effect and that effect will reduce as you get nearer the top of the range. The medium LOD will only be seen a long way away. That means you can use very aggressive triangle reduction. It should be like the lowest auto-LOD.  That's 1/64 the number of triangles. The lowest two LODs will do nothing.

The next range is 5x5x5 (r=4.33) to 10x10x10. Here, the low LOD becomes more and more important as you shrink. So you can be more generous with the medium LOD, which is now seen from nearer to, and keep the stringency for the lowest. Below 5x5x5, the lowest LOD becomes more important, and below 1m is dominant.

The second, and more effective thing you can do is to break your big mesh into smaller meshes as far as that is sensible. The reduction should be in all dimensions, preferably to well below the 5m mark where the lowest LOD can take effect.. This has obvious drawbacks, as you have the painful process of assembling them, and it obviously creates more overhead for the same amount of mesh. But it should result in the expected progressive LOD switching at the closer distance the system was designed for, and consequent prim cost savings.

It would be interesting to have some examples to see how well that works. (Oh dear, more work). It will depend a lot on the complexity left in the broken-down pieces, but my guess is that it should allow much more lower-level detail at then same time as lower overall prim cost.

Underlying this is the fact that the LOD switch on the basis of distance/radius ratio is entirely suitable for prims, but does not work effectively for meshes with highly variable scales of detail. I am going to make a feature request for an object-specific multiplier to be applied to the radius so that the LOD switch distances can be controlled without having to disassemble big meshes, preferably on rezzed objects, but at least at upload time. I think that is the long-term solution and is quite simple to code, but I don't imagine there is time to do it before release.

3. (/me restrains himself) While discussion of meshes in isolation is very useful, it is my opinion that it is dangerous to ignore comparison with sculpties. The latter will be used instead of meshes if they are allowed, especially at 64m, without further restrictions. This is a very real problem for the uptake of mesh that threatens the resource efficiency and visual quality improvements that are otherwise offered. I will not say more, as requested.

PS. I guess the requested resp[onse tom the sculpty comparison is that a sim full of sculpties will not reach the performance required. So if people do fill up their sims with sculpties, that is their problem. We can't stop sculpties messing thins up, but we can stop meshes doing the same. That way people will eventually reject the sculpties?

PPS. What about attachments? (/me runs away).



[Gaia Clary]

Hi.

Thank you very much for the clarifications on this theme! I read through it and i think that i can understand all arguments very well. I also see that none of the current values in the displays are final.

Let me make an extreme example and ask some questions to better understand all this. I admit this is surely of no practical value but maybe it can help to clarify more :

 

• Assume you had the simplest possible mesh, a single triangle.
• Since it is a single triangle, there is no way to reduce it further without making it invisible, so LOD2,1,0 are set to not applicable.
• Now take this mesh to ground. We see one single triangle from any point in the region and from any distance.

Let me citate a few parts of your article hoping that i extracted the essence:

---

Runitai Linden wrote:So what does "Prim Equivalence" mean?  Well, it means 1/15000th of a region's available resources.  

...

 "Streaming cost" more or less translates directly into a "visible triangle cost," given that the number of triangles visible from any given point on a sim is directly proportional to the number of bytes that need to be transferred.  The goal of "streaming cost" is to limit the number of visible triangles to a realistic budget at default settings, and that budget is much smaller than you might think compared to overall scene budget. 

. . .

What this means in practice is that triangles added to the high LoD are very cheap, while triangles added to the low LoD are very expensive (except for large meshes where the high lod is displayed over a large area).  This is because the low LoD tends to be displayed over a MUCH larger area than the high LoD, and thus this is the triangle count that will deduct from the 250 thousand triangle budget from most vantage points in the sim.

---

This would mean more or less, that if i place 250000 of my triangles on the ground then i would match the allowed streaming costs

...

Now let us look at another mesh which contains 64 triangles, lets say in a flat plane in the shape of a triangle. Now i define the LOD3,2,1,0 with 64,16,4,1 triangles and place this mesh on the ground.

From the resource costs assumption i could now use ~3900 of these meshes to match the 250000 triangle limit if i could see all my objects from somewhere. but due to enhanced LOD i am now by far better than with my single triangle mesh. So i assume i can place many more of my objects on the region than the 3900 objects with 64 triangles on each.

When taking into account that LOD0 shows 1 triangle, i think that placing 250000 of these objects into the region would still match the "250000 triangles from each location" constraint. This is an approximation. Maybe the true upper limit would be less...

Now here are some questions raising:

• So this means that meshes with few triangles will result in higher streaming costs (assuming that on each LOD we must display at least one single triangle)... is this correct or wrong ?
• And if the former is true, then meshes withmore triangles can become more cost efficient because we can reduce them better. Is this correct or wrong ?
• If we would reduce the number of triangles by a power of 3 for each level, then from the streaming cost presumptions meshes with 512,64,8,1 faces on LOD3,2,1,0 should be (in average) the most efficient meshes we can have ? Is this right or wrong ?
• The resource costs displayed during upload are meant as "this is the fraction of your 15000 budget that will be needed to rezz this object with the given scaling", correct or wrong ?
• Sorry, i am so curious but i realy want to know if we can have more than 15000 mesh objects on a region. From what Nyx told us last monday , this should be possible (he told us he could place 24000 spheres into a region).

[Drongle McMahon]

Gaia, can you clarify by what you mean by "small" and "larger" in this context ...

So this means that small meshes will result in higher streaming costs (assuming that on each LOD we must display at least one single triangle)... is this correct or wrong ?

And if the former is true, then larger meshes can become more cost efficient because we can reduce them better. Is this correct or wrong ?

Are you talking about fewer / more triangles, or smaller / larger in overall dimensions in meters? Just so I can understand the answer when it comes.

[Gaia Clary]

smaller and larger regarding the number of triangles per mesh. I just corrected this in my previous post ;-)

[DanielRavenNest Noe]

For discussion purposes we might want to use "complex" and "simple" models to refer to triangle count, and "large" and "small" to refer to the dimensions in meters.

[Gaia Clary]

While experimenting a bit further i came across this issue:

 

Both kettles are mesh objects. Both kettles have exact the same appearance, the same LOD decimation, everything is equal, exept :

 

1. I joined the 2 objects into one mesh object (i did that for each LOD separately)
2. i simplified the physics mesh

Here is the result:

 

Kettle	optimized (left side)	original (right side)
type	mesh	mesh
objects	1	2
faces	1280	1024+256
Physical objects	1	2
faces of phys. obj.	16	45
complex hulls	1	2
size	0.49,0.47,0.32	0.46,0.46,0.31 (kettle) 0.48,0.25,0.16 (handle)
Prim Cost	5	3
Streaming cost	7.2	3.2 + 4.4
upload cost	100L$	110L$

 

Maybe that needs either a fix or another explanation.

I am also not at all sure about what "upload cost" means here (Develop->Show Info -> Show Upload Cost).
This probably is also not fixed yet ?

[Drongle McMahon]

That seems very odd. I don't bthink it can be explained by size dependency of LOD.* Can you tell us what the streaming cost is reported by Develop->Show Info->Show Render Info are? It's on the next to bottom line, labelled selection streaming cost when you select the object. I think that is likely to be more sensible now.

*or can it.? Which has the most triangles, the pot or the handle? If it's the handle, then those triangles will be now part of a mesh with a bigger radius, and that would push out it's LOD distance and increase cost. That would be an excellent example of lowering cost by splitting the mesh as I was talking about above. Do you have the dimensions and Show Info version of the streaming costs for the two separate pieces?

[Gaia Clary]

I updated the table.

[guY Ralior]

Thank you so much for the explanation, I am still trying to completely understand but it seems like there is more to the problem to be resolved

[Drongle McMahon]

Ah. Those values from Show Render Info make sense, just about add up with a bit rounded off. Th radiuses of the separate meshes are not different enough to explain the differnces, I think. So it looks like we have to ignore the edit floater values for the time being. Could be the top value is showing only one of the meshes when it should be the sum. The Show Upload Cost thing is for the L$ upload charge. I think the mesh costs are fictitious, just seeing that the gadget works (or at least I hope so). When you have nothing selected, I guess they are for the whole region.

[Drongle McMahon]

Here is a new jira feature request relevant to this thread, in case anyone is interested. It is supposed to solve the problem of high prim equivalence for large complex meshes without cutting them up. Probably too late for release, but maybe we can hope. CTS-613 Adjustable LOD factor for meshes..

 

[Dain Shan]

Voted on ...

It would clear some important issues

 

[arton Rotaru]

I have a question about the testing case you described. You say fill the region up to it's prim limit with our meshes. Currently the region counts the number shown in the edit floater against the parcel/region prim limit. That means, I can copy about twice as many (and more) meshes on the region than I if would take the number shown in 'Selection Streaming Cost' for the given mesh.

So which number should we take to count up to the 15000 prim limit, to make the budget tests you mentioned above?

I apologize in advance, if I misunderstood something here, but I'm not a native english speaker, and I probably have a harder time reading and understanding all this.

[Ashasekayi Ra]

Thanks for that explanation. So, if an LOD has 3 times less triangles than the one above it, the object will have less prims than say an LOD that is only 2 times less?

[DanielRavenNest Noe]

Here is an example using agressive LODs:

Small Palm Tree

It's just under 1000 triangles, and uses the default medium LOD (about 250 triangles).  For low and very low I used two pairs of triangles with an alpha texture of the palm (so it could be seen from all sides) + placeholder triangles for the high LOD textures, so they come in at 6 triangles.  I used the low LOD model for the physics also.  This measures 2 prims on the edit window, and about 1.5 on the upload window.

When viewed from medium to long distance, you can get away with a "billboard" flat texture.  "Billboards" are often used in other graphics rendering to replace full models at longer view distances.  Commonly they are a single plane with an alpha image of the object. Since SL does not rotate billboards to face the camera (that I know of) we need to use several so it will be visible from all directions.

[Runitai Linden]

---

Dain Shan wrote:

So .. with all this bad and far too high numers we see 15000 / 2 cubes with 12 triangles each.

That 7500 x 12 = 90000 Triangles .. not even half of the limit 200000

 

---

THIS is exactly the kind of feedback I'm looking for.  Something isn't adding up here, so I took a look, and this is what I found out:

Not all triangles are equal -- the streaming cost assumes an average of 10 bytes per triangle, but on low poly or faceted objects, many vertices are used in only one triangle, so the average triangle size (in bytes) for those objects is often much more than 10 bytes, not to mention the overhead of the mesh header and LLSD tags, which can outweigh the size of a tiny mesh.  It's fair to charge more for these for a few reasons:

1. The extra data has to be streamed, creating overhead.

2. An extra HTTP request must be made for a tiny amount of data, creating overhead.

3. Actual render time performance of these objects is worse per triangle due to a high number of post TnL cache misses.

#3 will vary from machine to machine, and will be much more noticeable on low end machines.  As for 1 and 2, this could cut both ways -- using one small mesh multiple times will load better and require less bandwidth than baking multiple instances of that same mesh into one big mesh.  The streaming cost already ignores the header when determining cost, and the overhead of the LLSD tags is constant, so a deduction could be made there.  As for the extra HTTP request, that's a protocol issue builders have no control over, so it's really not fair to have that cost against you, and future protocol improvements could theoretically stream multiple meshes with one HTTP request.

Thanks for the example!  I'll need to see some large mesh builds to get a better idea of actual bandwidth usage and file sizes, but in the future we'll probably start talking about a visible mesh data size budget instead of a visible mesh triangle budget.

 

 

 

 

[Runitai Linden]

---

Gaia Clary wrote:

Now here are some questions raising:

1. So this means that meshes with few triangles will result in higher streaming costs (assuming that on each LOD we must display at least one single triangle)... is this correct or wrong ?

2. And if the former is true, then meshes withmore triangles can become more cost efficient because we can reduce them better. Is this correct or wrong ?

3. If we would reduce the number of triangles by a power of 3 for each level, then from the streaming cost presumptions meshes with 512,64,8,1 faces on LOD3,2,1,0 should be (in average) the most efficient meshes we can have ? Is this right or wrong ?

4. The resource costs displayed during upload are meant as "this is the fraction of your 15000 budget that will be needed to rezz this object with the given scaling", correct or wrong ?

5. Sorry, i am so curious but i realy want to know if we can have more than 15000 mesh objects on a region. From what Nyx told us last monday , this should be possible (he told us he could place 24000 spheres into a region).

---

1 & 2 -- In general, any mesh that can be reduced to a lower number of triangles will be cheaper than a mesh that cannot be reduced, depending on size and reuse of vertices.

3. If by "efficient" you mean the most triangles in the high LoD for the lowest cost, then yes, that would be very efficient.  

4. Yes, and if everything's working correctly, the number in the upload floater should match the number in the build floater after first rez -- it currently does not, which is a bug.

5. Yes, you can have more than 15000 objects per region provided you use the new physics options appropriately (link non-physics meshes together and set their physics shape type to "none."  However, this is unlikely and probably undesireable for meshes, as one of the benefits of meshes is less scene fragmentation, as we get with prims.  Ideally you'd want to find a balance between keeping the number of objects low, keeping the number of visible triangles low, and keeping the overall aesthetic quality high.  It's all very yin and yang.

 

[Runitai Linden]

---

Gaia Clary wrote:

While experimenting a bit further i came across this issue:

 

Both kettles are mesh objects. Both kettles have exact the same appearance, the same LOD decimation, everything is equal, exept :

 

1. I joined the 2 objects into one mesh object (i did that for each LOD separately)
2. i simplified the physics mesh

Here is the result:

 

Kettle	optimized (left side)	original (right side)
type	mesh	mesh
objects	1	2
faces	1280	1024+256
Physical objects	1	2
faces of phys. obj.	16	45
complex hulls	1	2
size	0.49,0.47,0.32	0.46,0.46,0.31 (kettle) 0.48,0.25,0.16 (handle)
Prim Cost	5	3
Streaming cost	7.2	3.2 + 4.4
upload cost	100L$	110L$

 

Maybe that needs either a fix or another explanation.

I am also not at all sure about what "upload cost" means here (Develop->Show Info -> Show Upload Cost).

This probably is also not fixed yet ?

---

The "upload cost" debug display is kinda useless right now, I wouldn't worry about that yet, and the "prim cost" not matching "streaming cost" is a bug -- go with "streaming cost" for now.

[Runitai Linden]

---

arton Rotaru wrote:

I have a question about the testing case you described. You say fill the region up to it's prim limit with our meshes. Currently the region counts the number shown in the edit floater against the parcel/region prim limit. That means, I can copy about twice as many (and more) meshes on the region than I if would take the number shown in 'Selection Streaming Cost' for the given mesh.

So which number should we take to count up to the 15000 prim limit, to make the budget tests you mentioned above?

I apologize in advance, if I misunderstood something here, but I'm not a native english speaker, and I probably have a harder time reading and understanding all this.

---

Use the "streaming cost" number in show render info and make it add up to 15000 for now -- hopefully we'll get a sim out soon that agrees with the viewer as far as streaming cost is concerned.

[Runitai Linden]

---

Ashasekayi Ra wrote:

Thanks for that explanation. So, if an LOD has 3 times less triangles than the one above it, the object will have less prims than say an LOD that is only 2 times less?

---

Exactly.  If you graph it, the cost gets exponentially lower until you hit a ratio of around 4x.

[Ashasekayi Ra]

Thanks. That part makes more sense now.