The Lab ending work on mobile viewers

[Madelaine McMasters]

10 minutes ago, Fluffy Sharkfin said:

It may still be that you're locked to your monitor refresh rate.  Apparently it's not uncommon when running in full screen and there are several posts on reddit and nvidia forums about the issue (some are quite old but then so is SL).

144hz - Most Games Capped at 60 FPS in Fullscreen

I only skimmed through the thread but there do seem to be some suggested fixes that have allowed people to achieve higher frame rates when running full screen.

Stop making me work, Fluffy!!! Here are two screenshots at medium window size. Speed scales with window size from about 60fps full screen to 130fps when the window goes so small the fps text gets trampled by other stuff in the menu bar.

Hi Rez, 98.6fps.

Lo Rez, 100.0fps

I watched the fps meter for a few seconds for each of those scenes and they were actually closer together than the snapshots indicate. I call it "no difference".

[Cali Souther]

39 minutes ago, Madelaine McMasters said:

 

 

Is that a burning heart?  I've heard of a bleeding heart but....

[Fluffy Sharkfin]

5 minutes ago, Madelaine McMasters said:

Stop making me work, Fluffy!!!

Hehe, sorry Maddy, I'm just pedantically curious at times! I promise I'll stop now. 😅

I wasn't disagreeing with your observations, just intrigued by the results.  The figures definitely suggest that texture load is far from the biggest cause of poor performance in SL.

[Ardy Lay]

Hey, cowboy.  Proof is in the pudding.  Put your money where your mouth is.  Show us your dog can run, put it in the race.

[Madelaine McMasters]

2 hours ago, Cali Souther said:

Is that a burning heart?  I've heard of a bleeding heart but....

It's a piece of reactive artwork I made back in 2008...

 

[Madelaine McMasters]

2 hours ago, Fluffy Sharkfin said:

Hehe, sorry Maddy, I'm just pedantically curious at times! I promise I'll stop now. 😅

I wasn't disagreeing with your observations, just intrigued by the results.  The figures definitely suggest that texture load is far from the biggest cause of poor performance in SL.

No worries, Fluffy. I love to rant.

And ramble.

Eventually I'll just set you on fire.

The following crude explanation is not directed at you so much as anyone who reads the thread. I'm making this all up from my general knowledge of computer graphics. All Y'all feel free to correct anything I got wrong.

The viewer uses JPEG2000 for its progressive download ability, which we see on log-in or TP as textures "come into focus" while we wait. I imagine the viewer fetches only up to the resolution it needs (maybe it goes all the way?). If scene requirements change, the viewer could request a higher resolution layer. All the while it saves everything it can fit into cache to build a local mip-map.

Once we have a texture at the proper scale for the triangle we wish to texture, it's the size of the triangle affects the texturing time, not the size of the texture. The texture mapping engine will only fetch those pixels from the texture that map to the triangle. This is why fps scales with viewer window size, not texture size.

Rendering speed also depends on scene geometry, the complexity of which is only roughly analogous to texture resolution. Triangles must be set up for texturing, taking a fixed time per triangle. That time is proportional to scene complexity, not window size. The number of triangles in a cube is the same whether it fills the scene or occupies only one pixel. If more complex geometry (avatars!) enters the scene, fps will drop.

There is no JPEG2000 equivalent for geometry compression, so we encourage designers to create objects at various levels of detail (LOD) so that we can substitute simple geometries for complex ones as we back the camera away. Just as the use of smaller textures saves memory space, the use of smaller geometries saves rendering time.

So, approximately...

Texture resolution has minimal impact on rendering speed, but does affect usage of precious memory.
Geometry resolution has significant impact on rendering speed, adding lower LOD models probably has a small impact on memory.

You want low resolution textures to save memory space, low resolution geometry to save time.

 

And, of course, textures and geometry are just two factors in rendering time. There's also lighting, transparency, shadows, reflections, etc.

The SL viewer currently describes all of these things in a way OpenGL understands. OpenGL is fading away, and that will require potentially significant rethinking of the render pipeline. The render pipeline is only a part of what the viewer does. That data management stuff Coffee mentioned can still kill things even if the graphics are whizzy. As I've seen from watching my system monitor tools, SL uses only one or two of my Mac's eight CPU cores.

Edited April 19, 2022 by Madelaine McMasters

[Coffee Pancake]

14 minutes ago, Madelaine McMasters said:

You want low resolution textures to save memory space

Due to the way the viewer beats the tar out of your GPU with draw calls.. 

You want a single HUGE textures that's shared between multiple objects.

Eg, you buy a coffee service .. Each cup having it's own individual texture is bad, even if they are all tiny. Having all the cups on a single huge texture is not only faster to process and decode, but faster to render too.

 

SL dies a death due to a thousand thousand tiny cuts.

 

JPEG2000 textures are pulled in a steps, get some data, hope its enough to get something we can use, if the object is big enough on screen to need more, get more data, rinse repeat. 

The viewer wont load full resolution textures for everything automatically, but it also doesn't toss data out once it's been forced to load the full thing (by zooming up close to everything).

It also wont ever load a full resolution anything when the avatar and camera is in motion. This is optimal for exploring the world, not so good for events.

The fastest way to get the viewer to load something you want to see (like a vendor) is to focus your camera on it, and then stop moving!

Edited April 19, 2022 by Coffee Pancake

[Coffee Pancake]

19 minutes ago, Madelaine McMasters said:

Geometry resolution has significant impact on rendering speed

Only if it's worn rigged mesh. 

The viewer must resolve the stack of animations playing on the avatar, and then apply that to each and every object sequentially. The more worn rigged objects you have on screen, the more time that takes, the more time it takes the less it's rendering.

Rezzed mesh is pretty cheap in the grand scheme of things, the overall complexity has far less impact that it would in a game engine were GPU time is directly tied to frame rate. 

This is why disabling even just your own avatar gives a huge speed boost, but disabling the bulk of world rendering (volume) doesn't.

[Madelaine McMasters]

2 hours ago, Coffee Pancake said:

You want a single HUGE textures that's shared between multiple objects.

Still, you'd want that one shared texture to be as small as possible.

The fact that aggregating textures into one file improves things is slightly counterintuitive. Imagine I have four place mats, each with different 128x256 textures. Does it really save time to splice those four textures into one, then offset/scale that one texture differently on each of the four different mats?

2 hours ago, Coffee Pancake said:

JPEG2000 textures are pulled in a steps, get some data, hope its enough to get something we can use, if the object is big enough on screen to need more, get more data, rinse repeat. 

The viewer wont load full resolution textures for everything automatically, but it also doesn't toss data out once it's been forced to load the full thing (by zooming up close to everything).

This is as I'd imagined, and you've answered my "(maybe it goes all the way?)" question with a "no". That's good.

2 hours ago, Coffee Pancake said:

It also wont ever load a full resolution anything when the avatar and camera is in motion. This is optimal for exploring the world, not so good for events.

Yep, I've noticed this.

And thanks for the detail on worn, rigged mesh. I understand that rigged geometry must be recomputed every frame, but I wasn't aware that was so much worse than the static mesh. Still, I imagine that zillion triangle static meshes taking up half the screen take longer to draw than hundred triangle meshes of the same size, regardless of texture size.

Edited April 20, 2022 by Madelaine McMasters

[Coffee Pancake]

1 hour ago, Madelaine McMasters said:

The fact that aggregating textures into one file improves things is slightly counterintuitive. Imagine I have four place mats, each with different 128x256 textures. Does it really save time to splice those four textures into one, then offset/scale that one texture differently on each of the four different cups?

Yes, absolutely. Always use the biggest texture you can, and pack as many things onto it as possible.

One fetch, one decode, one pass at render time.

 

It's not as obviously performant as it could be, but that's only because of everything else the viewer is doing. 

[Fluffy Sharkfin]

3 hours ago, Madelaine McMasters said:

Eventually I'll just set you on fire.

Duly noted, I shall be sure to dress appropriately!  

[Velk Kerang]

19 hours ago, Coffee Pancake said:

I work on Catznip with Kitty. We don't import feature code from other viewers, everything is a unique work that we have worked on together, every viewer even the official one has our work in it. We've been doing this for over 10 years.

All right. Awesome. I'm impressed already. I actually vetted your viewer about 3 years back and while at the time I would have liked to seen certain features implemented in it I was still impressed with the performance of it none the less. For some reason your signature didn't show for me the other day, but now it's clear as day. Anyway moving on.👍

 

19 hours ago, Coffee Pancake said:

Every viewer must operate with the systems LL have in place and stay up to date with their changes, and we are all bound by the not only the data that LL exposes, but how and when they expose it for their own viewer. This acts as a huge constraint on how far into the deep magic we can practically go as the viewer is dumb like a rock. We can't get creative with data we don't have.

The bulk of the code under the hood is at least functionally the same (and likely to be 99% identical to Linden). UI does make up a significant portion of the code base and tends to be where TPV developers focus.

Sounding reasonable so far.👍

19 hours ago, Coffee Pancake said:

Modding games and working on SL viewers are entirely different skills.

I can agree to that. I'll just add that both can be equally just as complex depending on the project.👍

19 hours ago, Coffee Pancake said:

In large part because the core engine was explicitly designed around the lower resolution assets.

Games are Formula1 race cars, everything is designed specifically towards a singular end. The engine and the context it operates in and the hardware it runs on are tightly balanced and optimized in unison. So while you could mod a F1 car to run with monster truck tires, it's not going to perform like a actual monster truck.

The SL viewer is more like a dump truck, built to handle a staggering amount of random stuff people put out for it to pick up.

Eh, I half agree with this. Reason I say that is I'm not sure if you familiar with what Total Conversion mods are, but they often can and do rebuild a game from the ground up. Software is software. It'll do whatever you tell it to if you know how to sweet talk it just right.👍

 

19 hours ago, Coffee Pancake said:

It's nothing personal. This topic or variations on it have been brought up thousands of times over the years.

I didn't take it personal and sorry I haven't really participated in to many of these type of topics let alone recall seeing them. My participation in this one was more of a in a passing by sorta way. Got my own thing going on and was more a tossing ideas to focus on for those more invested in this.👍

 

19 hours ago, Coffee Pancake said:

Games are entirely focused around getting the game rendered and maintain strict performance goals. The rendering end goal is dominant and everything else is constrained by that focus, right down to the logistics behind managing the raw data and assets. Games - Here's all your assets - Go go go go gogoooo !!

SL is focused on data management. The bulk of the work load is processing assets as the server pops them into existence, resolving avatars and animations and so on. Rendering is just something that happens after all that. SL - here's a few hundred breadcrumbs, that explode into more breadcrumbs. Busy with that? Here's a hundred more. Miss something and stuff breaks.

The rub .. that data management can't be fundamentally changed or shortcut because the SL servers dictate, and off the back of that the SL client can generate a significant amount of http traffic (all of which needs subsequent extremely specific processing) - we used to overwhelm the cheap home routers shipped by ISPs.

You want that running single threaded on a cell phone, yeah, good luck.

Games are easy .. simplify the assets, get more speed.

SL .. not so much. The only way you're getting significantly more speed is by reducing the overall load by dropping draw distance.

 

Specifically concerning textures, You can easily limit max texture resolution in the viewer .. you know what happens? You shorten the total time spent fetching and decoding. But the bulk of the work still has to happen and doesn't go away. Even if you sit on your hands and let it do all the work, the final frame rate after it's all done .. barely improves.

 

BUTT 🍑 ... it gets worse, because of course it does.

Sequentially resolving avatar animations and applying that individually for each and every worn rigged object (attachments can be many objects) consumes a huge amount of CPU and scales linearly.

 

Guess what isn't happening while the viewer is doing this processing ? Rendering.

Well first fun fact. As far as Androids go anyway. As I am sure you know they are based off Linux. I'm going to also assume you're familiar with Vulkan. So single threading is not so much an issue as a lot of Android devices take full advantage of Vulkan drivers. So that's really a non-issue at this point. Even if it was that is the very reason I referenced Singularity because I tested it with opengl purely on Linux using a toaster and it runs flawless and renders 5 times faster then any other viewer out there. So again, non-issue. There was an actual reason I recommended that specific one as a base model for Android devices.👍

Now hypothetically speaking and I'll see if I can explain this properly, if LL ever created an actual client that allowed for or created rather actual assets as the world loaded or when new items appeared similar to how the cache function works except actual assets are created then the platform very well could go go go go gogoooo!!!! Vroom! Vroom! Style. lol It definitely would reduce lag issues. I don't see that ever happening though given the way people crap their britches over content in this game. So that's out. lol🤣

Personally on the point as far as me wanting it running on my phone. I gotta be real honest with you. I don't really care. I am good with SpeedLight. That's my Slim Fast viewer right there. It does what I need it to. Tested the 3D on my phone. Thought it rendered fine. So my original suggestion seemed very feasible based off my own tests. I'm personally not going to be playing Second Life on my phone. I got an app for check in purposes should the need arise. So I'm good mate. lol👍

I was thinking more about the people with tablets and such or those who's PC's pooed the bed on them. I get it though. I do. You're basically saying it can't be done. My suggestion is crap. Not a problem. I got no problems yielding to the expertise of someone who's worked 10 years on Second Life viewer projects. My skill set doesn't include that. So it's all good in the hood as far as I am concerned.👍

 

19 hours ago, Coffee Pancake said:

No, you jumped to the same suggestions we have been handed hundred of times based on the same fundamentally wrong assumptions. 

SL looks like a game, it can run like a game, game optimizations must apply.

SL content takes ages to rez, therefore the content must be bad, fix the content and fix SL.

SL rendering looks simplistic, it must be simplistic, my phone can do better.

 

That all said ... 

The viewer and all the required dependencies are open source (builds without havok), the development tools are free, all you have to lose is time .. and perhaps some hair. 

The reason you haven't see any viewer devs doing this is all the years we have had the source isn't a lack of desire for a mobile client or laziness. Same reason there haven't been any big attempts to get SL to run on a "real" game engine. It's showing to a F1 meet with a dump truck.

If it was even remotely viable, someone would have done it just for the fun of it .. that is why we do third party viewer development in the first place.

Well I didn't just up and jump to the same suggestions. My opinions are my own and I've had them from day one ever since I vetted this platform out when looking for a solid mmo with a player base economy as an alternative to Star Wars Galaxies for a 500 player Guild/Fleet/Clan at the time. And while it does have one it never made the grade for so many reasons I won't even get in to because that's not the point. The point is my opinion is one that is decades old that I still stand by even to this very day.👍

So I am sorry if it is one that you heard before, but since what that tells me is if I am not the first or even the only one to have said it then I would totally get and even understand why you might be annoyed by the suggestion. What I am also taking away from this is that it is either not feasible to do as you said which I am happy to run with or to entirely much of a pain in the behind to do or even deal with which also I can relate to from my own projects. lol🤣

Having said that I am going to leave a few suggestions for you and/or anyone interested and people can ignore or do with them what ever they want. Just something I think is worth mentioning is all. There is a port called Devolution X for the PC version of the original Diablo. Very nicely done project. Awesome development team. So my first suggestion is rather then looking at building viewers from the ground up just either create a port for existing one or pack them as an apk making the adjustments needed in order for it to run. If it ain't doable then not an issue.👍

My second suggestion is a bit more on the technical side for people out here. Alternatives such as WINE or some other apps that allow you to run Windows programs on Android. There are also ways to run Linux programs on Android using certain apps as well too. So you can run programs and even games like The Lord Of The Rings Online for example with little to no problems. So you all could try rolling the dice with that.👍

My third and final suggestion for people. Perhaps my top suggestion if I were someone that I just needed to have a 3D viewer on an Android device for what ever reason then I would look in to Shadow Cloud Gaming. It does run about $29.99 a month if I recall, but the benefit to this you do not have to worry about or deal with all the issues that @Coffee Pancakewas kind enough to explain. Which thank you for that by the way. The application for it would save on disk space for your device and you would get the benefits of a high end gaming rig to boot. And honestly it would probably save wear and tear on your device hardware at the end of the day.👍

Anyway option three is my best helpful tip for the day. Stay safe and have a blessed day everyone. Peace, love, and hair grease.👍😎

Edited April 20, 2022 by Velk Kerang
Corrections.

[Cali Souther]

18 hours ago, Madelaine McMasters said:

It's a piece of reactive artwork I made back in 2008...

 

That is beautiful !    ♥♥♥  

[Anicha Heartsong]

Not understanding most of the technical talk here, but it seems like one of the major issues in crowded environments is the number of rigged mesh items and animations working in tandem.

So, would enforced nudity for all public events help to reduce the lag?

[Coffee Pancake]

2 hours ago, Anicha Heartsong said:

So, would enforced nudity for all public events help to reduce the lag?

No as mesh bodies are kind of the elephant in the room .. but old school system avatars would !

[Anicha Heartsong]

System avatars? No thanks. Losing interest in that idea rapidly.