Second Live 2.0 in Unreal engine

[DNArt]

Hello Animats 

I can only give like a simplistic explanation due to limited knowledge but here https://www.howtogeek.com/194756/cpu-basics-multiple-cpus-cores-and-hyper-threading-explained/  they explain the hardware part of your CPU very well.    My CPU has 16 real cores but also a virtual core per core so they appear to have 32 cores for the operating system. 

As far as i did read firestorm uses a max of 4 cores (please someone correct me if i'm wrong )  so as it stands now in theory the SLviewer can use 32/4 = 12.5% of my CPU    

If you scroll down abit on this page https://www.tomshardware.com/reviews/cpu-hierarchy,4312.html  you can see modern CPU's have lots of cores and they wil get more and more each year. 

 

Additional info.. At the time when the SL viewer was first build decades ago   the principle of giving CPU's more cores was not very common and the operating systems probably also didn't fully support that.  In those days a CPU had 1 core so 1 unit did all the tasks and if you wanted to have more done you had to boost the speed at which the CPU operated. At a point in time they reached a speed that made the processors melt so they had to come up with an other strategy for faster processors.  More cores per processor.  Where there was once 1 worker (core) to do the tasks we now have a team (multiple cores),  and each year the new CPU's give us bigger teams. That means the tasks has to be spit preferably in such a way all members of the team stay busy.   Of course LL knows this for years already and they do try to spit parts of the job to several workers if one is available. But because the whole order of things was made up at a time when just a view people where thinking in teams, the way in which you can spit of tasks is far from optimal.  The correct approach would probably be to rebuild huge parts from the ground up with compatibility and multi treading in mind.  Something very view people would jump up from joy todo.   To make things even more complicated   ......

As mentioned before by several people, SL cant do standard tricks that other games + game engine's use. This has lots todo with precompiling the graphics data.  Standard in games after they been developed the computer analyses the data and makes optimisations.     because SL allows us to change textures in a sim,  move objects, rez items etc etc . all the things we so love about SL and all at anytime we like,  most of those are impossible  todo before the data is send to our computer. toi make things worse our content creators are more concerned about selling there products and how many LI they cost than about how easy to render they are.  Understandable from there point of view.  

in an ideal world the CPU would call the GPU once each frame and tell it here is the data do your thing.  This call is called a draw call.   The ideal world doesnt excist for any game, but still the general rough rule of thumb is less draw calls leads to => higher fps  so a draw call in  AAA games make the GPU draw thousands of triangles at once (GPU's have many thousands of cores. here you have an overview of  GPU's from 2 generations ago https://www.nvidia.com/en-us/geforce/products/10series/compare/ to see for yourself) and in SL just 70 triangles on average.  this means our CPU will be busy lots with sending data to the GPU and sadly our viewer doesn't uses all the workers in our CPU. Here we see one of our bottlenecks  a GPU that appears to be idle and a CPU that sleeps as well since just a view cores are used. 

So the graphic cards will get more cores in the future  but our draw calls hardly use those cores since SL's draw calls there super small. and our future CPU will get more cores but just 4 appear to be useful since the current viewer cant spit the work load much better.   it all leads to a trend that doesn't favor our viewers performance. 

I fear it needs serious rethinking over the way the viewer now operates, leading to rewriting huge parts of the code and libraries. We can just  hope they also come up with some creative tricks to let all those idle cores do some ad hock optimisations as well.   So it will need LL to sit down and make this there priority its to huge a task to be done by 3th party developers.

From engadget "With Sansar off its books, Linden Lab is now "a very profitable company," according to Altberg. "Second Life is an older, very established cash machine that works really wel"       I would say this is realy great to hear but please invest a bit in the future of your cash machine so it keeps up with it time and keeps generating cash!  

 

      

  

 

 

[Ardy Lay]

Is the a metric that can be used that would be less scene dependent than Frames Per Second?  I was wondering about KTris/s.

[DNArt]

@gage wirefly 

yes a big draw distance leads to many more draw calls of even viewer triangles so highly in effective  since they draw the LOD level with very view triangles from the distance sims. 

I did put the distance so high just to illustrate that bottleneck even more.  they could maybe better send us some imposter textures from those sims  hard to notice details from that far but great to see something at all when flying or sailing   https://www.sidefx.com/tutorials/game-tools-imposter-textures/      oh look what google did find. https://www.gamedev.net/forums/topic/707584-automatically-grouping-objects-for-impostoring/5429455/    <---- LL using imposters for distance sims probably saves money server side and makes SL more fun for vehicles 

@Ardy Lay you are right Ktris/s would be less arbitrary.  and that 70 triangles on average for a draw call was something i did read a long while ago. But i can still imagine it to be true since current LI calculations lead to almost near 2 triangle at the highest LOD level for many products.    

 

 

Edited February 4, 2021 by DNArt

[Gage Wirefly]

23 minutes ago, DNArt said:

@gage wirefly 

yes a big draw distance leads to many more draw calls of even viewer triangles so highly in effective  since they draw the LOD level with very view triangles from the distance sims. 

I did put the distance so high just to illustrate that bottleneck even more.  they could maybe better send us some imposter textures from those sims  hard to notice details from that far but great to see something at all when flying or sailing   https://www.sidefx.com/tutorials/game-tools-imposter-textures/      oh look what google did find. https://www.gamedev.net/forums/topic/707584-automatically-grouping-objects-for-impostoring/5429455/    <---- LL using imposters for distance sims probably saves money server side and makes SL more fun for vehicles 

@Ardy Lay you are right Ktris/s would be less arbitrary.  and that 70 triangles on average for a draw call was something i did read a long while ago. But i can still imagine it to be true since current LI calculations lead to almost near 2 triangle at the highest LOD level for many products.    

 

 

I am really not sure what your issue is. I have far less high end hardware, i5-9400 and a GTX 1650 Super 4gb, 16 gb of ram +an nvme drive.  The only different thing is that I use a very low draw, 88m and 50k complexity for all avatars. I also let the GPU handle the anti aliasing and its cranked to max. I said in another post, I get 100fps or better on empty sims and about 40 or so on busier ones. So I am guessing it's either system configuration or your internet connection, but I am not sure on either.

[animats]

18 minutes ago, DNArt said:

As far as i did read firestorm uses a max of 4 cores (please someone correct me if i'm wrong )  so as it stands now in theory the SLviewer can use 32/4 = 12.5% of my CPU

I wish. Firestorm and the LL viewer do almost everything in one thread. A few things are in other threads. JPEG 2000 decoding of textures is, for example. Voice and media on a prim are in separate processes. But all the drawing and networking and the user interface are all in the main thread. When that thread becomes CPU-bound, frame rate must drop.

The way you'd do this in a new viewer would be to load all the active geometry and textures into the GPU and have a thread dedicated to keeping that refreshed as the point of view changed. Other threads would add, delete, and move active objects as the point of view and world changed. Then you could fully utilize a high-end gamer PC.

The problem is GPU memory. If you have enough GPU capacity, this isn't too hard. If you're GPU-limited, the viewer would have to reduce the draw distance (preferably automatically) and take other measures to cram enough scene into the available memory. In other words, on machines with Intel HD on-chip graphics and 4GB of RAM, which is where many older low-end laptops are, this is going to suck. On a gamer PC with an NVidia 1060 or better, it would be great. Even a current WalMart low-end EVOO laptop would be adequate. It's supporting older low-end machines that's the problem. About 20% of SL users have computers that can't even run Vulkan, which will run on almost anything reasonable newer than 5 years old.

[DNArt]

@ Gage Wirefly i gues the problem is that 88m draw distance  sounds not  that appealing to me in an open world game. thrust me system speed/ configuration and network are no bottlenecks. With a  low draw distance -> viewer draw calls  the CPU-> GPU botleneck  disappears. and suddenly the GPU can do what its for. but when you rent or visit a sim or like to use vehicles to explore the need for a bigger draw distance grows.          

https://gyazo.com/a95582864bc05f0250f1e857b6febd5a    https://gyazo.com/89c9f9db952c2c3c50182fa63b450b36     lower LOD  doesnt change that much btw in framerate at the moment 

Edited February 4, 2021 by DNArt

[Solar Legion]

1 hour ago, Gage Wirefly said:

I run SL at Ultra settings same as the OP but with a much lower draw distance and max avatar complexity . I average 100+ fps on empty sims and around 40 on busy ones.  I'm not sure where you're getting that from, unless it's just your opinion.

Nope, not just my opinion - Draw Distance does not set itself to 1024, Object LoD does not get set to 4, Terrain LoD does not get set to 4 and the Max # of non-imposter Avatars does not get set to No Limit when you use the Default Ultra settings suggestions.

Those go to 256, 3, 2 and 16 respectively.

Do note that I said nothing whatsoever about the Default Presets in that post - I mentioned the sliders directly for a reason.

My Setting? Ultra with Object LoD lowered to 2.75 and the Non-Imposter value also lowered to 12 with a Draw Distance usually around 128 meters. Sometimes set to 256 but that depends on the situation.

It's nice that you get FPS above a hundred. It's also meaningless unless you're streaming. Chalk that last one up to personal opinion - once it goes above sixty, I do not see a difference whatsoever. Do note that the average I get in most places is between forty five and near eighty with a sharp drop if there are high complexity avatars in view.

That's with a Ryzen 7 2700X, nVidia 980 TI, 32 GB of system RAM, no NVMe drive.

Edited February 4, 2021 by Solar Legion

[Solar Legion]

26 minutes ago, Gage Wirefly said:

I am really not sure what your issue is. I have far less high end hardware, i5-9400 and a GTX 1650 Super 4gb, 16 gb of ram +an nvme drive.  The only different thing is that I use a very low draw, 88m and 50k complexity for all avatars. I also let the GPU handle the anti aliasing and its cranked to max. I said in another post, I get 100fps or better on empty sims and about 40 or so on busier ones. So I am guessing it's either system configuration or your internet connection, but I am not sure on either.

Hmm, it couldn't have anything at all to do with him also having maxed out the LoDs ... oh no ... 

[Gage Wirefly]

9 minutes ago, Solar Legion said:

Hmm, it couldn't have anything at all to do with him also having maxed out the LoDs ... oh no ... 

I would say no and I'm going to leave it at that. Everyone has their opinions and thats what they are...opinions.

[DNArt]

Hi Solar  

For me the 6.5 fps was not the issue i maxed out the settings to illustrate where the bottle neck is. 

If you test things its often  good to test extremes i been told so problems present them self more clearly. What i tried to illustrate is that lots of CPU/GPU power stays unused currently and even more so in the future.  so now we have to lower draw distance which i find a shame and which in my opinion shouldn't be necessary the constant popping up of object or altering there shape also limits a feeling of immersion maybe im more visual oriented, no clue but to me these things brake parts of the illusion.    

 

Edited February 4, 2021 by DNArt

[Solar Legion]

8 minutes ago, Gage Wirefly said:

I would say no and I'm going to leave it at that. Everyone has their opinions and thats what they are...opinions.

That's nice. Higher LoDs equates to more rendering/more work for the hardware and keeping things at a higher visual quality for longer.

That's not an opinion - it's a fact.

What is an opinion is what is considered a reasonable frame rate to expect.

And that is where I'll leave it.

[Solar Legion]

3 minutes ago, DNArt said:

Hi Solar  

For me the 6.5 fps was not the issue i maxed out the settings to illustrate where the bottle neck is. 

If you test things its often  good to test extremes i been told so problems present them self more clearly. What i tried to illustrate is that lots of CPU/GPU power stays unused currently and even more so in the future.  so we have to lower draw distance which i find a shame and which in my shouldn't be necessary the constant popping up of object or altering there shape also limits a feeling of immersion maybe im more visual oriented, no clue but to me these things brake parts of the illusion.    

 

While it is understood that you were trying to illustrate the bottleneck, going to such an extreme wasn't really necessary. Most long time (and many newer) users either already know about that bottleneck or otherwise very quickly discover it.

For the latter, if/when it is brought up here in the Forum the rather quick answer - on top of suggesting a few settings to change - is that Second Life does not use more modern hardware the same way as more modern applications.

Sadly you'll also invariably get instances in such threads as between Gage and myself a moment ago. Someone will invariably chime in with an FPS in the triple digits and someone else will pop up with an FPS in the "normal" range as well. Sometimes on similar hardware, sometimes with hardware differences and all using different settings.

As I'd said in my response above, the usefulness of FPS in the triple digits is a personal thing, based more or less on what a particular user actually notices. Many are a bit more lenient on the difference between opinion and fact where the settings are concerned ... I'm not one of them. There's a reason the Object LoD (as an example here) is capped at four and will reset between use sessions if Debug controls are used to bump it higher - that reason partly having to do with the performance hit a higher LoD has and another part having to do with some content creators having used higher Object LoD settings when creating their wares, distributing Notecards with the "advice" to turn that setting up if the end product "falls apart" (looks like a mess of triangles) when in a viewing range that would normally be reasonable.

I don't have much tolerance when it comes to denying or handwaving those types of performance hitting settings. I also have very little tolerance for the mind set that one ought to by hyper-vigilant concerning kTris and such. That is not something the end user should have to be concerned with.

[DNArt]

Hi Solar im generaly happy with fps over 15 over 20 is joy i did redit an early post and did add some screen grabs  because i moved LOD from 4 to 1 just to try.     I didn't move which could be cheeting? but i noticed no difference in FPS maybe because even at a setting of 1 the amounts of draw calls getting issued stayed the same and the GPU could easily handle the more triangle per call? and CPU/GPU communications is the bottle neck for my graphics card?  (2080ti)  Im sorry if i pressed a button in your irritation zone but i appreciate your contribution 

 

https://gyazo.com/a95582864bc05f0250f1e857b6febd5a    https://gyazo.com/89c9f9db952c2c3c50182fa63b450b36     lower LOD  doesnt change that much btw in framerate at the moment 

just did it again the weird thing is yes i see things disapear at LOD 1 that are there at LOD 4  but fps is marginally higher at LOD 4 

https://gyazo.com/b6abec99272ebeb4000150d5052de4ac

https://gyazo.com/b6ff72cbcd5945b03ea6f633936f98b3

Edited February 4, 2021 by DNArt

[Solar Legion]

21 minutes ago, DNArt said:

Hi Solar im generaly happy with fps over 15 over 20 is joy i did redit an early post and did add some screen grabs  because i moved LOD from 4 to 1 just to try.     I didn't move which could be cheeting? but i noticed no difference in FPS maybe because even at a setting of 1 the amounts of draw calls getting issued stayed the same and the GPU could easily handle the more triangle per call? and CPU/GPU communications is the bottle neck for my graphics card?  (2080ti)  Im sorry if i pressed a button in your irritation zone but i appreciate your contribution 

 

https://gyazo.com/a95582864bc05f0250f1e857b6febd5a    https://gyazo.com/89c9f9db952c2c3c50182fa63b450b36     lower LOD  doesnt change that much btw in framerate at the moment 

 

LoD is one of those "fine tuning" tweaks - Draw Distance tells the Viewer how far out to render, which does have a fair bit more of a direct effect. Two to Three is - in general - the range more or less set by the higher presets for Object Lod.

Knowing that you're using a 2080TI is also a handy bit of information.

For a bit further tweakage - not really for showcasing the bottleneck but just to see if your 'normal' settings can be tweaked a bit - are you using the Dynamic Cache option and if so, what're the settings you're using? Also, do you usually have the terrain LoD up that high?

Mine's set to use a little over three GB as the minimum and the two sliders below set to ten percent.

ETA: Remembered that the EEP Viewer version of Firestorm also has a regression that can affect users. There's an option under the Rendering subsecton of the Graphics settings that reverts water Occlusion behavior to pre-EEP type when enabled that seems to help some. Seeing that yer on Windows it might also help to ensure your Color Range and GL Threaded Optimizations are set to Full and Enabled in the nVidia control panel - the former is, I think, global while the later is per application so would need to be set for the Viewer.

Edited February 4, 2021 by Solar Legion

[DNArt]

looking ..  

https://gyazo.com/308d6c92bb6bc1d5b6a58948b760d038

those are all set at max  the land LOD does have effect. 2 fps difference between 1 and 4. 

I'm not unhappy or things like that but i do find it a bit disappointing the SL viewer  runs at almost the same speed as it did on my previous system (1080ti )that i used up to 2 years ago the hardware is an overkill now. and doesn't mind most SL throws at it. The CPU seems to not get the draw calls there in time when the draw distance is increased and thats the limit that restricts the fps rate on my 2080ti.     If LL don't start addressing CPU/GPU communication soon i will not notice any difference in speed or looks when i upgrade in like 3 years time.   those with laptops will because there  low end GPU's will run faster and have a different bottle neck.        

 

ETA: thats handy to know thanks i never looked at the nVidea control panel 

Edited February 4, 2021 by DNArt

[Solar Legion]

Hmm ... I just find it odd that you're getting that kind of frame rate on that hardware, bottleneck nonwithstanding.

I'm getting a higher frame rate on average on an older GPU and a CPU with fewer cores on average with the only other difference being Operating System.

What's the location you're testing at again? I'm now curious what I'd get when there.

Edited February 4, 2021 by Solar Legion

[DNArt]

im currently standing in kare  your welcome . But i doesnt suprize me you get better framrates . my hardware now is an overkill  and yours better tweaked 

Edited February 4, 2021 by DNArt

[Gage Wirefly]

1 hour ago, Solar Legion said:

While it is understood that you were trying to illustrate the bottleneck, going to such an extreme wasn't really necessary. Most long time (and many newer) users either already know about that bottleneck or otherwise very quickly discover it.

For the latter, if/when it is brought up here in the Forum the rather quick answer - on top of suggesting a few settings to change - is that Second Life does not use more modern hardware the same way as more modern applications.

Sadly you'll also invariably get instances in such threads as between Gage and myself a moment ago. Someone will invariably chime in with an FPS in the triple digits and someone else will pop up with an FPS in the "normal" range as well. Sometimes on similar hardware, sometimes with hardware differences and all using different settings.

As I'd said in my response above, the usefulness of FPS in the triple digits is a personal thing, based more or less on what a particular user actually notices. Many are a bit more lenient on the difference between opinion and fact where the settings are concerned ... I'm not one of them. There's a reason the Object LoD (as an example here) is capped at four and will reset between use sessions if Debug controls are used to bump it higher - that reason partly having to do with the performance hit a higher LoD has and another part having to do with some content creators having used higher Object LoD settings when creating their wares, distributing Notecards with the "advice" to turn that setting up if the end product "falls apart" (looks like a mess of triangles) when in a viewing range that would normally be reasonable.

I don't have much tolerance when it comes to denying or handwaving those types of performance hitting settings. I also have very little tolerance for the mind set that one ought to by hyper-vigilant concerning kTris and such. That is not something the end user should have to be concerned with.

OH NO THE CHIMES. 

[Coffee Pancake]

17 hours ago, DNArt said:

Hello Coffee Pancake 

I know you know very well what your talking about. 

Do you have any suggestions what should be done by LL (and can we, the users, speed that up) to make sure the viewer doesn't run at just 6.1 fps and lets our computers sleep? 

Lower your draw distance.

Imagine your draw distance is a circle around your avatar, information about everything in that circle will be sent to your viewer by the servers. Your viewer will then explode this into lists of assets, meshes, textures, animations and so on, and then set about downloading, decoding, and assembling everything so it can be rendered and interacted with.

Just a few items can quickly balloon into thousand's of separate bits of data that need to be processed by your viewer.

While your viewer is doing this, it's not telling your graphics card to render nice fast frames.

If your draw distance is much too high (like yours is) this breaks even further as the volume of information being sent overwhelms your computers ability to even get a handle on processing it, and when it tries the order the data comes in is so out of whack, nothing seems like it's rezzing.

128M is fine. Start with that.

 

NEXT ! TURN DOWN THE OBJECT DETAIL SLIDER

If you set it to the max, you have broken dynamic object detail for all but a few unlucky edge case objects. Don't do this.

 

10 hours ago, DNArt said:

I can only give like a simplistic explanation due to limited knowledge but here https://www.howtogeek.com/194756/cpu-basics-multiple-cpus-cores-and-hyper-threading-explained/  they explain the hardware part of your CPU very well.    My CPU has 16 real cores but also a virtual core per core so they appear to have 32 cores for the operating system. 

As far as i did read firestorm uses a max of 4 cores (please someone correct me if i'm wrong )  so as it stands now in theory the SLviewer can use 32/4 = 12.5% of my CPU    

SL runs in 1 thread. That's ONE thread.

This will be smeared over all your cores because that's how cores like it, but it will never be using more than the output of 1 singular core.

It is possible to mess about with debug/dev settings and make SL look like its using 2 cores. It's not. There is no performance boost. You are however generating double the heat.

10 hours ago, DNArt said:

The correct approach would probably be to rebuild huge parts from the ground up with compatibility and multi treading in mind.  Something very view people would jump up from joy todo. 

We go to the TPV dev meetings and whisper VULKAN in the Linden's general direction ... they worry 20% of SL users can't even use VULKAN, everybody cries.

There is no point doing the rework needed so long as the rendering is done by opengl. The bottlenecks involved more than kill any performance gains. We have tried

 

10 hours ago, DNArt said:

 So it will need LL to sit down and make this there priority its to huge a task to be done by 3th party developers.

There is no way a hobbyist developer working for free on a third party viewer is going to roll a new state of the art render pipeline.

It's not even remotely feasible.

It requires a significant investment of time and expertise and a medium sized boat filled with cash.

 

 

[Profaitchikenz Haiku]

2 hours ago, Coffee Pancake said:

We go to the TPV dev meetings and whisper VULKAN in the Linden's general direction ... they worry 20% of SL users can't even use VULKAN, everybody cries.

You're going to hate me for this, I'm one of that 20%. My PCs are antiquated. The one laptop I have that might conceivably handle Vulkan has a failed graphics card that apparently needs reballing, and sadly there's no other card will fit that particular laptop so it's currently a paperweight until I strike lucky.

But if it makes you feel a little better towards me, I manage to run 32-bit viewers reasonably well on my antiquated hardware, and the viewer that works best of all is Catznip. Much faster than Firestorm, quite a bit faster than Singularity. Almost crash-proof (unless I visit places with masses of new mesh buildings or come too close to an avatar wearing tiny mesh jewellery with a linked 8-metre square transparent prim over their head to make sure those earrings don't degenerate into triangles at 10 metres away).

Keep up the hobbyist development, you're helping the pensioners

Edited February 4, 2021 by Profaitchikenz Haiku

[Coffee Pancake]

2 hours ago, Profaitchikenz Haiku said:

But if it makes you feel a little better towards me, I manage to run 32-bit viewers reasonably well on my antiquated hardware, and the viewer that works best of all is Catznip. Much faster than Firestorm, quite a bit faster than Singularity. Almost crash-proof (unless I visit places with masses of new mesh buildings or come too close to an avatar wearing tiny mesh jewellery with a linked 8-metre square transparent prim over their head to make sure those earrings don't degenerate into triangles at 10 metres away).

Keep up the hobbyist development, you're helping the pensioners

Thanks and I will pass this on to Kitty

[MarissaOrloff]

On 1/25/2021 at 7:44 AM, Profaitchikenz Haiku said:

I'd suggest searching for Magrathea, they should be able to customise a world for any millionaire with a desire for something a little bit different.

Ask for Slarty. He loves doing coastlines.