Does anyone know the real truth behind Second Life Servers?

[Lord Derryth]

I'm trying to wrap this around my head.  A sim costs $300.  A homestead is $150.  That's what it will cost you to buy or rent or whatever the case.  Sims come with 20k Prims with the exception of upgrading to 30k.  Homesteads are locked in at 5k prims.  Yet, both sims and homestead hold the same value in size.  Help me understand the whole concept here.  Why are homesteads half the price, nearly 85% less in prims?  Why is it that there is a cap in scripts and objects that causes the sim to seriously lag out to the point, it's not worth having a sim.  what is the point of having a 30k prim sim if you can't use 20k prims or else it lags.  Example, building a massive urban city with apartments and stores.  Obviously, furniture has script in them.  

1537 scripts taking a total of 2.546 ms and using 590,651 KB <---------Why is this a death killer on a sim you pay for $300 that comes with 30k prims?  How small actually are the servers in SL that is worth spending $300 when they are probably worth less than $100.  There needs to be a full investigation what goes on behind closed doors.  I feel there's a scam going on here.  Why get a full sim if you can't utilize it's full potential?  I know some of you will bicker and say "Don't get a sim, nobody is forcing you."  I get that.  I enjoy the urban role play and driving around in a city.  I'm in constant lag with over 600mbps speed to Portland, Oregon.  I should have no issues with lag.  It seems the technology and servers are old or lacking in power.  Is $300 for a sim really a scam?  

[Quistess Alpha]

You're not just paying for the "technology" or computation power, you're also paying for access to Secondlife(tm)'s virtual world, and the social environment it provides.

If you want all the prims and compute power you can pay for, maybe a kitely opensim server would be a better fit for you, just don't expect anyone to visit, or be able to find nice-looking and legal(not copy-botted) clothing.

[bigmoe Whitfield]

Second life is now run on Amazons aws servers out of Oregon,  they have no physical servers anymore,  costs according to oz linden whom was the one whom did most of this,  has said,  it costs LL more to use aws than it did having their physical servers (which were upwards of 10 to 12 years old),  so as of now,  bringing down costs is not happening.

I wish we had a full run down on their aws virtual machines running the sims, but they have not disclosed that information, but they are no slouch,  they are still as of last news working and optimizing how everything runs.   

[Kardargo Adamczyk]

Its because 2 homesteads share core and memory on a server, 4 open space regions share a core and memory on a server , a full region gets its own core and memory.

 

Edited May 16, 2022 by Kardargo Adamczyk

[Alwin Alcott]

1 hour ago, Kardargo Adamczyk said:

Its because 2 homesteads share core and memory on a server, 4 open space regions share a core and memory on a server , a full region gets its own core and memory.

 

The numbers changed several times with newer servers in their own farms, now on amazon it changed again i think.
It's been one full region on one core, 4 homesteads on one core, and 8 open space regions on one fro long time .. think after that the info never updated. And not really public info since in the cloud.. íf it's even relevant now.

[Love Zhaoying]

I expect we won't be told the actual specs of the (virtual) AWS hosts.  What's the reasons, again? a) Because the specs are "irrelevant" / open to interpretation; b) too likely to draw criticism and controversy; c) subject to change; d) contractually to be kept between just AWS and Amazon (NDA, etc.); e) Other, or some combination?

[bigmoe Whitfield]

2 hours ago, Alwin Alcott said:

The numbers changed several times with newer servers in their own farms, now on amazon it changed again i think.
It's been one full region on one core, 4 homesteads on one core, and 8 open space regions on one fro long time .. think after that the info never updated. And not really public info since in the cloud.. íf it's even relevant now.

I'll go with this for now myself, since we have no hard info.

[Qie Niangao]

9 hours ago, Lord Derryth said:

1537 scripts taking a total of 2.546 ms and using 590,651 KB <---------Why is this a death killer on a sim you pay for $300 that comes with 30k prims?  

What else is accounting for the rest of the 22.2 ms frame duration? If as I suspect there's substantial Spare Time in the frame, stop looking at scripts nor anything else server-side; it would have nothing to do with whatever lag you're experiencing. (If, on the other hand, there's a whole lot of Physics Time, It's probably pretty obvious what's causing that, and that will take priority over scripts getting much chance to run, and overall will feel laggy.)

It may help to know what region this is about. It certainly doesn't seem to be Embraced, which has many times that number of scripts, burning a total of about 11 ms Script Time per frame, yet nonetheless running 100% of the scripts and leaving almost 10 ms of Spare Time. (I was the only avatar in the region at the time, but it must be a different region at issue anyway.)

One other observation: that 590,651 KB number, if that's the sum of script memory, it would take a minimum of 9,228 scripts to use that much. But if it's actually not script memory, but rather Physics "Memory Allocated" (publicly visible in the statistics bar) then indeed the sim would be in big trouble, usually following a lot of terraforming between restarts. (That's apparently a very hard to find memory leak, considering how old the bug is.) Again, though, that's not the current state of Embraced, if that happens to be the region in question.

[Lord Derryth]

3 hours ago, Qie Niangao said:

What else is accounting for the rest of the 22.2 ms frame duration? If as I suspect there's substantial Spare Time in the frame, stop looking at scripts nor anything else server-side; it would have nothing to do with whatever lag you're experiencing. (If, on the other hand, there's a whole lot of Physics Time, It's probably pretty obvious what's causing that, and that will take priority over scripts getting much chance to run, and overall will feel laggy.)

It may help to know what region this is about. It certainly doesn't seem to be Embraced, which has many times that number of scripts, burning a total of about 11 ms Script Time per frame, yet nonetheless running 100% of the scripts and leaving almost 10 ms of Spare Time. (I was the only avatar in the region at the time, but it must be a different region at issue anyway.)

One other observation: that 590,651 KB number, if that's the sum of script memory, it would take a minimum of 9,228 scripts to use that much. But if it's actually not script memory, but rather Physics "Memory Allocated" (publicly visible in the statistics bar) then indeed the sim would be in big trouble, usually following a lot of terraforming between restarts. (That's apparently a very hard to find memory leak, considering how old the bug is.) Again, though, that's not the current state of Embraced, if that happens to be the region in question.

I'm on Embraced.  There have been times where the sim runs perfectly fine.  The next minute, I can barely move around.  I was told my script count is too high and causing the lag.  I have 7 days left to renew my sim.  I'm a bit skeptical spending another $300 to keep this city running.  Do I mind spending money on something I enjoy?  No.  If it doesn't work, is it worth it?  Either Firestorm is a crappy viewer or something is up with SL servers.  My Internet speed and rig should not be wrestling with lag.  Especially using Alchemy which is heavily run on GPU vs FS and LL's heavy CPU usage.

[Love Zhaoying]

3 minutes ago, Lord Derryth said:

My Internet speed and rig should not be wrestling with lag.  Especially using Alchemy which is heavily run on GPU vs FS and LL's heavy CPU usage.

Did you try the newest LL viewer yet? It is especially tuned for "performance", and lots of people have seen major improvement.

[Coffee Pancake]

8 hours ago, Kardargo Adamczyk said:

Its because 2 homesteads share core and memory on a server, 4 open space regions share a core and memory on a server , a full region gets its own core and memory.

This has already been mentioned but it's such a huge misconception it bares repeating.

Once upon a time, way back when, years and years ago. LL organized their servers into "classes", bigger number more better, the usual thing. Then they stopped telling us about how they managed their datacenter, allowing people to presume things were the same even though they were cleverly finding ways to pack more regions onto one machine. This lack of updates is why people STILL freak out over script memory. It wasn't even true when people were freaking out, it's certainly not true now.

LL have spent years moving stuff off the regions and onto external services. Eg: back in the long forgotten darks days the region would send you all the data, now the region just tells you the name of the data and your viewer grabs it from entirely different services. This has dramatically reduced the amount of work regions actually need to do and can pack more regions together on the same host.

Fast forward to today and it's all on Amazons cloud. One physical machine could be carved up into multiple sim hosts each running an unknown number of regions and homesteads, you might not even stay on the same sim host between region restarts (and you can certainly request a laggy region be moved to a different one)

 

18 minutes ago, Lord Derryth said:

I'm on Embraced.  There have been times where the sim runs perfectly fine.  The next minute, I can barely move around.  I was told my script count is too high and causing the lag.  I have 7 days left to renew my sim.  I'm a bit skeptical spending another $300 to keep this city running.  Do I mind spending money on something I enjoy?  No.  If it doesn't work, is it worth it?  Either Firestorm is a crappy viewer or something is up with SL servers.  My Internet speed and rig should not be wrestling with lag.  Especially using Alchemy which is heavily run on GPU vs FS and LL's heavy CPU usage.

Script time numbers can only be trusted when the region is performing normally. Once a region is in the toilet, script numbers can end up all out of whack.

Also, don't mix up region performance and viewer performance - server side lag and viewer lag are entirely different, entirely unconnected things.

Also also .. All viewers, ALL VIEWERS are built upon the same code from LL and function fundamentally the same way. We fiddle around the edges and improve things we feel are important, but we can't stray too far from the Linden viewer .. or adding updates from LL becomes an impossible amount of work and stuff just breaks.

Some viewer projects really like to talk up the magic their project accomplishes ... don't buy the hype. Alchemy stomps on your CPU in all the same ways as every other viewer, maybe more! Always have the linden viewer in your arsenal of test viewers.

 

Finally, don't look at the maximum biggest numbers when working out what a region can do. You can have UPTO a certain number of stuffs, UPTO a certain amount of scripts, UPTO a certain number of avatars. You probably can't have everything to the max all at once - That would be what the $900 event regions are for .. and even then, if you max out the avatars, you can't max out the scripts and objects.

 

Edited May 16, 2022 by Coffee Pancake

[Lord Derryth]

35 minutes ago, Coffee Pancake said:

Some viewer projects really like to talk up the magic their project accomplishes ... don't buy the hype. Alchemy stomps on your CPU in all the same ways as every other viewer, maybe more! Always have the linden viewer in your arsenal of test viewers.

Misinformation, as I have run numerous tests on all viewers.  Alchemy was the preferred viewer for less lag.  Upon gathering information, it's been said; Alchemy utilizes the GPU more over CPU.  This is why the viewer offers more FPS and eliminates some of the lag.  Firestorm was by far the worst viewer when it came to lag despite the average FPS.  The Second Life viewer is literally nonexistent since textures take forever to load and lacks features that are QoL like Animation Overrider which is essential in eliminating extra huds that kill FPS and causes more lag.  

We were told that moving up to the cloud would make things easier, faster and more reliable.  Unfortunately, that has become false, and we are seeing far worse lag than we did before.  There needs to be an explanation as to why there will be days of perfection as if my sim server is in my home running on my network and then most of the time the server feels it's on planet Pluto.  One of my friends that worked as real estate agent said, "We received a memo stating prices may drop once we go to the cloud."  What happened, why didn't the prices drop?  Is it because of inflation and they are abusing the customer/user by keeping the prices high while paying less for servers?

Again, I appreciate the trolls not chiming in with comments about not getting a sim or leaving it for business.  I use a sim for pleasure, and I enjoy building urban city.  It's come to a point where $300 is not worth spending on a sim that is in constant lag and can barely hold more than 10 people with low script count.  In the end, it always comes down to "wow, you have a lot of scripts running, that's your problem."  Why can't it ever be, "Well, SL is old, the coding is old and nothing will ever work."?

[Coffee Pancake]

I guess there is no point in giving you actual information as you have already decided.

[Love Zhaoying]

1 minute ago, Coffee Pancake said:

I guess there is no point in giving you actual information as you have already decided.

Did anyone offer to go to their sim and see if it also lags for us?

[Lord Derryth]

13 minutes ago, Coffee Pancake said:

I guess there is no point in giving you actual information as you have already decided.

I have not made a decision in anything.  It's not easy to work hard on something and toss it to the side.  This is a huge decision to make and hopefully trying to find a solution.  

 

12 minutes ago, Love Zhaoying said:

Did anyone offer to go to their sim and see if it also lags for us?

Anyone is welcome to come see.  I've removed all my character alts and it's still laggy.  
http://maps.secondlife.com/secondlife/Embraced/41/103/1993

Warning: Adult Sim

Edited May 16, 2022 by Lord Derryth

[Qie Niangao]

I went back and looked at the statistics bar and I'm not seeing any lag at all, server-side. There's still about 8 ms Spare Time, so 100% of scripts get a chance to run each frame—a testament to that recent improvement in script performance, seeing as there are 9800-odd scripts in the region at the moment.

Viewer-side I get between 70 and 90 FPS with full shadows, which seems fine to me. I have an okay i5-based Windows 11 box with RTX 3060 GPU, but nothing state-of-the-art or anything.

It's very possible other folks will experience lag here, but I'm not finding it myself, so we need more data points.

[Aishagain]

I visited the region, as Qie did, just a few minutes ago and while my FPS was not as good as Qie's (Windoze 10 Ryzen 9 3900X GTX1660Ti) my viewer (FS) ran as well as it does in most "good" places at High-Ultra and 256m DD at about 25-30FPS with no detectable lag. so to my mind with only 2 other avs on-sim, (one being Lord Derryth) all seems well.

Bearing in mind that my avatar is above average complexity and highish script count, I am wondering what the fuss is all about. I wandered around for about 5 mins without mishap.

Edited May 16, 2022 by Aishagain

[bigmoe Whitfield]

lag is viewer side.  nobody is lagging an aws instance...

[Mollymews]

7 hours ago, Lord Derryth said:

Anyone is welcome to come see.  I've removed all my character alts and it's still laggy.  
http://maps.secondlife.com/secondlife/Embraced/41/103/1993

Warning: Adult Sim

i went and had a nosey on the Linden Performance Improvement viewer.  A pic of my settings and my computer dets

 

Quote

Second Life Release 6.6.0.571736 (64bit)
Release Notes

You are at 46.9, 103.7, 1,993.0 in Embraced located at simhost-01e69941a35b74281.agni
SLURL: http://maps.secondlife.com/secondlife/Embraced/47/104/1993
(global coordinates 239,407.0, 324,456.0, 1,993.0)
Second Life Server 2022-05-05.571557
Release Notes

CPU: Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz (3400.02 MHz)
Memory: 16318 MB
OS Version: Microsoft Windows 10 64-bit (Build 19043.1706)
Graphics Card Vendor: NVIDIA Corporation
Graphics Card: GeForce GTX 1050 Ti/PCIe/SSE2

Windows Graphics Driver Version: 27.21.14.5671
OpenGL Version: 4.6.0 NVIDIA 456.71

Window size: 2560x1417
Font Size Adjustment: 96pt
UI Scaling: 1
Draw distance: 128m
Bandwidth: 3000kbit/s
LOD factor: 1.125
Render quality: 5
Advanced Lighting Model: Enabled
Texture memory: 512MB
Disk cache: Max size 204.0 MB (102.3% used)

J2C Decoder Version: KDU v7.10.4
Audio Driver Version: FMOD Studio 2.02.03
Dullahan: 1.12.3.202111032221
  CEF: 91.1.21+g9dd45fe+chromium-91.0.4472.114
  Chromium: 91.0.4472.114
LibVLC Version: 3.0.16
Voice Server Version: Not Connected
Packets Lost: 28/14,499 (0.2%)
May 16 2022 18:37:49

 

ps. I like the radio stream

[Lord Derryth]

Remember, dropping draw distance in enclosure helps with FPS.  When in an open world like an Urban City, the experience is ruined.  I run at 168m in order to see from a distance.  I've been plagued with issues for years now and SL seems to be the only place I get lag and struggle.  

[Lucia Nightfire]

3 hours ago, Mollymews said:

i went and had a nosey on the Linden Performance Improvement viewer.  A pic of my settings and my computer dets

 

 

ps. I like the radio stream

10,000+ active scripts...

1300+ script events per second...

[bigmoe Whitfield]

1 hour ago, Lord Derryth said:

Remember, dropping draw distance in enclosure helps with FPS.  When in an open world like an Urban City, the experience is ruined.  I run at 168m in order to see from a distance.  I've been plagued with issues for years now and SL seems to be the only place I get lag and struggle.  

turn off shadows also is a big helper.

[Mollymews]

3 hours ago, Lucia Nightfire said:

10,000+ active scripts...

1300+ script events per second...

ouch!

[Qie Niangao]

But from the sim statistics, this is not suffering any kind of server-side lag. After the recent-ish improvements, these AWS-hosted regions can handle way, way more scripts than we used to expect.

A deep draw distance isn't sim lag either. It's all down to viewer hardware, network bandwidth, and CDN. How to distinguish among those three requires more patience and expertise than I'm prepared for this morning.

One thing occurred to me overnight that probably isn't relevant, but nonetheless: There's a very specific kind of "lag"—a bug, actually—that occurs on some regions, some of the time, that's nothing to do with sim performance, and that's a nasty 2-3 second delay in rezzing objects. I mean, that could be something folks would notice on a role-play region, for example if there are projectiles involved, but no clue if that's a thing here. (I seem to recall this bug is not merely "accepted" but actually in the work queue to be resolved at some point.)

[Lord Derryth]

12 hours ago, Qie Niangao said:

But from the sim statistics, this is not suffering any kind of server-side lag. After the recent-ish improvements, these AWS-hosted regions can handle way, way more scripts than we used to expect.

A deep draw distance isn't sim lag either. It's all down to viewer hardware, network bandwidth, and CDN. How to distinguish among those three requires more patience and expertise than I'm prepared for this morning.

One thing occurred to me overnight that probably isn't relevant, but nonetheless: There's a very specific kind of "lag"—a bug, actually—that occurs on some regions, some of the time, that's nothing to do with sim performance, and that's a nasty 2-3 second delay in rezzing objects. I mean, that could be something folks would notice on a role-play region, for example if there are projectiles involved, but no clue if that's a thing here. (I seem to recall this bug is not merely "accepted" but actually in the work queue to be resolved at some point.)

I know what you mean about that delay.  It's almost like a 3 second rollback.  Maybe my second life days is over. The constant lag is causing stress on my fingers typing and moving around.  Too much to handle.  That's my speed to Portland.  I should have no issues with servers.