Deploy Plan for the Week of 2020-10-26

[Mazidox Linden]

Second Life Server

No roll to Second Life Server this week.

Second Life RC

We will be moving a couple thousand additional simulators to the cloud during the normal RC window

Scheduled Wednesday 2020-10-28 07:00-10:30 PDT

Uplifted Simulators

We put out an updated version of our cloud-based simulator today to all currently uplifted regions. The release notes for that deploy can be found at:
https://releasenotes.secondlife.com/simulator/2020-10-26.551155.html

(To be clear, these release notes comprise just a small fraction of the changes we had to make to get this far. 90% or more of the effort was things behind the scenes that needed to be changed so that from your perspective, everything works the same.)

On Region Restarts

This is where you expect me to say that you should expect rolling restarts, right? And it's not out of the question, but we're currently in the process of uplifting the Grid Poking Bot that handles the restarts, so you may not see them this week.

[Lucia Nightfire]

I hope it fixes all the tp timeout crashing. I had it happen again trying to teleport from Simon's UG...

[Teresa Firelight]

The Region that one of my stores is on (uses caspervend) got uplifted today and it is performing flawlessly. Vendors/scripts all working, easy to TP in/out... working great so far

[Rachel1206]

Regions I visited, now running on AWS, everything smooth and faster; texture loads, scripts, avatar rezzing - looking good the uplift  👍.

[arabellajones]

I did a little bit of wandering around yesterday, and any mainland region is likely close to another running on the Cloud. A roughly 1 in 10 chance of a region being on the Cloud, and a possible 8 regions adjacent to each Mainland region. At the moment we have the domain name in Help>About as a clear sign, there are also tools which report server-version changes, and the Cloud regions are the only ones on 551155.

The way server code and physical hardware are arranged could affect the apparent randomness, but random surveys that aren't are a well known problem in the science of statistics. I have a tiny sample size, and I spotted about twice the number of Cloud regions than a crude 1 in 10 would imply. My considered judgement on that is "So what?"

Things such as the weekly sailing, flying, and driving events easily pass through a hundred regions, and it would be remarkable if none of them were on the Cloud. 

[Logan Elf]

As far as I can tell, they uplifted all of RC BlueSteel

[Miller Thor]

We were migrated to the cloud today at 2:30am CET and in the course of long-term monitoring, as has already been mentioned, we were able to see that both the texture load and the overall performance were able to experience a noticeable increase. But it is particularly striking that the stability increased very much, both in general and in particular in the area of script performance. The fluctuations in script performance, which were previously massively noticeable, are now limited to a minimum of + - 1 to 1.5%. This is a massive advantage, especially for heavily used regions.  However, it does not in any way relieve the region owners of regularly restarting the regions and, despite everything, to pay attention to scriptusage. So no carte blanche for lazy people who now believe they can be even more lazy through the cloud. The change from one region to another is also almost imperceptible over time. Of course, there may be a slight wobbler one or the other time, but this is so minimal that it hardly stands out.

Edited October 29, 2020 by Miller Thor

[AlettaMondragon]

7 hours ago, arabellajones said:

The way server code and physical hardware are arranged could affect the apparent randomness, but random surveys that aren't are a well known problem in the science of statistics. I have a tiny sample size, and I spotted about twice the number of Cloud regions than a crude 1 in 10 would imply. My considered judgement on that is "So what?"

Things such as the weekly sailing, flying, and driving events easily pass through a hundred regions, and it would be remarkable if none of them were on the Cloud. 

There's one way to get a proper sample size. Drive through all continents on the roads, then on the seas, lakes, channels to cover those regions as well, and scan them all with an "uplift detector". By the time you'd finish with the first scan though, there would be probably more few thousands uplifted in the next round, so you still couldn't make accurate statistics of the uplift process.

Edited October 29, 2020 by AlettaMondragon
i cnat tpye

[Candide LeMay]

1 hour ago, Miller Thor said:

We were migrated to the cloud today at 2:30am CET and in the course of long-term monitoring, as has already been mentioned, we were able to see that both the texture load and the overall performance were able to experience a noticeable increase. But it is particularly striking that the stability increased very much, both in general and in particular in the area of script performance. The fluctuations in script performance, which were previously massively noticeable, are now limited to a minimum of + - 1 to 1.5%. This is a massive advantage, especially for heavily used regions.  However, it does not in any way relieve the region owners of regularly restarting the regions and, despite everything, to pay attention to scriptusage. So no carte blanche for lazy people who now believe they can be even more lazy through the cloud. The change from one region to another is also almost imperceptible over time. Of course, there may be a slight wobbler one or the other time, but this is so minimal that it hardly stands out.

There have been some reports (and I have seen it myself on one sim that migrated to aws) that the same set of scripts take up more milliseconds of the frame time on the aws sims compared to non-cloud. So my question is - is the scripts run % and script events/s higher on your migrated sim compared to previously or just more stable?

[Lucia Nightfire]

3 hours ago, Candide LeMay said:

So my question is - is the scripts run % and script events/s higher on your migrated sim compared to previously or just more stable?

Ideally, both would be higher, unless there are so many active, but idle scripts that the "cost savings" is negated.

Idle scripts currently use more script time on AWS.

[Aishagain]

43 minutes ago, Lucia Nightfire said:

Idle scripts currently use more script time on AWS.

Why is that?

[Phate Shepherd]

Home region is uplifted. I am curious... if you are on an uplifted region, will you stay on AWS even after a restart, or could you be migrated back?

Many of my scripts are running at double the speed as before. I saw the speed increase on the beta grid, but was wary that it would translate to the production grid.

[Lucia Nightfire]

5 hours ago, Aishagain said:

Why is that?

I brought this to developers' attentions Tuesday.

The reply I got from them was "That's interesting.".

AFAIK, there is no current investigation into why nor one planned.

1 hour ago, Phate Shepherd said:

Home region is uplifted. I am curious... if you are on an uplifted region, will you stay on AWS even after a restart, or could you be migrated back?

Many of my scripts are running at double the speed as before. I saw the speed increase on the beta grid, but was wary that it would translate to the production grid.

I've done various script function tests and on average, most perform 2x - 2.5x as fast on AWS than not. There are some weird random exceptions, such as llListReplaceList() not having a significant performance increase.

There is still the issue with KVP reads taking 3x longer to execute and writes taking 6x longer, but that is mostly to do with the KVP servers still needing migration.

Edited October 30, 2020 by Lucia Nightfire

[Logan Elf]

2 hours ago, Phate Shepherd said:

Home region is uplifted. I am curious... if you are on an uplifted region, will you stay on AWS even after a restart, or could you be migrated back?

A  restart won't revert. Unless there's some really catastrophic failure, I very much doubt a mainland region will be migrated back. Private estates can ask to be temporarily migrated back , but will have to provide very good reasons to do so.