Jump to content

Deploy plan for the week of 2022-01-10


Recommended Posts

  • Lindens

Second Life Server

The main channel will roll to the same version that it is already on, just to keep the simulators running smoothly.

https://releasenotes.secondlife.com/simulator/2021-12-01.566406.html

Tuesday 2022-01-11 06:00-11:00 PST

Second Life RC:

All of the RC regions will be rolled to a new build, on the same tooling project that they were previously on.

https://releasenotes.secondlife.com/simulator/2022-01-06.567269.html

Wednesday 2022-01-12 07:00-09:30 PST

On Region Restarts:

Given that rolling restarts are happening grid-wide, no simulators with >10 day uptime are anticipated to exist after the rolling restarts.

Edited by Maestro Linden
Added release notes link.
  • Like 2
  • Thanks 2
Link to comment
Share on other sites

Just saw a main channel region come back from the restart with the same version in about 4 minutes.  But my restart monitor said it was restarted about 1 hour, 37 minutes ago, and was down for about 1 hour, 37 minutes.

It compares llGetEnv("region_start_time") to the last restart time every 30 seconds, in order to detect restarts.  So the downtime has always been accurate within about 30 seconds.

That seems to suggest that the restart used a backup that was over 90 minutes old, instead of getting a new backup when the region goes offline.  I've never noticed or remember that happening since I've had a monitor set up almost 6 years ago.

Link to comment
Share on other sites

19 minutes ago, Sayrah Parx said:

Just saw a main channel region come back from the restart with the same version in about 4 minutes.  But my restart monitor said it was restarted about 1 hour, 37 minutes ago, and was down for about 1 hour, 37 minutes.

It compares llGetEnv("region_start_time") to the last restart time every 30 seconds, in order to detect restarts.  So the downtime has always been accurate within about 30 seconds.

That seems to suggest that the restart used a backup that was over 90 minutes old, instead of getting a new backup when the region goes offline.  I've never noticed or remember that happening since I've had a monitor set up almost 6 years ago.

Unless your monitor device saves and queries(after restart) a timestamp to/from an external database, the best script memory alone can offer you after a restart is the last saved data before a sim state save occurred.

What seems to have occurred is a time loss of over 1.5 hours. Meaning if the region was only down 4 minutes, but your script sees a time difference of over 1.5 hours, the last sim state save had to have occurred over 1.5 hours ago. This is "typically" the max random time period that elapses before a save state is made.

The sim state save point is the point in time where a region is brought back from, excluding other factors, controlled or not.

A sim state save captures(saves) the state of objects, terrain, land ownership, script memory, etc.

Not all restarts, yield instant sim state saves.

Edited by Lucia Nightfire
Link to comment
Share on other sites

Yeah, that's exactly what I mean.  It's supposed to get a new backup after it goes down for a controlled restart.  That's a major security requirement.  Otherwise it would be easy to farm no-copy objects.  I've never seen the backup anywhere close to over 90 minutes old.  So it seems like there was a crash during the restart process, or there was a bug in the restart process that caused it to not get a new backup.  I don't know if we're allowed to mention region names.

Link to comment
Share on other sites

41 minutes ago, Sayrah Parx said:

It's supposed to get a new backup after it goes down for a controlled restart.  That's a major security requirement.  Otherwise it would be easy to farm no-copy objects.

To be fair, most users that commonly abuse this know there is a chance a sim state save might not be made during the shutdown and do not wait until a region is in countdown before rezzing things. They do it hours before, else they risk losing things to the ether. Also, the ones that commonly abuse it don't even wait on weekly region restarts. They have their estate renting/owning friends do planned restarts for them repeatedly or they repeatedly crash the sim themself without needing controlled restarts.

This is why the argument of not giving users scripted access to knowing if a region is in countdown mode is silly at best.

41 minutes ago, Sayrah Parx said:

So it seems like there was a crash during the restart process, or there was a bug in the restart process that caused it to not get a new backup.

Sim state saves can fail and fail repeatedly, dragging out the time loss if a restart occurs afterwards. I've seen time losses up to 19 hours without planned rollbacks.

I've mostly seen such times in sandbox regions, so IDK if script lag, physics lag or what is a factor.

Edited by Lucia Nightfire
Link to comment
Share on other sites

 

 

3 hours ago, Sayrah Parx said:

Just saw a main channel region come back from the restart with the same version in about 4 minutes.  But my restart monitor said it was restarted about 1 hour, 37 minutes ago, and was down for about 1 hour, 37 minutes.

Two of the regions I am keeping tabs on for the pilot user-monitoring study were down for 89 and 149 minutes, based on the two wall-clock times recorded immediately before restart and immediately after.

 

ETA

Those regions were both mainland. I just got the figures back from the private island that is taking part in the study and they were back in 5 minutes.

Does having neighbouring region perhaps make it take longer?

 

Edited by Profaitchikenz Haiku
Link to comment
Share on other sites

I lost 4 no copy breedables after the restart Tuesday morning.  This was on mainland.  The four lost cats were attached to my avatar for an hour before the restart.  I dropped them back on land about 10 minutes before the restart.   It's not my land so I didn't have a region monitor to see the downtime .  But it surely did not restart with a recent safety  backup, as used to be normal.

 

 

 

Link to comment
Share on other sites

4 hours ago, Jaylinbridges said:

I lost 4 no copy breedables after the restart Tuesday morning.  This was on mainland.  The four lost cats were attached to my avatar for an hour before the restart.  I dropped them back on land about 10 minutes before the restart.   It's not my land so I didn't have a region monitor to see the downtime .  But it surely did not restart with a recent safety  backup, as used to be normal.

 

 

 

Sim state saves are not guaranteed with every controlled region shutdown and restart.

Link to comment
Share on other sites

@Lucia NightfireReally?  Why not?  Surely, the implication is, considering the preceeding post from Jaylinbridges, that we are to expect data loss at every restart?  I'm pretty sure that didn't happen pre uplift.  Presumably LL have lost the fine control of the server restart process they used to have at the data centre?

How is that going to be acceptable to users and if users become disenchanted, they will leave and SL will cease.

Edited by Aishagain
Link to comment
Share on other sites

1 hour ago, Aishagain said:

Surely, the implication is, considering the preceeding post from Jaylinbridges, that we are to expect data loss at every restart?

I don't know how to say it any differently than what I have been.

1 hour ago, Aishagain said:

I'm pretty sure that didn't happen pre uplift.

It's been the case ever since I started tracking region restarts 15 years ago.

1 hour ago, Aishagain said:

How is that going to be acceptable to users and if users become disenchanted, they will leave and SL will cease.

This is nothing new.

Link to comment
Share on other sites

That all depends on what you mean by "pudding".

One use refers to the generic desert..."how can ye have yer pudding if ye don't  eat yer meat?"

The second use refers to a rich, often baked desert frequently eaten with a sauce.

The 18th century political rhyme "Georgy porgy pudding and pie" would suggest they are different things.

It could also be that I don't know what I am talking about...."Nurse! The screens!"

Exit, stage left, wibbling.

 

Edited by Aishagain
Insanity
  • Haha 3
Link to comment
Share on other sites

My home region (Hunburgh) is currently running 2022-01-06.567269 and I noticed (it has been lasting for a few days) that when logging in it, the friends list is not updated (online friends appear as offline in the list), while when logging in in a neighbouring region with a different server version, everything works fine in this respect.

Anyone else noticed it ?

Edited by Henri Beauchamp
Link to comment
Share on other sites

8 hours ago, Henri Beauchamp said:

My home region (Hunburgh) is currently running 2022-01-06.567269 and I noticed (it has been lasting for a few days) that when logging in it, the friends list is not updated (online friends appear as offline in the list), while when logging in in a neighbouring region with a different server version, everything works fine in this respect.

Anyone else noticed it ?

It sounds like this bug, which as far as I know has never been fixed  BUG-7557 - Online Friends are not showing when login (Once again)

Restarting the affected region should fix it.

Edited by Whirly Fizzle
  • Thanks 1
Link to comment
Share on other sites

@Monty LindenSo...are LL knowingly going to promote a bug to Main Server tomorrow?  After all the fuss in various in-world groups, the noise cannot have failed to register with the Lindens, and now you have heard it from a trusted source in Whirly and Henri.

Come on, it would be stupid to promote a bug that affects nearly all accounts.  Wouldn't it?

Edited by Aishagain
Link to comment
Share on other sites

26 minutes ago, Aishagain said:

@Monty LindenSo...are LL knowingly going to promote a bug to Main Server tomorrow?  After all the fuss in various in-world groups, the noise cannot have failed to register with the Lindens, and now you have heard it from a trusted source in Whirly and Henri.

Come on, it would be stupid to promote a bug that affects nearly all accounts.  Wouldn't it?

Eh, that bug is over 8 years old. Another year won't hurt  ;)

Edited by Whirly Fizzle
  • Like 1
  • Confused 1
Link to comment
Share on other sites

@Whirly FizzleI am saddened that you are so blasé about this.  It is generating a LOT of issues in SL and if ALL regions are bugged how on earth are we supposed to get the names to show?

Does the issue resolve with a region restart still?

ETA 17.01.2021:  I am corrected by Whirly.  This appears to be a new but outwardly similar bug.

It is surely even more important that it isn't propagated across the Grid.

 

Edited by Aishagain
Link to comment
Share on other sites

13 minutes ago, Aishagain said:

@Whirly FizzleI am saddened that you are so blasé about this.  It is generating a LOT of issues in SL and if ALL regions are bugged how on earth are we supposed to get the names to show?

Does the issue resolve with a region restart still?

 

 

I thought she was paraphrasing someone from management, personally. 😆

  • Haha 2
Link to comment
Share on other sites

43 minutes ago, Aishagain said:

@Whirly FizzleI am saddened that you are so blasé about this.  It is generating a LOT of issues in SL and if ALL regions are bugged how on earth are we supposed to get the names to show?

Does the issue resolve with a region restart still?

 

 

I think you are fretting about a different bug, which is new & started a few days ago.

I just filed it. BUG-231680 - Friends user names often stuck as (Loading) over the last few days for many users
This one is being complained about a lot in groups.

Edited by Whirly Fizzle
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...