Deploy Plans for the week of 2017-11-06

[Caleb Linden]

Second Life Server (Rolled Back):

https://wiki.secondlife.com/wiki/Release_Notes/Second_Life_Server/17#17.10.25.510119

Scheduled Tuesday 2017-11-07 03:00-09:00 PDT

 

No RC Rolls on Wednesday 2017-11-08 07:00-10:30 PDT

See comment from Grumpity

Edited November 7, 2017 by Caleb Linden

[DD Otoole]

Did something happen to the weekly 300L we normally get on tuesdays? (not rushing on the money problem, just i didn't see it yet) And yay more fixes.

 

And my sim went though another restart did something happen...

Edited November 7, 2017 by DD Otoole

[Cindy Evanier]

2 hours ago, DD Otoole said:

Did something happen to the weekly 300L we normally get on tuesdays? (not rushing on the money problem, just i didn't see it yet) And yay more fixes.

 

And my sim went though another restart did something happen...

Haven't had mine yet but they are aware of an issue  https://secondlife-status.statuspage.io/incidents/5hr42m3xzq5t

 

[DD Otoole]

Yup All good now, thank you. Cindy.

[Theron Zemenis]

Were the rolling restarts for the main server channel cancelled or stopped in the middle of them? The grid status page says the restarts are complete as of 08:43 PST, yet i have six regions that i am an estate manager for that show their current server version to be Second Life Server 17.10.06.509394, the version that was deployed a couple/few weeks ago and not the version that was supposed to be deployed today.

[Ayesha Askham]

This is beginning to look like an LL FUBAR.  My region was not rolled so I contacted support.  It was rolled to a new sim but not to the sever version, that according to Caleb above, it should be!

Shrek ears anyone?

ETA , having contacted support again, it appears that the roll was abandoned today.  Watch for updates to the OP.

Edited November 7, 2017 by Ayesha Askham
New info

[Lucia Nightfire]

I had a region restart on the new version then less than an hour later restarted back on the old version:

[11:04:10] Env Monitor (KVP): [REGION_NAME] has restarted.
Start Date & Time: 2017-11-07 @ 04:43:50 PST
Last Est. Stop Time: 2017-11-07 @ 04:07:12 PST, Est. Downtime: 36 minutes, 38 seconds
Last Sim State Save Date & Time: 2017-11-07 @ 04:07:13 PST, Sever Time Loss: 36 minutes, 37 seconds
Last Est. Uptime: 2 days, 23 hours, 37 minutes, 36 seconds
Environment Change: Sim Version & Sim Host have changed.
Current Sim Version: 17.10.25.510119, Prior Sim Version: 17.10.06.509394
Current Sim Host: sim10506.agni.lindenlab.com, Prior Sim Host: sim10047.agni.lindenlab.com

[11:04:10] Env Monitor (KVP): [REGION_NAME] has restarted.
Start Date & Time: 2017-11-07 @ 05:59:02 PST
Last Est. Stop Time: 2017-11-07 @ 05:43:36 PST, Est. Downtime: 15 minutes, 26 seconds
Last Sim State Save Date & Time: 2017-11-07 @ 05:43:38 PST, Sever Time Loss: 15 minutes, 24 seconds
Last Est. Uptime: 59 minutes, 46 seconds
Environment Change: Sim Version & Sim Host have changed.
Current Sim Version: 17.10.06.509394, Prior Sim Version: 17.10.25.510119
Current Sim Host: sim10668.agni.lindenlab.com, Prior Sim Host: sim10506.agni.lindenlab.com

[Ayesha Askham]

This gets curiouser and curiouser.

Wirtz, a mainland LeTigre sim was also restarted today by LL, but while one might reasonably assume some anomaly had been found in 510119 to cause the reversion to 509394, it remained on 510119!

Now why on EARTH restart an RC channel region/server today and stay on 510119, if it had been found to have a problem/exploit?  This whole restart/roll mess today looks like a complete mix-up, so far totally unexplained or even noted. 

[Nalates Urriah]

At the Server-Scripting UG meeting today, Simon Linden told us:

2017/11/07 12:02]  Simon Linden: The update this morning didn't go well so it was canceled.   The issue wasn't the specific new code, but just the load when a few thousand regions were restarting around the same time
[2017/11/07 12:03]  Simon Linden: Tomorrow there isn't an update for the server RCs
[2017/11/07 12:03]  Simon Linden: There should be some new things next week

[Ayesha Askham]

30 minutes ago, Nalates Urriah said:

2017/11/07 12:02]  Simon Linden: The update this morning didn't go well so it was cancelled.   The issue wasn't the specific new code, but just the load when a few thousand regions were restarting around the same time

So what is so different about today's roll compared to the many other Main Server restarts.  Is this a coded message saying that the SL infrastructure is in danger of collapse?  Or was it simply a case of kack-handed server operation?

That does not explain the unusual restart of the LeTigre RC region.

[Nalates Urriah]

33 minutes ago, Ayesha Askham said:

So what is so different about today's roll compared to the many other Main Server restarts.  Is this a coded message saying that the SL infrastructure is in danger of collapse?  Or was it simply a case of kack-handed server operation?

That does not explain the unusual restart of the LeTigre RC region.

The Lindens at the meeting were unaware of the RC region restarts. No one was sure if these were individual regions restart requests, detected problem restarts, or some rouge restarts triggered by the planned main channel restart.

The only thing different about today's planned restart seems to be the unusual backend load they experienced. It is too soon to know why.

[Grumpity Linden]

This morning’s SLS roll went sideways and so changes were rolled back. However, there was an estate-level fix on Mainland that we needed to bring in, so all mainland regions, including those on RC, were restarted.   We do apologize for the unexpected disturbance due to those restarts - we should have communicated them better.  
In other news, there will be no RC roll tomorrow.  

[Ayesha Askham]

Thankyou Grumpity

That was just the reassurance we needed after today's oddities, most welcome!

I AM a bit perplexed by the now-you-see-it now-you-don't....now-you-do behaviour of the GSP over the support downtime!  Never mind, so long as the work get done!

[Grumpity Linden]

That's (fortunately!) got nothing to do with server rolls, but it's due to the handy scheduling feature in the GS blog, which stops being handy when you re-schedule the downtime - it kicked on automatically anyway ... 

The _actual_ downtime should be tomorrow.  

[Willow Wilder]

9 hours ago, Grumpity Linden said:

That's (fortunately!) got nothing to do with server rolls, but it's due to the handy scheduling feature in the GS blog, which stops being handy when you re-schedule the downtime - it kicked on automatically anyway ... 

The _actual_ downtime should be tomorrow.  

 

[Ayesha Askham]

On ‎08‎/‎11‎/‎2017 at 6:51 AM, Grumpity Linden said:

That's (fortunately!) got nothing to do with server rolls, but it's due to the handy scheduling feature in the GS blog, which stops being handy when you re-schedule the downtime - it kicked on automatically anyway ... 

The _actual_ downtime should be tomorrow.  

Following on from that post, Grumpity, I see that despite the scheduled postponement of the Support Maintenance until next Monday...

"Hi Everybody!

Well, this is embarrassing. Our support upgrade needs to be postponed yet again.

Outside factors have come into play and we will be unable to upgrade our service until next week.

What this means is that for our residents:

Support access will remain normal until next Monday the 13th at 3:00 pm. We will then move forward with this upgrade which will have the same expected downtime."

...The GSP is showing Support as "Under Maintenance" today!  So is that still this "handy feature" fooling us all, or is it more "miscommunication"?   

Edited November 9, 2017 by Ayesha Askham