Deploy Plans for the week of 2018-01-08

[Caleb Linden]

Happy New Year!

Second Life Server (restarts only):

https://wiki.secondlife.com/wiki/Release_Notes/Second_Life_Server/17#17.12.01.511131

Scheduled Tuesday 2018-01-09 03:00-09:00 PST

 

Second Life RC BlueSteel:

https://wiki.secondlife.com/wiki/Release_Notes/Second_Life_RC_BlueSteel/18#18.01.08.511751

Scheduled Wednesday 2018-01-10 07:00-10:30 PST

 

Second Life RC LeTigre:

https://wiki.secondlife.com/wiki/Release_Notes/Second_Life_RC_LeTigre/18#18.01.08.511751

Scheduled Wednesday 2018-01-10 07:00-10:30 PST

 

Second Life RC Magnum:

https://wiki.secondlife.com/wiki/Release_Notes/Second_Life_RC_Magnum/18#18.01.08.511751

Scheduled Wednesday 2018-01-10 07:00-10:30 PST

Edited January 10, 2018 by Caleb Linden

[Mitch Absinthe]

You are showing 75% packet loss on your servers. They probably need to be rebooted. Strange that your network people did not pick this up

[Theresa Tennyson]

6 hours ago, ispdude said:

You are showing 75% packet loss on your servers. They probably need to be rebooted. Strange that your network people did not pick this up

Theresa Tennyson looks at this statement and the account name of the person who wrote it and murmurs to herself, "Well, that explains a lot..."

[MBeatrix]

6 hours ago, ispdude said:

You are showing 75% packet loss on your servers. They probably need to be rebooted. Strange that your network people did not pick this up

Huh?

[Ayesha Askham]

As regards the OP, I guess we all feel the same about the post.  I will not add to it.

As regards the "scheduled" restarts to Main Server sims,  during a period of base service maintenance...IS THIS WISE?  I am also minded that some Main Server simulators were restarted last week (mine was), and if that was LL, why are you doing it again this week?  You left some RC channel Mainland sims un-restarted for 3 weeks just prior to the end of 2017!

Happy New Year, Linden Lab, but please resolve to know where your ass is so that you can find it in the dark with at least ONE hand this year.

[Minuet Voir]

3 hours ago, Ayesha Askham said:

As regards the OP, I guess we all feel the same about the post.  I will not add to it.

As regards the "scheduled" restarts to Main Server sims,  during a period of base service maintenance...IS THIS WISE?  I am also minded that some Main Server simulators were restarted last week (mine was), and if that was LL, why are you doing it again this week?  You left some RC channel Mainland sims un-restarted for 3 weeks just prior to the end of 2017!

Happy New Year, Linden Lab, but please resolve to know where your ass is so that you can find it in the dark with at least ONE hand this year.

I never feel the same way as you do about these posts. 

I guess you can blame me, in part, for the restarts, as I suggested and encouraged it at the last SBUG based on support issues observed. 

Happy New Year, Ayesha, but please resolve to be less of a nag. 

[Ayesha Askham]

45 minutes ago, Minuet Voir said:

I never feel the same way as you do about these posts. 

I guess you can blame me, in part, for the restarts, as I suggested and encouraged it at the last SBUG based on support issues observed. 

Happy New Year, Ayesha, but please resolve to be less of a nag. 

Oh dear Minuet

I am sorry that you think me a nag.  As to a resolution to nag less, sorry no can do.  The Maintenance took priority today and going by the Grid Status page Main server restarts never happened.  As I said some (I don't know just how many) occurred last week and the RC channels are, in the case of some regions, in dire need of a restart by LL (being on Mainland).  I am surprised that the SBUG was not aware of it.

I COULD go through my second life not caring one jot about others and just blithely going about my SL, but I am simply not made that way.  Linden Lab set themselves standards.  If they fail to meet them I will remark on it, sometimes in a way that irritates others.  That is my cross to bear and I bear it. 

[Minuet Voir]

19 hours ago, Ayesha Askham said:

Oh dear Minuet

I am sorry that you think me a nag.  As to a resolution to nag less, sorry no can do.  The Maintenance took priority today and going by the Grid Status page Main server restarts never happened.  As I said some (I don't know just how many) occurred last week and the RC channels are, in the case of some regions, in dire need of a restart by LL (being on Mainland).  I am surprised that the SBUG was not aware of it.

I COULD go through my second life not caring one jot about others and just blithely going about my SL, but I am simply not made that way.  Linden Lab set themselves standards.  If they fail to meet them I will remark on it, sometimes in a way that irritates others.  That is my cross to bear and I bear it. 

Right. Well martyrdom is serious business.  

[Norton Burns]

On 09/01/2018 at 5:24 AM, ispdude said:

You are showing 75% packet loss on your servers. They probably need to be rebooted. Strange that your network people did not pick this up

Ahh, that explains everything ! </sarcasm>

 

[animats]

There really is excessive load on some servers even when nothing is happening in an individual sim. Here I am on a nice morning in Vallone, in Kama City. I'm the only avatar in Vallone. Yet lag is awful. Can't drive a vehicle properly. Viewer lag meter reports both the sim running slow and network lag. The network lag is all at SL's end. I'm running a ping to the sim server in the background, and it's consistently reporting 40-50ms ping times with no packet loss. Traffic on the network link is very low (it's a 30Mb/s link loaded to 0.088Mb/s). (Remember, 1 Byte/s = 8 bits/s).Viewer side, CPU load is low (4-core client machine). It's been like this for days.

Okay, how many sims did they load onto this poor server?

Some days, I suspect that all of Kama City is running on one 1U rackmount server.

[Qie Niangao]

15 hours ago, animats said:

Viewer lag meter reports both the sim running slow and network lag.

I peeked in on the sim to see if I could find any of "the usual suspects" for Mainland lag, but this one is different than I usually see. The only lag I found was very spiked "Pump IO". Performance will be cruising along perfectly normally for a few seconds and then suddenly add hundreds of milliseconds of "Pump IO" to the simulation frame. A little slice of the spikes get accounted to "Net Time" but the vast majority goes to "Spare Time".

Vallone is running BlueSteel 18.01.08.511751 -- not sure if this warrants a jira, or if anybody would want to investigate what's happening before a restart removes all evidence.

[animats]

11 hours ago, Qie Niangao said:

I peeked in on the sim to see if I could find any of "the usual suspects" for Mainland lag, but this one is different than I usually see. The only lag I found was very spiked "Pump IO". Performance will be cruising along perfectly normally for a few seconds and then suddenly add hundreds of milliseconds of "Pump IO" to the simulation frame. A little slice of the spikes get accounted to "Net Time" but the vast majority goes to "Spare Time".

Vallone is running BlueSteel 18.01.08.511751 -- not sure if this warrants a jira, or if anybody would want to investigate what's happening before a restart removes all evidence.

It's a spiky problem, as you said. No lag, then huge lag for a second or so, then no lag again.

I put up a "lag meter" column on my lot. It drops to yellow and red briefly every 10 seconds or so. If I cross the street from Vallone to Kama Center, with zero avatars now in Vallone, that stops. But the neighbor across the street has an identical lag meter (mine is a copy of his) and it doesn't drop to yellow or red when I'm on that sim. I don't have an exotic avatar or any unusual outfits. I've tried going back to a standard-issue newbie avatar from White Tiger, and the lag still happens.

If you want to see this, go to Vallone (249, 7, 36). Try crossing the streets while watching the lag meter at the corner.

Edited January 15, 2018 by animats
Suggest test

[Ayesha Askham]

This spiky lag issue is apparently due to "child agent" creation events when changes occur on neighbouring regions.  Apparently the adjacent region does not need to be on the same server, which I find remarkable.

The phenomenon has been observed on Mainland and there is a JIRA on it , though I don't have the number.

Whether the issue is being actively investigated or not I do not know, but raising another JIRA wold soon show that.

I would guess that it has been introduced accidentally but it has been observed for some time on SL Ponyplay sims that are not discrete islands.

Edited January 16, 2018 by Ayesha Askham
Further text

[Ayesha Askham]

On a different topic I see that the Grid Status Page announced belatedly at 6am SLT that there was a Main Server restart in progress.  No details are available in this forum, so just what is to be rolled is anyone's guess.  An educated guess would be that Main Server is getting the software version rolled to the Release Candidate channels last week.

Is it too much to expect a timely announcement of such actions?  I know there was a special day on Monday but what was stopping someone putting up the notice in the days prior to it?

I think I already know the answer.

 

ETA at c2pm SLT: 

I see from other posts in this forum on another thread that they were encountering:  "Cannot connect to a simulator".  well of course that simply means that their destination region was offline being restarted.  This is the sort of thing that can occur when no post is made in this forum regarding the rolls and at that time, 3 hours into the normal schedule for a Main Server Restart/Roll there was nothing on the GSP.  People are on the whole unaware of the info given out by LL, and the principal reason is that the GSP and even this forum are no longer reliable indicators of what is happening in SL. in SL's 15th year this is very poor customer service.

Edited January 16, 2018 by Ayesha Askham
Further information