What's up with sim crossings?

[Odaks]

4 minutes ago, MBeatrix said:

applying a band-aid hoping to cure a serious disease with it.

That's exactly what I was hoping they hadn't done, too! So, fingers crossed!

[Aishagain]

Having TP'd about a fair bit at Fantasy Faire just now and crossed a few regions on foot at the same time I will just say that either today is my lucky day or whatever LL rolled this week seems to have made a substantial improvement.  I hope I can still say this in a few days!

[animats]

Region crossings are not fixed. I tried going from Bellasaria to Jeoghot in a speedboat, which I've been able to do before. I got as far as crossing from Nudibranch to Pearl Drop, just west of Jeoghot. Those are all new sims of open water. Boat disappeared. Stuck at bottom of ocean. Controls stuck. Avatar health "Stop avatar animations and revoke permissions" didn't get control back. A short teleport on the world map did.

I have a swim HUD, so I tried swimming to a nearby island. A Coast Guard boat and helicopter went by, and I shouted to them, but they didn't reply. I tried swimming after the boat, and at every region crossing, there was a 10 second pause (exactly 10 seconds) and then I was forced to the bottom of the sea. The swim HUD would bring me back up. Kept swimming after the Coast Guard boat, but never caught up. I'd see the Coast Guard boat in the distance once in a while.  They didn't seem to make it all the way to Bellasaria. They disappeared south of the new continent, and later showed up at the new airstrip in Coral Waters.

Swam all the way back to the southern lighthouse of Bellasaria. That took half an hour. Rezzed a boat at the new rez zone. Once in the boat, everything worked fine, and I boated back to my house and docked the boat.

We're still at about 1-2 failures per hour. Worse for multiple passengers.

 

[Odaks]

11 minutes ago, animats said:

A Coast Guard boat and helicopter went by, and I shouted to them, but they didn't reply.

They really missed out on a golden opportunity for some headline-grabbing publicity.

I'd keep working on your sea survival skills if I were you!

[MBeatrix]

1 hour ago, animats said:

Region crossings are not fixed. I tried going from Bellasaria to Jeoghot in a speedboat, which I've been able to do before. I got as far as crossing from Nudibranch to Pearl Drop, just west of Jeoghot. Those are all new sims of open water. Boat disappeared. Stuck at bottom of ocean. Controls stuck. Avatar health "Stop avatar animations and revoke permissions" didn't get control back. A short teleport on the world map did.

I have a swim HUD, so I tried swimming to a nearby island. A Coast Guard boat and helicopter went by, and I shouted to them, but they didn't reply. I tried swimming after the boat, and at every region crossing, there was a 10 second pause (exactly 10 seconds) and then I was forced to the bottom of the sea. The swim HUD would bring me back up. Kept swimming after the Coast Guard boat, but never caught up. I'd see the Coast Guard boat in the distance once in a while.  They didn't seem to make it all the way to Bellasaria. They disappeared south of the new continent, and later showed up at the new airstrip in Coral Waters.

Swam all the way back to the southern lighthouse of Bellasaria. That took half an hour. Rezzed a boat at the new rez zone. Once in the boat, everything worked fine, and I boated back to my house and docked the boat.

We're still at about 1-2 failures per hour. Worse for multiple passengers.

 

Well, I tried about everything (even hit a corner) and didn't manage to get a crash/disconnect, both by sea and air.

Experience tells me that SL "eating" a vehicle is a symptom of an unhealthy sim, not a vehicle crash due to a transition between regions.

If you're on Firestorm, and you get stuck after a vehicle crash, you can try re-creating the LSL bridge. Though it doesn't always fix it, sometimes doing that will get your direction keys working again.

Edited April 25, 2019 by MBeatrix
typo correction

[Fluf Fredriksson]

1 hour ago, animats said:

Region crossings are not fixed. I tried going from Bellasaria to Jeoghot in a speedboat, which I've been able to do before. I got as far as crossing from Nudibranch to Pearl Drop, just west of Jeoghot. Those are all new sims of open water. Boat disappeared. Stuck at bottom of ocean. Controls stuck. Avatar health "Stop avatar animations and revoke permissions" didn't get control back. A short teleport on the world map did.

 

 

I know you don't want to hear this, but that's not the problem that LL have been trying to fix for a month. You didn't get disconnected. The random disconnection issue is fixed.

[JoyofRLC Acker]

2 hours ago, animats said:

Region crossings are not fixed. I tried going from Bellasaria to Jeoghot in a speedboat, which I've been able to do before. I got as far as crossing from Nudibranch to Pearl Drop, just west of Jeoghot. Those are all new sims of open water. Boat disappeared. Stuck at bottom of ocean. Controls stuck. Avatar health "Stop avatar animations and revoke permissions" didn't get control back. A short teleport on the world map did.

I have a swim HUD, so I tried swimming to a nearby island. A Coast Guard boat and helicopter went by, and I shouted to them, but they didn't reply. I tried swimming after the boat, and at every region crossing, there was a 10 second pause (exactly 10 seconds) and then I was forced to the bottom of the sea. The swim HUD would bring me back up. Kept swimming after the Coast Guard boat, but never caught up. I'd see the Coast Guard boat in the distance once in a while.  They didn't seem to make it all the way to Bellasaria. They disappeared south of the new continent, and later showed up at the new airstrip in Coral Waters.

Swam all the way back to the southern lighthouse of Bellasaria. That took half an hour. Rezzed a boat at the new rez zone. Once in the boat, everything worked fine, and I boated back to my house and docked the boat.

We're still at about 1-2 failures per hour. Worse for multiple passengers.

 

One or two per hour is a big improvement on last week.  Many of the people I race (and myself) with would get a disconnect after 15 - 30 sims.  

Another data point ... we had two races today around noon SLT Blake (and China thru Baltic).  4 boats completed both course.  This is a first in many weeks. 

Regarding the 10 second pause, this is interesting.   Today I was racing a boat that is not the best at sim crossings generally (it has other virtues and I was in the mind for a stress test).
It always pauses for a variable 4 - 8 "seconds" (using 'one missippi' type counting on good sample of crossings )  on sim borders - today the delays were less variable.   And heres the odd thing, today I think the boat did actually stop - there was 'leap ahead' when the freeze ended (I have the viewer extrapolation turned off).

My sense is (anecdotal, not hard data) is that something is making things better, maybe by masking the problem, but that its not totally fixed.

One other thing that have noticed all along, and its probably just a symptom that when the avi and vehicle get separated (part of the warm up for a disconnect) there is no change event for the detach triggered in the scripts.

 

[MBeatrix]

53 minutes ago, Fluf Fredriksson said:

 

I know you don't want to hear this, but that's not the problem that LL have been trying to fix for a month. You didn't get disconnected. The random disconnection issue is fixed.

Exactly.

I still had no disconnects on teleports (or any other way.) Less than 30 minutes ago, I even teleported from Midas (Fantasy Faire region) to another place without any delay. And Midas is an extremely laggy region, which is a pity, as it's an amazing build.

[animats]

1 hour ago, Fluf Fredriksson said:

I know you don't want to hear this, but that's not the problem that LL have been trying to fix for a month. You didn't get disconnected. The random disconnection issue is fixed.

In which server release? Nothing like that was mentioned in the release notes or at Server User Group.

[Fluf Fredriksson]

14 minutes ago, animats said:

In which server release? Nothing like that was mentioned in the release notes or at Server User Group.

Resolved - We have finished updating main grid regions with a fix which should reduce incidence of teleport disconnects. Thank you for your patience during this time.
Apr 25, 15:37 PDT

 

https://secondlife-status.statuspage.io/incidents/10gthjdtjjky

https://secondlife-status.statuspage.io/incidents/3g3fshq350wq

[Jo Clip]

8 hours ago, MBeatrix said:

as it would be just like applying a band-aid hoping to cure a serious disease with it.

Seems that's exactly what they did. Ok true I won't disconnect anymore, but instead the boat will just unsit me, exactly like this :

 

4 hours ago, animats said:

Region crossings are not fixed. I tried going from Bellasaria to Jeoghot in a speedboat, which I've been able to do before. I got as far as crossing from Nudibranch to Pearl Drop, just west of Jeoghot. Those are all new sims of open water. Boat disappeared. Stuck at bottom of ocean. Controls stuck. Avatar health "Stop avatar animations and revoke permissions" didn't get control back. A short teleport on the world map did.

Unfortunately Fluf is right they now claim the issue is addressed...

I can't even imagine racing anymore in those horrible conditions... And who knows how long it will now take for them to admit the new issues ? 2 months ? 3 ? and 1 another to fix it ? "if" only they can someday ?

Edited April 25, 2019 by Jo Clip
Typo

[Theresa Tennyson]

42 minutes ago, Jo Clip said:

Seems that's exactly what they did. Ok true I won't disconnect anymore, but instead the boat will just unsit me, exactly like this :

Unfortunately Fluf is right they now claim the issue is addressed...

I can't even imagine racing anymore in those horrible conditions... And who knows how long it will now take for them to admit the new issues ? 2 months ? 3 ? and 1 another to fix it ? "if" only they can someday ?

Unsitting is a "new" issue? Seriously?

[Fluf Fredriksson]

@animats It sounds like you've collected a lot of information on the "old style" sim crossing problems. If you haven't already, it's probably worth creating a JIRA bug with the problems you know of laid out as clearly as possible.

[Jo Clip]

As a replacement of the previous crashes, yes it's new.

but yes, xDDDDDDDDDDDD true, not 'so' new.

[animats]

3 hours ago, Fluf Fredriksson said:

@animats It sounds like you've collected a lot of information on the "old style" sim crossing problems. If you haven't already, it's probably worth creating a JIRA bug with the problems you know of laid out as clearly as possible.

Done over a year ago.

[JoyofRLC Acker]

Ok, now Im confused.   Could someone please classify what the various problems discussed here are.   Also please conform or correct my understanding:

- vehicle sim crossings have always had a number of problems and the frequency has varied over time.   these are include (but theres probably more Im overlooking):
      submarining / flying
      falling off, agent and vehicle appear to be in different sims (not always adjacent) - this sometimes follows apparently normal travel through 1 or 2 sims but the vehicle is non-responsive
      cam angle /focus goes wonky
      getting logged out (rare until the last few weeks)

- TP can 'stall' for a number of reasons; and that may follow with a disconnect - this was not too common until the last few weeks

My understanding is that a disconnect during a vehicle crossing is thought to be strongly related to the TP disconnect problem.   For me the vehicle crossing problem was far more critical as I wasnt having any unusual problems with TP.

I have also had it said to me, and not just recently, that the other region crossing problems are caused by 'bad server hand offs' which sounds an awful like what is being said specifically about the TP problem.

LL has done a server release that will "REDUCE incidence of TP disconnects" (my emphasis).   They do not say what they have done, nor that the fix will ELIMINATE the disconnects.   They don't even say how much of a reduction there should be.

The big concern obviously is that if they haven't gotten to the root cause, the band-aid will fall off at some point.

Hopefully the next Server User Group will shed more light on the situation.   Personally I'm still nervous.

 

 

 

 

[Imsaho Chau]

So has this been fixed or not?

Edited April 27, 2019 by Imsaho Chau

[animats]

23 minutes ago, Imsaho Chau said:

So has this been fixed or not?

TP, maybe. Region crossings, no.

[Imsaho Chau]

My interest in this is .. I started sailing with friends and found it fun but .. marred by the sim crossing issue particularly last week. I have been reading in this forum because my day job involves to some degree multi processor communications and I can appreciate that it looks like the thing is a nasty issue to fix. From what I can tell it's a random failure .. the worst kind because it's the most difficult to reproduce. The only way is to instrument the issue but that has its own problems.. namely by changing timings and seemingly fixing the issue. So that was why i asked the question .. do we know if the lindens have fixed it, or have they just put logging services in and changed the operation so that it 'seems' more reliable

Edited April 27, 2019 by Imsaho Chau

[Fluf Fredriksson]

4 hours ago, Imsaho Chau said:

So has this been fixed or not?

As of April 25th. Yes this is fixed for both sim crossings and teleports.

"This" is random disconnections on sim crossings or teleports. Having a problem crossing a sim boundary smoothly was not fixed and was not what LL was trying to fix.

But I find I'm getting fewer "unsit" bugs on sim crossings now as well as no random disconnect issues.

If you are still getting disconnected on sim crossings or teleports, it's possible it's a problem with your computer / network. If that all looks good then you probably need to raise a JIRA describing the problem, because for apparently the vast majority of people, the random disconnection issue is solved.

Also. Nope. The Linden's haven't told us what they did and it seems highly unlikely that they will.

[Qie Niangao]

5 minutes ago, Fluf Fredriksson said:

Also. Nope. The Linden's haven't told us what they did and it seems highly unlikely that they will.

Yeah. And it's really none of my damned business whether this was a bug in the EEP code, say, or some gotcha in the OS upgrade, or something else altogether. Except, that is, to know how far back the grid is now from where we thought we were a few weeks ago, before everything went to hell in a handbasket. They might kinda owe us that much after all this. Although I'd gladly stay ignorant in return for a substantial improvement in sim crossings over status quo ante.

[MBeatrix]

2 hours ago, Imsaho Chau said:

My interest in this is .. I started sailing with friends and found it fun but .. marred by the sim crossing issue particularly last week. I have been reading in this forum because my day job involves to some degree multi processor communications and I can appreciate that it looks like the thing is a nasty issue to fix. From what I can tell it's a random failure .. the worst kind because it's the most difficult to reproduce. The only way is to instrument the issue but that has its own problems.. namely by changing timings and seemingly fixing the issue. So that was why i asked the question .. do we know if the lindens have fixed it, or have they just put logging services in and changed the operation so that it 'seems' more reliable

I'm not hoping that sim crossings get fixed so soon (so soon=years) as it's always been a cyclic thing. There has been periods when they got really bad, and then, without any apparent fix, they get back to normal (normal=occasional vehicles crashes.)

LL never showed they cared about it since I joined SL, except some months ago when they said they were trying something to make it less bad, but whatever they did clearly didn't work.

[EDIT] Also, please keep in mind that sometimes regions enter an unhealthy state due to traffic or some other reason, so not all crashes are caused by bad communication between regions. I mean, sometimes we cross into a region that is already buggered and that's why we crash and our boat is "eaten", and/or mesh we are wearing disappears, the boat is seen in bits, etc.

Edited April 27, 2019 by MBeatrix
adding more info

[Imsaho Chau]

48 minutes ago, Qie Niangao said:

Yeah. And it's really none of my damned business whether this was a bug in the EEP code, say, or some gotcha in the OS upgrade, or something else altogether.

My only concern is that if the problem has been masked by debug code rather than fixed then it is there to come back and bite us some time in the future. As someone said earlier .. if it's just a band aid then eventually the band aid comes off. Having said that, i would agree with you. I suspect that the lindens have run a risk analysis and estimated that the solution would cost more than living with the fault.

[MBeatrix]

11 minutes ago, Imsaho Chau said:

[..] lindens have run a risk analysis and estimated that the solution would cost more than living with the fault.

That's exactly what they always have done, unfortunately. But faults will ultimately bite them back, and possibly will cost them more in the long term.

Edited April 27, 2019 by MBeatrix

[Imsaho Chau]

21 hours ago, Fluf Fredriksson said:

As of April 25th. Yes this is fixed for both sim crossings and teleports.

OK, agree, when I posted I hadn't looked at the Linden status report.

"We have finished updating main grid regions with a fix which should reduce incidence of teleport disconnects. ". Mind you, 'reduce' is not the same as 'eliminate' which is the solution I would prefer. Looks like the band-aid solution.

Hopefully they've now added logging to the process so that when it happens again they can identify and fix it faster

Edited April 28, 2019 by Imsaho Chau