Jump to content

New kind of region crossing failure


Recommended Posts

Have not seen this before.

burnsneumogenfail.thumb.jpg.40c7c2d6d828fe1af8c651f700286438.jpg

Region crossing fail at Burns->Neumogen.

This is a failure I haven't seen before. I was driving around Robin Loop fast, and the second time I approached Neumogen from Burns, it wasn't visible. I drove into the crossing expecting Neumogen to appear. Instead, my motorcycle disappeared, and I was stuck like this.

What's unusual is that this was more or less permanent. Teleporting out didn't work; about two minutes of waiting and then back here. Logout would work cleanly. While stuck, could not IM. Attempts to group chat returned errors. Could not read profiles. But I could look around in mouselook.

I've had this happen twice now at this crossing. Seeing this once wouldn't be that much of a problem, but twice indicates something is wrong.

 

 

  • Thanks 1
Link to post
Share on other sites

Uh oh. This fix may have introduced a new bug:

2020-08-24T21:00:45.547626

Thursday, August 27, 2020

Release Notes

  • Internal fixes (to help Lindens make better tests)
  • Logging improvements
  • SL-13785 improves some instances where vehicles would stop responding to control inputs after a region crossing. We don’t believe we’ve fixed all instances of this issue yet, but this patch should offer some relief for the issue.

Resolved Issues

  • SL-13785 Vehicle LSL scripts do not see any passengers shortly after a region crossing
Link to post
Share on other sites

Seeing lots of strange stuff. Approaching a region fast, the region ahead draws, then disappears. Then I end up sitting on the edge of the region as above.

Slow region crossings used to be OK with my bikes. They'd stop and wait for the avatar to catch up. Now that's stopped working reliably. When the region crossing fails, it used to be possible to get off, go to the bike, and get back on. Now I have to TP home. And after failed crossings, I keep ending up at <0,0,1> in the region.

Link to post
Share on other sites
6 hours ago, animats said:

What's unusual is that this was more or less permanent. Teleporting out didn't work; about two minutes of waiting and then back here. Logout would work cleanly. While stuck, could not IM. Attempts to group chat returned errors. Could not read profiles. But I could look around in mouselook.

I started seeing something like this yesterday when TP-ing. TP failed, an attempt to try again got a message saying TP was currently disabled/not working, it wasn't possible to move around, attempts to IM failed, ultimately I crashed out. It happened about 8 or 9 times, on both Firestorm and Catznip, both normally very stable viewers for TP-ing.

Link to post
Share on other sites

Drove around same roads at Burns, Neumogen, no problems - and this in an ACS based vehicle, which has more impact.

Meet #animats and drove with him as passenger, no problems encountered. Smooth flawless region crossings, no passenger hanging or eject, vehicle passed crossings perfect.

I would say the above problems, are due to a region not appearing online or a region in trouble. Hence it is always good to drive a vehicle with mini maps open, and simply stop and return if a region appear offline for what ever reason.

 

 

  • Like 2
  • Thanks 1
Link to post
Share on other sites
7 hours ago, Rachel1206 said:

Drove around same roads at Burns, Neumogen, no problems - and this in an ACS based vehicle, which has more impact.

Meet #animats and drove with him as passenger, no problems encountered. Smooth flawless region crossings, no passenger hanging or eject, vehicle passed crossings perfect.

I would say the above problems, are due to a region not appearing online or a region in trouble. Hence it is always good to drive a vehicle with mini maps open, and simply stop and return if a region appear offline for what ever reason.

I'm thinking this depends on SL load. No fails after 10:30PM or so SLT last night. Two this morning.

crossing-fail-vine.thumb.jpg.0989cd75d835a36062cf2f1e1b309458.jpg

Fail at Vine. Nowhere near a region corner.

This is happening far too often.

Edited by animats
Link to post
Share on other sites
18 hours ago, Rachel1206 said:

I would say the above problems, are due to a region not appearing online or a region in trouble. Hence it is always good to drive a vehicle with mini maps open, and simply stop and return if a region appear offline for what ever reason.

yes I think so as well. Sometimes the region isn't there for our connection, when it is for other people. So we just have to wait for it

is not as frequent this as it used to be. And yes agree. I watch the minimap. When the minmap fully updates to show the next region then is usually ok to cross.. When the minimap still shows the next region as dark blue then it isn't yet fully there at least for our connection. And we we will pretty much lock up our viewer every time we cross into dark blue

  • Like 2
  • Thanks 1
Link to post
Share on other sites
1 hour ago, Aethelwine said:

Frustrating SL-13785 is hidden.

Yes. Once a problem is accepted, it goes into some kind of internal JIRA queue where users can see neither the contents nor the status.

Closing a user JIRA as a duplicate of an SL JIRA is kind of bad, though. The user JIRA should remain live, but dependent on the internal one. Users may have more info to add.

This was discussed at Server User Group today. The server dev team is totally focused on getting SL onto AWS by the end of the year. The two big server side problems right now seem to be region crossings and group chat, both of which have been modified in preparation for the move to AWS and both of which seem to have been badly impacted by recent mods.

Simon Linden has way too much to do. LL is trying to hire more people to work on the server side of SL.

  • Like 2
Link to post
Share on other sites

The weekend Grid Drive had a lot of problems. Even walking, no vehicle, every region crossing took at least six seconds.

It was a timed run, and I logged 88 grid crossings, So a minimum of 8m 48s, and the leader board being used displayed time from under 5 minutes to 12 minutes. So, obviously, this all varies a lot. I was seeing maybe one third of my total time down to the region crossings.

I connect from the UK, with an internet ping of about 150ms. The draw distance may also have some effect on total load on my end of the connection. I was using a vehicle I was pretty familiar with.

I can't avoid a feeling that the Lindens have never been able to cope well with the idea that customer connections can vary so much. The cry of "It's your connection!" almost seems like their Get-Out-Of-Jail-Free card. And both AWS and the Content Delivery Network risk making sensitive assumptions about the communications.

 

Link to post
Share on other sites

@arabellajones: You highlight an issue that pervades the whole operation of SL.  They, like so many other Americans in the US, seem incapable of understanding that there is a WHOLE world out here.  It is akin to the idiotic attitude of some Brits who consider that "****(perjorative word redacted) begin at Dover".

If your ping to SL is greater than 125ms, you must have a problem (or live overseas) and therefore do not fit our functional model of the World. 

Link to post
Share on other sites

Once on AWS, Linden Lab can put sims in various Amazon data centers around the world, if they want.  Estate sims should be allowed to pick their AWS region, near where most of their users are.

All of SL doesn't have to be in the same data center. Right now, the beta grid is partly in AWS-West and partly in the old Arizona data center. Sometimes for adjacent sims you can cross between. That's working reasonably well.

Link to post
Share on other sites

Here is some raw ping times from Scandinavia:

Second Life server
Pinging sim10786.agni.lindenlab.com [216.82.56.76] with 32 bytes of data:
Reply from 216.82.56.76: bytes=32 time=148ms TTL=50
Reply from 216.82.56.76: bytes=32 time=148ms TTL=50
Reply from 216.82.56.76: bytes=32 time=149ms TTL=50
Reply from 216.82.56.76: bytes=32 time=148ms TTL=50

Ping statistics for 216.82.56.76:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 148ms, Maximum = 149ms, Average = 148ms


AWS
Pinging aws.com [13.33.152.114] with 32 bytes of data:
Reply from 13.33.152.114: bytes=32 time=12ms TTL=241
Reply from 13.33.152.114: bytes=32 time=13ms TTL=241
Reply from 13.33.152.114: bytes=32 time=13ms TTL=241
Reply from 13.33.152.114: bytes=32 time=11ms TTL=241

Ping statistics for 13.33.152.114:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 11ms, Maximum = 13ms, Average = 12ms

Google.com
Pinging google.com [172.217.21.238] with 32 bytes of data:
Reply from 172.217.21.238: bytes=32 time=19ms TTL=115
Reply from 172.217.21.238: bytes=32 time=19ms TTL=115
Reply from 172.217.21.238: bytes=32 time=22ms TTL=115
Reply from 172.217.21.238: bytes=32 time=20ms TTL=115

Ping statistics for 172.217.21.238:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 19ms, Maximum = 22ms, Average = 20ms

-

Clearly AWS and Google are routed to geographic nearby data servers or hub mirroring. Ping time to my local IP is 5-6 ms.

 

Edited by Rachel1206
Link to post
Share on other sites
On 9/15/2020 at 7:42 AM, arabellajones said:

The weekend Grid Drive had a lot of problems. Even walking, no vehicle, every region crossing took at least six seconds.

 

I did a similar test today, when the number of logins was around the minimum. It's a poor measure, but it's one of the few bits of hard data we can get.

The region crossing delays were less than one second.

All the measures of my connection were similar.

Link to post
Share on other sites
On 9/15/2020 at 7:42 AM, arabellajones said:

And both AWS and the Content Delivery Network risk making sensitive assumptions about the communications.

What I am thinking of here is the relative location of the asset servers, the sim servers, the CDN, and the user. Past models have has the asset servers close to the sim servers, and the CDN is caching certain types of content being sent to the user. It hasn't much mattered whether the asset servers and the sim servers have been the source, they're in essentially the same place.

But let's assume a university wants to run a set of sims. They could run OpenSim and have the whole operation under their control. But they choose to take advantage of the Linden Lab support structure, and the huge amount of conter other users have created.. Let's call AWS-target the optimum AWS location for the sim servers. There might be a CDN node at the university itself. What happens with the asset servers, and all the others. Where does texture baking get done?

And that's before we even think about different legal systems. Look at all the fuss we have with US financial laws. It's simple enough when the servers are all in Arizona, but when the servers could be anywhere, what then?

Link to post
Share on other sites

1) Are the prim object and mesh assets passed between simulators?

If yes, is what is passed the entire asset or just the digested representation used by the simulator?

If no, is what is passed like an inventory item?  (a pointer to the asset needed, and the simulator must request, receive and digest the asset to get the physics information it needs)

2). Are scripts in task inventory saved as an asset to the asset store any time during a typical region crossing or are they passed from simulator to simulator directly?

We know that texture, animation, sound, script source and some additional assets are not used by simulators so assume those have no effect on region crossings.

Link to post
Share on other sites

More problems with delays before the next sim appears. This time I have video and Firestorm logs, and will post once I edit the video down. It's strange. The sequence of event is:

  • Sim ahead is drawn, but only with the ground texture. No objects.
  • A few seconds later, sim ahead disappears, leaving empty space.
  • 5-30 seconds later, sim ahead reappears, this time with all objects. Then it's safe to enter the sim.

Light load, I was the only avatar around most of the time. Never saw this before the next week or two.

Link to post
Share on other sites

More info. Here's what this bug looks like, from recorded video.

simdisappear1.thumb.jpg.fe5e70f2b50f4f2b7c0170f4cd602699.jpg

Approaching a sim crossing, next sim is visible, but only the terrain is drawn.

simdisappear2.thumb.jpg.36b975bfe66a798c626d6c4a92c1f52a.jpg

Then, next sim is gone. Driving over the edge will leave the avatar stuck and a logout will be necessary.

simdisappear3.thumb.jpg.4b5481bfe2b33b65b748634e7fdc3e48.jpg

But wait a few seconds (sometimes as long as 20) and the next sim reappears and is fully drawn with its objects and terrain.

I've seen this six or seven times now. It's usually at the same places. Here I'm driving round Robin Loop. This is the Burns - Neumogen crossing, and I've seen it there at least three times now. Anyone else seeing this?

Link to post
Share on other sites
2 hours ago, animats said:

But wait a few seconds (sometimes as long as 20) and the next sim reappears and is fully drawn with its objects and terrain.

I've seen this six or seven times now. It's usually at the same places. Here I'm driving round Robin Loop. This is the Burns - Neumogen crossing, and I've seen it there at least three times now. Anyone else seeing this?

I just did a drive of the Robin Loop (2020-09-25 22:45 PM SLT) and did not encounter any of the problems above.

Last time we both were there, I had the problems as you just had - I would not call it a bug, but some latency internet problems.

BTW - proof I was there, got a ticket passing The Nightbell Fire Station😂:

RS Wasp v1.25,
Owned by  Rachel1206 Resident
53.072479 KpH
Fine L$23

Edited by Rachel1206
Link to post
Share on other sites

Go around the loop a few times without stopping. This seems to happen on repeat visits. I suspect it may be that the region didn't fully disconnect yet from the previous visit, and that may be why you see the region shown as bare terrain first.

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...