Random Disconnects

[ringgoddess]

I have been experiencing random disconnects for a few months now. 3-5 a day on average. Sometimes several within a short time period. Sometimes hours apart. They occur in various regions and SIMs. And in both low and high lag environments. I have not been able to determine any kind of commonality or consistency to them.

I am using the latest Firestorm viewer (Firestorm-Releasex64 6.4.12.62831) but these disconnects began while on the previous version and have continued with this latest version. I've been in contact with Firestorm support and they explained to me the difference between a 'Crash' and a 'Disconnect'. I have experienced both, but disconnects seem to be the greater problem. Evidently disconnects are much more difficult to trouble shoot.

There are warning signs when a disconnect is about to occur: I can not post to 'Nearby Chat', HUDs will not respond, and can not move forwards or backwards only rotate view. Then the screen will freeze, go black and white and a window will appear (see attached screen shot). I was wondering if there were any known issues that might be causing this problem for me. Thank you for any assistance or support you can provide.

[Nova Convair]

I run pings of 180 to 250 for many years. That's a normal ping for an US west cost connection from europe. Zero problems, zero crashes - ok maybe 1-2 crashes per year.
Paket loss is 0 all the time though - I think thats more important than the ping.
AWS didn't change anything. Seems the ping is pretty constant at 230 now.

[Kyrie Deka]

Sharing that I beat my head against a brick wall on this one. It was awful, often every login every day and just drove me nuts.  But it could be so many things. So I let it go to spend energy on other things...however,

Apparently all the Windows 10 updates that I've gotten over a very long time have not bothered to update my WIFI driver.  Tripped across a decades old driver associated with my Netgear network adapter while routine skimming through Device Manager.  Found much much newer driver online - shockingly newer.  Didn't give much hope to it fixing anything, I mean, lots of very old simple drivers are still the best to use.  I installed it and rebooted still not hoping for much difference. After months of pain, totally fixed my FS viewer cutting out!  And not one problem since.

I tried just to let it go rather than mull:  Why wouldn't all those fancy Windows 10 updates be checking for that update too?  I mean, the list of other common brand name drivers Windows 10 does update is lengthy, but it doesn't also include a common Netgear?  

I expected Windows 10 updates to be taking care of that and it turns out that over a very long period of time *at least for this pc* it hasn't been updating it.  Might want to check that just in case... FYI my scanning for a newer driver in Device Manager said I already had the latest.  Lotsa help that was.  Glad my Ten of Wands personality went the extra step to Mfrs website to check and then install.   

Edited February 4, 2021 by Kyrie Deka

[Lindal Kidd]

I have been getting the same thing for the last two versions of Firestorm.  Plus random crashes, where the entire viewer simply goes away without warning.  They seem to be graphics related.  If I am putting too much stress on my graphics card, they happen more often.

[ringgoddess]

I get random crashes too. They are usually followed by a 'BugSplat Crash Report' popup. I have been sending those in.

My computer is a Predator 17 with an NVIDIA GeForce GTX 1080 graphics card, an Intel Core i7-6700HQ CPU @ 2.60GHz processor, and 32GB DDR4 installed RAM. I am not really very Tech Savvy so I don't know how good or bad all that really is. I assume is more than adequate. It's about 4yrs and 3mnths old. I purchased it in early 2019.

Thank you for you assistance.

Edited December 28, 2020 by ringgoddess

[Nalates Urriah]

3 minutes ago, ringgoddess said:

I get random crashes too. They are usually followed by a 'BugSplat Crash Report' popup. I have been sending those in.

My computer is a Predator 17 with an NVIDIA GeForce GTX 1080 graphics card, an Intel Core i7-6700HQ CPU @ 2.60GHz processor, and 32GB DDR4 installed RAM. I am not really very Tech Savvy so I don't know how good or bad all that really is. I assume is more than adequate. It's about 4yrs and 3mnths old. I purchased it in early 2019.

It is more than adequate. The 2.6GHz is slow. But it is a bad idea to overclock laptops. 

I suggest you get the free app HWMonitor and check the temperatures inside the laptop. Anything over about 70°C indicates a problem. Laptops tend to gather dust in the cooling pathways and need to be cleaned.

You can try to clean it without disassembly. It probably will help. YouTube has disassembly videos for almost every model of laptop. They can be tricky to disassemble. Any phone repair shop can clean the laptop.

Another possible cause is your connection to SL. Run through the steps in Troubleshoot Your #SL Connection to eliminate that possible problem. Don't think a good Internet connection means you have a good connection to the SL system.

[ringgoddess]

Thank you for your assistance and advice.

There is a application called 'Predator Sense' that gives Temperatures. I just had another disconnect and it indicates CPU: 56 C, GPU: 54 C and System: 39 C. I have noticed those temps to be higher, mid to upper 60s at times for the CPU. I will look into having someone more qualified than I clean my laptops cooling pathways.

Trouble shooting the SL connection: I went to 'BandWidth Place' and selected 'US - San Jose, CA' in the Server selection drop down and received the following results: Download: 29.87 Mbps; Upload: 3.19 Mbps; Ping: 90 ms. I know that is not necessarily very good, but is it bad enough to be the issue?

 

[LittleMe Jewell]

11 minutes ago, ringgoddess said:

Trouble shooting the SL connection: I went to 'BandWidth Place' and selected 'US - San Jose, CA' in the Server selection drop down and received the following results: Download: 29.87 Mbps; Upload: 3.19 Mbps; Ping: 90 ms. I know that is not necessarily very good, but is it bad enough to be the issue?

Those values don't indicate any problems. 

In the viewer, press Ctrl + Shift + 1 and keep that little window up off to the side or something.  Go about your normal SL activities, keeping an eye on the Ping Time and Packet Loss values.  You don't want super high ping times and you definitely don't want to be seeing packet loss -- though the occasional 0.1% or 0.2% isn't a problem.  

[ringgoddess]

Thank you for the tip. I will try to keep that open and an eye on it.

Packet loss steady at 0%; Ping Sim is averaging between100-125 ms with drops and spikes from 91 to 225 ms

Edited December 28, 2020 by ringgoddess

[LittleMe Jewell]

15 minutes ago, ringgoddess said:

Thank you for the tip. I will try to keep that open and an eye on it.

Packet loss steady at 0%; Ping Sim is averaging between100-125 ms with drops and spikes from 91 to 225 ms

The 225 can be problematic, but only if it goes there a lot.  Otherwise, the numbers don't indicate any issues.

We used to be able to do a tracert to the SL server, but I don't think we can do that with the cloud.  Thus I'm not sure how to track down exactly what the issue is.

I'd suggest keeping the Stats window open.  It should freeze the values at the point that you get that disconnect message, so maybe you'll see a really high ping or packet loss value at that point.

 

[ringgoddess]

Thank you for your time and assistance.

I will keep it open and see. Should it do so and the number is high, where and/or to whom should I report it?

[LittleMe Jewell]

6 minutes ago, ringgoddess said:

Thank you for your time and assistance.

I will keep it open and see. Should it do so and the number is high, where and/or to whom should I report it?

High numbers would show an issue with your connection.  If you are wireless, then definitely get hard-wired if you can.  You can also do a reset of the modem & router -- power off and wait a minuter before powering on.  If the problem persists, you can contact your ISP, but it is incredibly difficult to get them to work with you unless your internet performance is totally crappy all over.  It is also possible that it is a temporary things related to increased overall traffic during the holidays.

[Ardy Lay]

Some routers just can not deal with the number of flows Second Life creates.  Some are already dealing with a bunch of flows from other applications.  Some carriers discard UDP traffic when their network is experiencing congestion.

Do not fear rebooting the firewall / router thing that is between your computer and the Internet.

I work in the core network of a carrier and access provider.  Sometimes my home service attached PC will drop off SL.  When I check connectivity to the simulator host I was connected to when I dropped what I see makes me suspect I was the victim of some overzealous DDoS mitigation system.  Further attempts to connect to that host die in transit. If I login to a different region I am successful.  Immediate attempts to teleport to or login to the region I was in when dropped fail.  However, I can participate in IM chat with others in that region.  Half an hour later I may enter the region. 

[ringgoddess]

Thank you for your time and advice.

I am generally able to log back in immediately to the location I was at when the disconnect occurred. I have tried restarting my Modem and Router multiple times since these disconnects started. The router I had went kaput and I have replaced it with a new one (NetGear AC1600 WiFi Router, model R6260). I hoped that might have been the problem, a failing router, but problem persists.

[ringgoddess]

Had another disconnect. I noted that on the viewer 'Statistics' window I had open; Packet loss was still 0%, but Ping Sim spiked to 1000 ms.

[Rolig Loon]

20 hours ago, ringgoddess said:

Ping Sim spiked to 1000 ms.

That would do it.  If your Ping Sim values are normally between 100 and 150, that's a bit high for most places in the U.S. but has been normal for much of Europe.  SL residents in India, Australia, and Japan have typically had to deal with longer ping times, simply because of how far they are from Linden Lab. I don't know how "normal" may have changed now that we are in the AWS cloud, but I wouldn't be surprised if "normal" values were still about the same today.  (For comparison, I am still seeing 70-90 msec here in the midwest U.S., but I notice that my packet loss is now averaging around 0.7% instead of 0.1% as it had been for years. Go figure.)

Anyway, spiking to 1000 msec will be enough to get your viewer and the SL servers out of sync and make you disconnect.  If you're using a satellite connection or DSL and can switch to cable, do it.  Otherwise, I guess all you can do is argue with your ISP.  Years ago, I convinced mine to check out the box at the street when I was having connection issues. They repaired some corrosion damage and my problems went away.

[ringgoddess]

Thank you for your insight. I've been monitoring it closer and while packet lose has not been any higher than 0.4%, my Ping Sim value has jumped to over 2000 ms at times. I am on cable internet. I had my provider come and check lines at my house. They replaced a couple and put new connectors on some. Said all was testing properly at the time. But I am still experiencing problems randomly. I am in the SE US so wouldn't expect the same issues encountered by those over seas. It is interesting that SL switch to the 'Cloud" seems to coinside with these issues, but I'm not Tech savvy enough to know why that would be. I can only assume it is something more local to my area as others I know on here aren't having the frequency of disconnects I am.

 

[Rolig Loon]

Yeah, you certainly shouldn't have Ping Sim values anywhere near that high in the SE US, and definitely not with cable.  There's something odd about your connection .  You're going to have to do some digging to find out whether it's something close to home or something more distant.  Take a look at Nalates's blog to get an idea of what tools might help you figure out what's wrong.  That article is a bit old now but her advice is still good.

Edited December 29, 2020 by Rolig Loon
typos. as always.

[ringgoddess]

Thank you again. I will take a look at Nalate's article. I do appreciate your and everyone's time and assistance.

[Skell Dagger]

4 hours ago, Rolig Loon said:

If your Ping Sim values are normally between 100 and 150, that's a bit high for most places in the U.S. but has been normal for much of Europe.  SL residents in India, Australia, and Japan have typically had to deal with longer ping times, simply because of how far they are from Linden Lab. I don't know how "normal" may have changed now that we are in the AWS cloud, but I wouldn't be surprised if "normal" values were still about the same today.

UK here, on a 10Mbps down ADSL connection. Average pings for me now we're on AWS are pretty much the same as before: between 150ms and 185ms. I've noticed my packet loss has increased more regularly since uplift, though.

[Kitteh Whiskers]

Same with this new update. It's random, 0 warning. Really annoying 😐

[Nalates Urriah]

The Linden viewer has changed how they report which server you are using. Unfortunately the naming they use does not provide the information needed to ping the server. However, Firestorm and other third party viewers do. Firestorm reports the server I am in just now as: ec2-18-236-208-124.us-west-2.compute.amazonaws.com. I suspect the red part will be the same for all SL regions. It is for the two regions I tested it on. Plus the full address can be pinged and the Amazon servers reply. You can't ping Amazon using just the red part. However, there are now more routers along the path that don't reply to TRACERT. So, TRACERT is not as useful as it once was... or maybe it is more accurate to say it is more complicated with IPv6 coming online.

PING can be high, 2000 to 5000ms and not break the connection. You won't want to use a connection that is that slow. If the high ping continues for more than 15 or 30 seconds the chances of being dropped WAY increase. If you are also dropping packets and have high ping-time then you are more likely to be dropped and that can happen just a few seconds.

The packet loss percent number is a running average. This tends to keep it at zero even when you are losing packets. Open the viewer's HELP->About... scroll down to Packets Lost and you'll see the number lost and received along with a percent. When being disconnected it is usually because your system is loosing packets. The only way to tell if that is the case is to look in the SL log file. But, even that is  not definitive as it doesn't tell you when the packets were lost... at start up? Very common. During the session? Also common to lose some throughout the session. If you don't know those numbers you can't know if there was a burst of packet loss just prior to the disconnect.

So, whether you 0.4% is good or bad can depend on how long you were connected. 

A few people are noticing more packet loss. It is hard to know if this is related to Amazon cloud or higher net use numbers from CoVid lockdowns. Because we know the Lab changed over we associate the increase with the change. But that isn't a safe assumption. But it is a hypotheses worth considering. And I consider the problem likely something on your side or your ISP's as if it were on the Amazon or Linden side we would have lots of people complaining.

PS: Ping as measured by the viewer is not real PING. It is a hybrid ping that includes packet travel time AND SL server response time. You can be in a region and see 1000 to 3000ms PING and not disconnect because most of the time is lagging server response time caused by a lagging server. In such cases you are more likely to disconnect yourself or teleport away because you can't put up with the lag than to have the server disconnect you. I used to see this a lot in combat regions. As Time Dilation and Server FPS worsen so does the viewer's ping time. But packet loss will kill the connection.

Edited January 3, 2021 by Nalates Urriah
grammer

[ringgoddess]

Thank you very much for your time and assistance. Very informative and I agree if it were on the SL side there would be a lot more having the issue. One of the reasons I asked this was to see if there were and maybe if just more from my area. Knowing that would help narrow the possibilities. At this time I am inclined to believe it is something more local to my location. Thank you again for your time and effort.

[Angelina Sinclair]

I've been having this same issue for the past few months and have yet to pin-point the matter. I use the Kokua viewer and have been in talks with the creator about it in a bug report. So somethings to note for the tech savy:

• -I have Fiber optic internet (Verizon FIOS)
• -I've ran ping to the sim I am on for hours while waiting for my usual disconnection to happen and when it did, the ping reported 0% packet loss.
• This happens on Firestorm and Kokua viewer
• This happens on a different computer on the same network
• This happens at separate occasions from the other computer IE: I may disconnect but the other one remains online and visa versa
• The connections to said router are all wired, not wireless (will try wireless soon when an adapter arrives in the mail)
• Used other pinging programs and none found any issue before and after my disconnection
• I've reported it to LL customer support, they say use their viewer then get back to them (obvious response is obvious)
• I've collected the crash logs and all show similar messages to this: 2021-01-28T20:15:40Z WARNING # llmessage/llcircuit.cpp(1053) LLCircuitData::checkCircuitTimeout : LLCircuitData::checkCircuitTimeout for 34.212.102.90:13000 last ping 104.090404s seconds ago.
• I've collected detailed crash logs with network messaging inside of them (Develop -> Network -> Enable message log). Again nothing out of the ordinary until sudden disconnect and everything spirals into a disconnection with similar messages as above.
• When experiencing a disconnection it is immediately apparent as people stop chatting, things stop moving and I can't move. I can usually tell majority of the time. Also when UDP packets received is suddenly 0, then I am disconnected. I realize it long before SL. Prior to that moment I could be locked in an engaging conversation, typed my response and get disconnected right after. I am connected, at fiber optic speeds, until suddenly I am not. Like someone clipped the line or pull the plug.
• I've port forwarded all ports I can for SL, to the best of my ability.
• I've disabled the firewall on my PC and router and still disconnections.
• Immediately after disconnecting or realizing I disconnect, I restart and can login instantly.
• Disconnections happen at any time while I am listening to music or watching a video aka data is still streaming to my PC from other sources.
• This does not happen with any other MMOs I play. It is only with SL.

 

Now with that said there are a few things I haven't tried yet:

• Contacting my ISP (cause lord knows I'm gonna get looped back to SL support team, then 3rd party viewer, and back around)
• Connect to SL via another internet provider like a neighbor's wifi or hotspot (waiting for adapter in mail)
• Perform a summoning ritual and beg the old gods to fix my internet.

 

That's it. That's basically all I am left to do to try and figure out where the source of this issue is and I am 99.99% certain it isn't my hardware or network. This PC of my mine is brand new and this issue happened on my old PC as well.

I think what is the most frustrating thing about this matter is that when I contact support or others about this; no matter how much detail or knowledge I provide on the issue and the number of things tested. They want me to test something again or go try a new avenue, basically pointing to a network hardware issue when I am pretty damn sure it is a viewer or SL server issue at this point.

Still if you got any other ideas of what I can try let me know so that when I do contact support again I can dump a laundry list of things I tried and give them stink eye to finally realize the issue is not me but something else.

 

[Angelina Sinclair]

And to the curious about my PC specs:

Kokua Release FTRLV 6.4.11.49726 (64bit)
Release Notes

CPU: Intel(R) Core(TM) i7-10700K CPU @ 3.80GHz (3792.01 MHz)
Memory: 32691 MB
OS Version: Microsoft Windows 10 64-bit (Build 19041.746)
Graphics Card Vendor: NVIDIA Corporation
Graphics Card: GeForce RTX 2070/PCIe/SSE2

Windows Graphics Driver Version: 27.21.14.5671
OpenGL Version: 4.6.0 NVIDIA 456.71

Window size: 1920x1017
Font Size Adjustment: 96pt
UI Scaling: 1
Draw distance: 64m
Bandwidth: 1300kbit/s
LOD factor: 4
Render quality: 5
Advanced Lighting Model: Enabled
Texture memory: 2048MB
VFS (cache) creation time: December 03 2020 16:41:20

RestrainedLove API: RestrainedLove viewer v2.09.28.03 (6.4.11)
J2C Decoder Version: KDU v8.0.5
Audio Driver Version: FMOD Studio 2.01.01
CEF-Version: Dullahan: 1.7.0.202008031800
  CEF: 81.3.10+gb223419+chromium-81.0.4044.138
  Chromium: 81.0.4044.138
LibVLC Version: 2.2.8
Voice Server Version: Not Connected
Built with MSVC version 1916
February 01 2021 13:24:10

[Chic Aeon]

On 12/28/2020 at 7:33 AM, Lindal Kidd said:

I have been getting the same thing for the last two versions of Firestorm.  Plus random crashes, where the entire viewer simply goes away without warning.  They seem to be graphics related.  If I am putting too much stress on my graphics card, they happen more often.

I have been also although I didn't update to the EEP viewer. "I" thought it was FS related but now I am thinking not. Mine was like yours, no warning, black screen and most of the time I need to reboot. It sort of stopped for awhile and then came back with a vengeance  when we were finally all moved to the cloud (but apparently with not much testing  as they were in a hurry to get things done == likely before the end of the year and that seems reasonable).   

 

While definitely graphics card related I finally figured out it really wasn't my card (I mean over six months and it would have quit by now I am pretty sure) and again it was fine for awhile.  So NOW I am suspecting that it is connected with the uplift and hopefully it will get remedied over time. I only crashed once this last weekend and I looked at a lot of sales.  NORMALLY (read that as before Uplift began) I only crashed about twice a year.  :D.  But once a weekend isn't a big deal.   In my case I am really not asking my graphics card to do all that much. It can just go to black screen on a whim when nothing at all is happening.  It never happens on MY sim however so I suspect it does have something to do with heavy mesh and or heavy textures.  Alas, that isn't getting any better.