Jump to content

CDN broken again?


You are about to reply to a thread that has been inactive for 2516 days.

Please take a moment to consider if this thread is worth bumping.

Recommended Posts

1 hour ago, Callum Meriman said:

Not long after the first part of pipelining went in there was a lot of trouble with textures and mesh not loading for *some* users only.

This has started happening again, and has been affecting multiple people I know for around 2 weeks now.

Yep, it has been happening to me too, but it may be more than the CDN, as inventory has been awfully slow.

Note: it is much worse when the US are awaken.

Edited by MBeatrix
note added
Link to comment
Share on other sites

Yes, this is now a new issue within SL.  I did wonder if it was my router or my ISP until it began to be clear that many folk from several geographic regions were seeing the same issue.  Bearing in mind LL's averred intent to move inventory fetching to http, this gives me cause for concern since inventory loss is already a problem for a fair number of folk.  The CDN has worked fairly well overall, but poor texture delivery has been a frequent issue since it was set up, so one has to wonder if LL's collaborators are "up to stuff".  I hope that this simply is a side-effect of the server-changes needed to implement http inventory.

Link to comment
Share on other sites

OK I'm being stupid (not for the first time) but something is being moved to http from UDP in the near future (I am vague on the actual timing).  If it is not inventory is it perhaps asset rendering or something?  I recall being told by someone at The Lab or one of the FS devs that it was something that required both server-side and client-side changes and that the client-side changes would be "pretty major", with a timescale running from Spring to Summer this year.

Is this something of which you know O Whirly One?

 

Edited by Ayesha Askham
spelling
Link to comment
Share on other sites

9 minutes ago, LittleMe Jewell said:

Yes, it is related to assets, but that is as much as I know -- from the alternative viewer info:  http://wiki.secondlife.com/wiki/Release_Notes/Second_Life_Release/5.0.5.325940

From the release notes: "This viewer moves fetching of several types of assets to HTTP. This means that landmarks, wearables, sounds and animations will now be fetched the same way as textures, via a direct query to CDN instead of through the simulator. This should generally make loading of such content both faster and more reliable."

Shame that what it's written there isn't happening... I tried that very same SL Viewer version the other day because I wanted to check if it was a Firestorm issue, and nope, it was even worse... Inventory loading was slooooow. Changing outfits was simply painful.

  • Like 1
Link to comment
Share on other sites

Debugging issues with the CDN isn't very easy, but it's impossible if we don't get very careful and thorough reporting. For example, including where you are, how you are connected to the network (who is your ISP), exactly what sort of problem you had loading (images, inventory, mesh objects, whatever), what region you were on, what time it was (including the timezone or explicitly in SLT).

We have occasionally been able to diagnose and correct problems when we had enough information to go on....

  • Like 2
Link to comment
Share on other sites

26 minutes ago, MBeatrix said:

From the release notes: "This viewer moves fetching of several types of assets to HTTP. This means that landmarks, wearables, sounds and animations will now be fetched the same way as textures, via a direct query to CDN instead of through the simulator. This should generally make loading of such content both faster and more reliable."

Shame that what it's written there isn't happening... I tried that very same SL Viewer version the other day because I wanted to check if it was a Firestorm issue, and nope, it was even worse... Inventory loading was slooooow. Changing outfits was simply painful.

I was unlclear on my post.  I simply meant that Ayesha was correct in thinking that there was something in the works related to assets moving to http and possibly something going on server side in prep for it.

 

Link to comment
Share on other sites

4 minutes ago, Oz Linden said:

Debugging issues with the CDN isn't very easy, but it's impossible if we don't get very careful and thorough reporting. For example, including where you are, how you are connected to the network (who is your ISP), exactly what sort of problem you had loading (images, inventory, mesh objects, whatever), what region you were on, what time it was (including the timezone or explicitly in SLT).

We have occasionally been able to diagnose and correct problems when we had enough information to go on....

Hi, Oz, thanks for posting here.

OK, I am in Portugal, and tried the SL Viewer this Monday, from about 7 AM to around 9 AM GMT (it's SLT +8) and later on, around 4 PM my time (8 AM SLT) and the second time it was much worse — that's why I wrote that when the US are "awake" it all gets worse. The loading issues applied to everything — textures, meshes, etc. Even each time I opened the landmarks window there was a very noticeable delay.

I'm not going to provide here info about my ISP, but of course I will do that for Support if the problems become critical to the point it's impossible to do anything, so I'm going to give it a few days and see how it goes (using Firestorm.)

Link to comment
Share on other sites

2 minutes ago, LittleMe Jewell said:

I was unlclear on my post.  I simply meant that Ayesha was correct in thinking that there was something in the works related to assets moving to http and possibly something going on server side in prep for it.

Yes, I got that, no worries. I just took the opportunity to make clear that the improvement not only didn't improve anything for me as it even made it worse. :)

Link to comment
Share on other sites

If you are having an issue that is Second Life related, not viewer specific, the first thing you need to do is get off of Firestorm and into the latest release from Linden Lab. It's not enough to turn it on and "see" if the issue is the same. You need to run it all day, multiple days. In different places doing all the things you normally do. They need those details, they need those logs. If you use it for 5 minutes decide it's slow too and leave your giving almost no data. Also, everytime some tells you they are seeing the same thing tell then to get on the LL viewer as well. Give Oz and his team a fighting chance to solve it with some meaningful data.

Sorry to be all stand on the soap box but issues like this need to be looked at inside the official viewer if you want resolution, like the viewer or not.

Link to comment
Share on other sites

17 minutes ago, seanabrady said:

If you are having an issue that is Second Life related, not viewer specific, the first thing you need to do is get off of Firestorm and into the latest release from Linden Lab. It's not enough to turn it on and "see" if the issue is the same. You need to run it all day, multiple days. In different places doing all the things you normally do. They need those details, they need those logs. If you use it for 5 minutes decide it's slow too and leave your giving almost no data. Also, everytime some tells you they are seeing the same thing tell then to get on the LL viewer as well. Give Oz and his team a fighting chance to solve it with some meaningful data.

Sorry to be all stand on the soap box but issues like this need to be looked at inside the official viewer if you want resolution, like the viewer or not.

Yes, you are quite right in all you just wrote. But there is a different perspective: some of us (you included, possibly) pay to have a service working, and not exactly to spend many hours helping debugging it. Also, the changes were not a decision we took... We weren't even asked if we wanted them.

Really, reversing roles isn't going to help drawing people to Second Life. Possibly, not even helping some staying in. I know that it's not politically correct to write what I just did, but I think that each one should assume their proper roles, and mine is the one of a customer. Doesn't sound right to me paying for a service and spending time debugging it (or helping to) instead of enjoying it.

Edited by MBeatrix
correcting typos
Link to comment
Share on other sites

I'm curious if anyone experiencing the slow rezzing of textures & mesh are using Webroot antivirus?
Webroot is known to cause severe slow loading of mesh & textures (even when the scene has been cached) plus it can often pin disk use at 100% when an SL viewer is running causing sluggish viewer performance & sluggish system performance.
There are other antivirus apps that can cause the same problem too.

Whitelisting the viewer cache folder in the antivirus generally fixes the problem.  However the standard version of Webroot doesn't allow whitelisting a folder like pretty much all other antivirus do, only the business edition of Webroot allows folder whitelisting.

Webroot & some other antivirus & firewall programs are also incompatible with Http Pipelining.
This can cause pretty severe texture & mesh corruption when pipelining is enabled (rainbow coloured textures or textures remaining grey or the wrong texture appearing on an object & exploding meshes).

Relevant JIRA issues:
BUG-11314 - Graphical Errors - Webroot + Pipelining incompatibility
BUG-8631 - Complete texture corruption followed by viewer crash, only when HttpPipelining is enabled
FIRE-17661 - HttpPipelining - Webroot Crash with fault error ntdll.dll mesh fails to render, texture corruption

Link to comment
Share on other sites

I do use Webroot, but the only time I ever have any rezzing issues are when I first TP into a super busy place and every now and then when I go somewhere with a lot of super high resolution textures.  Other times, everything truly rezzes almost instantly.  If Webroot is part of the issue, there must be some other stuff at play that I don't have going on.

Link to comment
Share on other sites

7 hours ago, Whirly Fizzle said:

I'm curious if anyone experiencing the slow rezzing of textures & mesh are using Webroot antivirus?
Webroot is known to cause severe slow loading of mesh & textures (even when the scene has been cached) plus it can often pin disk use at 100% when an SL viewer is running causing sluggish viewer performance & sluggish system performance.
There are other antivirus apps that can cause the same problem too.

 

I don't use any antivirus or firewall software other than those provided by Windows 10 itself (Windows Defender.) Why? Because I learned more than a decade ago that when we are careful enough antivirus only can cause a lot of trouble... For what? Detecting a virus once in 5 years or so?

My use of the internet is a quite conservative one, and that applies to all other activities my computers are for.

Link to comment
Share on other sites

11 hours ago, MBeatrix said:

Yes, you are quite right in all you just wrote. But there is a different perspective: some of us (you included, possibly) pay to have a service working

Presumbaly you are paying Linden Lab. The service you are paying for is what Linden Lab provides, and that includes the LL viewer. LL is not responsible for any other viewer, and you are not paying them for use of any other viewer. I'm sorry that you are having the problem, but you can't expect LL to debug the services that you are not paying them for.

The problem may not be the viewer, but it may be, so you do need to use what you pay for, so that they can chase the problem, or simply continue with the problem until things are working better.

Edited by Phil Deakins
Link to comment
Share on other sites

1 hour ago, Phil Deakins said:

Presumbaly you are paying Linden Lab. The service you are paying for is what Linden Lab provides, and that includes the LL viewer. LL is not responsible for any other viewer, and you are not paying them for use of any other viewer. I'm sorry that you are having the problem, but you can't expect LL to debug the services that you are not paying them for.

The problem may not be the viewer, but it may be, so you do need to use what you pay for, so that they can chase the problem, or simply continue with the problem until things are working better.

You too are right, but Firestorm works well (excluding the moments when meshes and textures take a couple minutes to rez) unlike the official viewer when I tried it. And unless everything becomes bad enough, I am not going to intensively use any viewer that offers me no control over the frame rate. The 120+ FPS I get using the official viewer is only good to reduce the lifetime of my computer system (I'm sure that is good for the hardware industry, but not at all for my bank account.)

Yes, going on with it as it is seems to be the best approach, at least for me.

Link to comment
Share on other sites

OK, I won't stand by and listen to all this without commenting. 

Firstly it is NOT necessary to run the default Linden-provided viewer for days to establish whether or not a particular issue is client or server based.

Secondly, while Linden Lab may not be directly responsible for the experiences of users accessing SL via third-party viewers they ARE responsible for the system that feeds into those clients and a large proportion of the rendering code in MOST of them.

US corporate responsibility has been cited many times over the last decade as a "get out" route for service providers; many US ISPs are still doing precisely that today.

I identified this particular issue some days ago using several clients, all of which exhibited exactly the same symptoms.  Having eliminated my router and ISP in general from the equation and also my computer, I was left in no doubt that data was not being supplied to the clients in a timely manner, though in ALL cases (bar the "official" client) increasing the fetch request rate produced a moderate improvement.  It is simply that as a result of some throttling in the journey from the SL server to the client, data, be it textures, assets or several other classes of data, in not arriving at the clients in a timely manner.  Maybe "simply" is a bad choice of words; issues in SL are rarely simple.

Now whichever way you cut this, the responsibility for this poor performance is Linden Lab's, either directly or as a result of a failing at one of their network of collaborating operators.

All that having been said I am not holding my breath for LL to provide a transparent solution.  Despite what Oz said they already have more than enough information to be going on with, had they the motivation to address this issue.

Edited by Ayesha Askham
qualification of two statements
  • Like 1
Link to comment
Share on other sites

19 minutes ago, Ayesha Askham said:

Despite what Oz said they already have more than enough information to be going on with, had they the motivation to address this issue.

No they don't.  No-one has filed a JIRA issue yet with the details Oz asked for and viewer logs.
Someone who can reproduce this problem needs to file a JIRA issue to get the ball rolling.

Seeing as you can reproduce the problem on several viewers @Ayesha Askham, can you do that?
https://jira.secondlife.com/secure/CreateIssue!default.jspa

 

 

 

  • Like 2
Link to comment
Share on other sites

@Whirly

Will be doing just as soon as I fix a little issue with zipping up logs.  The main problem is that some of the log files apparently don't have the specific info that Oz wants, but I think I know how to get it.  I am astonished that no-one else has yet filed a JIRA submission on this.

Edited by Ayesha Askham
additional information
Link to comment
Share on other sites

OK I am an idiot

I had checked so many things to ensure that my system was not at the root of what I was seeing.  But not everything.  At some point in the recent past an update to my Windows Firewall (which I tied in with my Bullguard Security routine) had been updated and the "allowed programmes" set to default.

I now suspect that it was THAT that was causing the issues I was seeing.  I shall monitor the situation and if I am wrong I will be sending my logs to a JIRA for LL to peruse.

But for the meantime I am sitting in the naughty corner and sorting my system out.

Edited by Ayesha Askham
minor correction
  • Like 1
Link to comment
Share on other sites

44 minutes ago, Ayesha Askham said:

OK I am an idiot

I had checked so many things to ensure that my system was not at the root of what I was seeing.  But not everything.  At some point in the recent past an update to my Windows Firewall (which I tied in with my Bullguard Security routine) had been updated and the "allowed programmes" set to default.

Definitely not an idiot.  Windows likes to challenge us by randomly changing things when updates are applied.

Link to comment
Share on other sites

Thankyou for those kind words.  I wish that I could say that all is now well, but though better, there is definitely still an issue.  I logged in and performed a series of teleports using first FS 64, then the current "default" viewer, and finally the current RLV (I have tested other viewers but none actually use the current render code so they are not relevant).

I have still seen delayed texture delivery and also certain assets are slow to arrive, though none that are cached locally now, thank goodness.

The report is @ https://jira.secondlife.com/browse/BUG-100683

Edited by Ayesha Askham
syntax and spelling
  • Like 1
Link to comment
Share on other sites

You are about to reply to a thread that has been inactive for 2516 days.

Please take a moment to consider if this thread is worth bumping.

Please sign in to comment

You will be able to leave a comment after signing in



Sign In Now
 Share

×
×
  • Create New...