Black Dragon not rezzing everything

[Zeta Vandyke]

Since maybe two or three weeks, I have troubles with items just not showing up.

I log in BD, and 80% of the stuff comes up, but other items, completely random do not show up. The items seem to be there, as I can bump into them, or I walk on it. Its just not showing. If someone else moves it a bit, it pops up right away. Or if I just log of and back on again, most of the items will show up. So seems to be only the textures not loading at all (not grey but invisible)

A friend has the same issues, only re logging does not always fix hers. And she also experiences it since around the same time as I am.

So I am curious if anyone else is experiencing this, or have an idea on how to fix it? Did something change, or did we maybe touch a setting we should not have. Tried some things as deleting all the local profiles and reinstalling, but it persists.

 

EDIT: right now having the issue and re logging doe snot fix it either, so that seems to be random too. TPing away and back in did "solve" it

Edited December 26, 2021 by Zeta Vandyke

[Profaitchikenz Haiku]

This isn't viewer-specific, all of them show it. I posted my set of workarounds in another thread recently, but you've already found the most reliable one, TP out and back again. I'll go find the link to the thread...

https://community.secondlife.com/forums/topic/480867-stuff-vanishing-but-stilll-sort-of-there/

Edited December 26, 2021 by Profaitchikenz Haiku

[NiranV Dean]

Very well known issue currently. Been getting daily reports of this.

It seems to be triggered by relogging from one Viewer to another (whichever comes last seems to permanently corrupt the cache for some reason). I can only assume this is due to the Viewer requesting a new list of objects when switching Viewers as opposed to just updating what is already cached, for some reason this new list is corrupted (or gets corrupted somewhere between the server and until everything is writting into the cache). Hence why it doesn't happen to the first Viewer as this one was most likely the one to TP in whereas the other one is just requesting what is there. Couple this with a quick relog (from one Viewer to another) this might be causing faulty messages or a faulty login, SL in general has a lot of issues with doing quick relogs (doing too many too fast can result in your login session completely breaking, to the point you get disconnected shortly after again).

[Profaitchikenz Haiku]

I have a slightly different take on this: I use separate caches for different viewers so I think that rules out cache corruption due to viewer change.

What I have observed is

This happens in all the viewers I use, to varying extents, Singularity and Alchemy Beta are perhaps a year old and show it so that rules out recent viewer changes, other viewers are up to date and still show it.

This happens in the Opensimulator grids too, so that rules out SecondLife server code changes, LL's content delivery system, EEP, etc

It began to increase in frequency in the couple of months leading up to Christmas, so I an tending towards it being a general internet traffic loading issue, some packets are just not getting to their destination in time.

As an example to justify this, consider the other day when I logged into SecondLife and TP-ed somewhere I spend  lot of time at, so all of the required textures must be in my cache.

There are two clumps of four trees straddling a railway line. The clumps are sculpted trees, each object having one prim for the foliage, one prim for the trunks and branches.

Both clumps were well inside draw distance.

Both clumps showed the foliage but only one clump showed the trunks.

Wireframe showed me the invisible trunks were there. So the same bark texture, in cache, was showing on one object and not showing on another object alongside it?

The likely scenario I envisage is that a packet of information from the server saying "object blah at position blah-blah has texture blah-blah-blah on it" got through for the first clump of trees, but not for the second.

Against that hypothesis is this observation: editing the clump of trees and selecting the face for the trunks showed the preview of the texture and all the correct offset and scales, but didn't toggle it back into view.

So it knew what should have been there, it just wouldn't show it.

Edited December 27, 2021 by Profaitchikenz Haiku

[NiranV Dean]

37 minutes ago, Profaitchikenz Haiku said:

I have a slightly different take on this: I use separate caches for different viewers so I think that rules out cache corruption due to viewer change.

What I have observed is

This happens in all the viewers I use, to varying extents, Singularity and Alchemy Beta are perhaps a year old and show it so that rules out recent viewer changes, other viewers are up to date and still show it.

This happens in the Opensimulator grids too, so that rules out SecondLife server code changes, LL's content delivery system, EEP, etc

It began to increase in frequency in the couple of months leading up to Christmas, so I an tending towards it being a general internet traffic loading issue, some packets are just not getting to their destination in time.

As an example to justify this, consider the other day when I logged into SecondLife and TP-ed somewhere I spend  lot of time at, so all of the required textures must be in my cache.

There are two clumps of four trees straddling a railway line. The clumps are sculpted trees, each object having one prim for the foliage, one prim for the trunks and branches.

Both clumps were well inside draw distance.

Both clumps showed the foliage but only one clump showed the trunks.

Wireframe showed me the invisible trunks were there. So the same bark texture, in cache, was showing on one object and not showing on another object alongside it?

The likely scenario I envisage is that a packet of information from the server saying "object blah at position blah-blah has texture blah-blah-blah on it" got through for the first clump of trees, but not for the second.

Against that hypothesis is this observation: editing the clump of trees and selecting the face for the trunks showed the preview of the texture and all the correct offset and scales, but didn't toggle it back into view.

So it knew what should have been there, it just wouldn't show it.

I wouldn't rule out cache corruptions just because you use different caches for different viewers, thats the default so thats a given. In fact this all the more screams cache corruption, the malformed interest list is being written into cache, writing possibly bogus or missing data which results in the same cached (vanishing objects) visuals over and over with 100% reproducibility.

But i assume the actual bogus data comes from the AWS service which was taken partly down for a bit a while ago, they might broke something really hard there.

[Profaitchikenz Haiku]

8 minutes ago, NiranV Dean said:

In fact this all the more screams cache corruption, the malformed interest list is being written into cache, writing possibly bogus or missing data which results in the same cached (vanishing objects) visuals over and over with 100% reproducibility.

The idea of malformed data being written to cache is one I had't thought of: so after it has happened, TP-ing away and back again causes more recent information t be returned from the server and so makes good the cache error?

ETA just realised you're talking about the object-cache not the texture cache here, so that might explain why the right-click and "refresh textures" trick fails.

As much as it would be fun to blame it all on AWS, one of the Opensimulator grids I use is entirely on UK-based servers, so unless there's a loot of tromboning on the backbone I'll stick to it being a general innernet issue rather than a specific one.

I must get your viewer up and running again, I should have installed it on my new (old but new to me) PC which does now have a bit more resources, but things got in the way.

 

Edited December 27, 2021 by Profaitchikenz Haiku

[NiranV Dean]

2 hours ago, Profaitchikenz Haiku said:

The idea of malformed data being written to cache is one I had't thought of: so after it has happened, TP-ing away and back again causes more recent information t be returned from the server and so makes good the cache error?

ETA just realised you're talking about the object-cache not the texture cache here, so that might explain why the right-click and "refresh textures" trick fails.

As much as it would be fun to blame it all on AWS, one of the Opensimulator grids I use is entirely on UK-based servers, so unless there's a loot of tromboning on the backbone I'll stick to it being a general innernet issue rather than a specific one.

I must get your viewer up and running again, I should have installed it on my new (old but new to me) PC which does now have a bit more resources, but things got in the way.

 

That would be a far spread general internet issue then. Why would it happen to basically everyone... that sounds like a failure of cataclysmic proportions 

[Profaitchikenz Haiku]

1 hour ago, NiranV Dean said:

That would be a far spread general internet issue then. Why would it happen to basically everyone... that sounds like a failure of cataclysmic proportions 

Not if it's random and aperiodic,  bit like the telephone noise that Mandelbrot investigated. Errors arre rare but cluster. So you would every now and then get a blip, something no textured. Somebody TP-ing in beside you would see what you couldn't, because they hadn't experienced the blip, their hops to SL and back wouldn't be the same as yours.

 

I tested Black Dragon earlier, no problems, but I also tested four other viewers and again no problems, Whatever it is that causes these issues, it's in hiding today.

 

[NiranV Dean]

12 minutes ago, Profaitchikenz Haiku said:

Not if it's random and aperiodic,  bit like the telephone noise that Mandelbrot investigated. Errors arre rare but cluster. So you would every now and then get a blip, something no textured. Somebody TP-ing in beside you would see what you couldn't, because they hadn't experienced the blip, their hops to SL and back wouldn't be the same as yours.

 

I tested Black Dragon earlier, no problems, but I also tested four other viewers and again no problems, Whatever it is that causes these issues, it's in hiding today.

 

I'm getting a constant amount of reports every day (including today) this issue certainly is not hiding. I do not know the exact reproduction steps (if there are any) i just know that after switching Viewers it happens. I did switch to Alchemy and the LL Viewer, but only on my alt account (because i was using my main to talk to an user who needed help). I have not yet tested switching with my main both in cached and uncached regions. But whatever it is i doubt that even reproducing would help fixing it since it's not a Viewer side issue.

[Profaitchikenz Haiku]

1 hour ago, NiranV Dean said:

But whatever it is i doubt that even reproducing would help fixing it since it's not a Viewer side issue.

That is certainly my conclusion, and as it's showing up just as frequently in other grids, that's pointing towards infrastructure.

[Coffee Pancake]

I don't think it's a cache problem, or specific to BD. 

I've had objects fail to display, relogging or teleporting away and back solved it.

[Henri Beauchamp]

3 hours ago, Coffee Pancake said:

I don't think it's a cache problem, or specific to BD. 

I've had objects fail to display, relogging or teleporting away and back solved it.

I saw it happening, once only so far, with the Cool VL Viewer. I was ridding a mesh vehicle and it vanished from view while being still ”being there” (still riding it and controlling it just fine)... No amount of hacks (objects visibility refresh feature, forcing 360° interest list in agent sim, mesh ”boosting” and such) could make it reappear... And of course, I could not relog or TP away and back without loosing that vehicle...

I am wondering if this issue (which is quite recent and did not exist only a few weeks ago) could be related with server-side changes that would have been implemented for improving the 360° snapshot feature support...

Edited December 31, 2021 by Henri Beauchamp

[Profaitchikenz Haiku]

20 minutes ago, Henri Beauchamp said:

I am wondering if this issue (which is quite recent and did not exist only a few weeks ago) could be related with server-side changes that would have been implemented for improving the 360° snapshot feature support...

That wouldn't explain the growing incidents of it in grids outside of SL.

My current hypothesis is that its delayed network traffic. I haven't seen any SL incidents now since before Christmas, and only a ref recently in a couple of other grids. However Niran favours cache corrupption. Presumably th TP away and return results in a refresh of somethhing, eithr cached data or fresh data from the region server.

When such an object is not showing, it is possible to right-click it, edit it, and then view the textures on the face(s) that you are able to get to with the limitations of wireframe view. Invariably these show the textures that should e showing, together with the expected scale, rotation and offset parameters. How this information is obtained by the viewer is outside my knowledge,but could be significant:

You (and Niran) would be in a perfect position to implement some sort of "Check this object" action. The way I see it, you would right click on the missing object (possibly after first going into Wireframe to actually find it) and then choose an option called something like "check". This would be rather lie a ix of the old inspect function, it would determine the object vertices and the textures on each face from what it had in cache and compare then to requested information from the server similar to what a refresh command does. There are a few pitfalls obviously, texture UUIDs cannot be listed for items with insufficient perms, so preview thumbnails wold have to be used instead.

If differences are detected then some sort of reload completely action is required.

[Henri Beauchamp]

6 minutes ago, Profaitchikenz Haiku said:

That wouldn't explain the growing incidents of it in grids outside of SL.

I never saw this issue happening in OpenSim grids. However, with the latter, the world (and especially meshes) may take (sometimes many) minutes to load where it takes only seconds for an equivalent scene in SL... So you might get mistaken in your diagnosis that it is the ”same issue” as what is seen in SL, when it is just a (much, much) longer delay (possibly worsened by bad network conditions) for rezzing meshes in OpenSim.

[NiranV Dean]

6 hours ago, Profaitchikenz Haiku said:

That wouldn't explain the growing incidents of it in grids outside of SL.

My current hypothesis is that its delayed network traffic. I haven't seen any SL incidents now since before Christmas, and only a ref recently in a couple of other grids. However Niran favours cache corrupption. Presumably th TP away and return results in a refresh of somethhing, eithr cached data or fresh data from the region server.

When such an object is not showing, it is possible to right-click it, edit it, and then view the textures on the face(s) that you are able to get to with the limitations of wireframe view. Invariably these show the textures that should e showing, together with the expected scale, rotation and offset parameters. How this information is obtained by the viewer is outside my knowledge,but could be significant:

You (and Niran) would be in a perfect position to implement some sort of "Check this object" action. The way I see it, you would right click on the missing object (possibly after first going into Wireframe to actually find it) and then choose an option called something like "check". This would be rather lie a ix of the old inspect function, it would determine the object vertices and the textures on each face from what it had in cache and compare then to requested information from the server similar to what a refresh command does. There are a few pitfalls obviously, texture UUIDs cannot be listed for items with insufficient perms, so preview thumbnails wold have to be used instead.

If differences are detected then some sort of reload completely action is required.

A check that goes through all objects on the SIM and checks whether they are there would A not help, B that's basically the cache (which we see not working in this case) and C would unnecessarily add more work ontop of what the Viewer already does, further impacting performance in one of the slowest parts of the Viewer. Objects that vanish in this manner cannot be right clicked, they are completely gone and do not exist to the Viewer, thus the Viewer cannot search for them and find them. If the Viewer is never told that there is an object there then said potential object is never being tracked. You can't request said specific object either because you'd first have to know it exists, the only solution is a full on interest list request which would give you everything on the region again, essentially what you do when teleporting away and back.