Missing inventory (repetitively) in latest Firestorm

[Rick Nightingale]

Yes, interesting opposing experiences.

The quick shutting down is certainly the key, for me. It reproduces the exact fault every time, now I've cottoned on to it. No wonder me sitting here watching the cache files being written, waiting for a glitch, was unproductive.

When I restart, there is a remnant, partial, uncompressed cache file left behind, truncated at whatever point its thread got killed I guess (or the write cache did). I'll have to investigate the write caching on that SSD out of curiosity.

The fix for me, at least, is now easy. Give it a minute before shutting down. I liked the new, quicker viewer close too, lol.

Edit to add re. the SSD cache:

Write caching is enabled, and Windows cache-flushing is also enabled. That should be OK; Windows should be telling the SSD to flush cache anyway when it shuts down. Write caching should only be able to cause data loss if the system crashes or loses power.

I do not think it is SSD cache related for the following reason:

While watching the inventory cache file being written after logoff, which I have been doing for weeks now, I have got a pretty good feel for how long it takes to create and gzip that cache file.

When Windows shuts down, it shuts the PC off more quickly than I think those inventory cache files take to write. So flushing a hardware cache isn't the issue; it's the Firestorm thread trying to create the cache in the first place which is being killed during Windows shutdown. I'm never really done much inside the inner workings of Windows, but surely there should be a way to tell Windows "this thread is important... don't go away until it's finished."

As final evidence, I have just run a test with SSD write-caching disabled, so writes go straight there (supposedly, anyway). The inventory loss still happened just as before, with the remnant, truncated cache file when I restarted. Looks like the thread is being killed.

Edited October 22, 2022 by Rick Daylight

[ConnieG Sorbet]

Happens to me at least 3 times a day 

[Rick Nightingale]

Has anyone else ever typed a bit of text into notepad, or your word-processor of choice, or indeed entered data into any sort of application which does not automatically save that data? And then tried to shut down your PC? You get a nice little notice that you need to save the data and your PC stops shutting down and sits there waiting. Even my mouse cursor being over on the screen of another PC will stop this PC shutting down (it's a tiny remote keyboard/mouse utility from MS Garage to control another PC that you can see the screen of - in my case a media player PC on the TV). Add to that the common occurrence of a brief, pop-up message when Windows is shutting down to the effect of, "Windows is being prevented from shutting down by [some thread that hasn't finished its housekeeping]". Those generally disappear as quickly as they appeared, as their threads finish.

Apparently Firestorm doesn't do anything like that even though it is saving critical data after pretending to close. Don't know about the official viewer because I don't use it; maybe it's the code in that and it does it too so it's an LL issue, not a FS one.

Seems as far as support goes... I just have to guess when it's safe to shut my PC down and if I shut it down before a program that looks finished, is really finished, of course I will lose data. It's my fault, or Microsoft's. Nothing to see here, move along.

Edited October 22, 2022 by Rick Daylight

[Jaylinbridges]

I always leave my desktop systems powered up 24/7.  Only thing that powers down is the monitor displays, with a screen saver after 20 mins, and sleep mode after 30 mins.  I never have missing inventory problems.  I guess with some laptops, when you close the screen it logs out windows and powers down. Never power down in Windows until all active program processing is completed.  Must be that everyone is in a hurry these days.  Slow down and you will live longer.

I had a partner that had a desktop laptop, and a horribly flat and large SL inventory.  For 5 years nobody explained to her how to make a sub folder, so she had none, just a 250K very flat Main inventory.  It would take 3 mins for her to log out of SL.  She was always shutting down before she got the explorer desktop back, thinking something had crashed.  The result was missing inventory for her every day she was impatient.  It took years to convince her to take a break, walk around, get a snack, and stop turning her windows laptop off just because the screen was black.  I gave her the recovery procedure - log in with an alt - find the settings folder for her main SL account, and delete it.  That removed the .gz inventory file that was damaged from her early shutdowns.  Then clear the cache, and relog with her main account. Her inventory would return in full, until the next time she grew impatient and shut down windows before FS had written the full .gz file.

SL and FS has made this problem worse now with the instant (fake) shutdown.  Apparently logging off and shutting down is some sort of modern fad.

 

 

Edited October 23, 2022 by Jaylinbridges

[Rick Nightingale]

The trick is... how on Earth is the average user supposed to know when an application that has apparently closed, has actually stopped all its processing? Do you wait five seconds? Ten? What if something else goes on (that you can't see) and slows things down more?

When I'm closing the viewer, I have almost always finished with that PC so naturally I am going to shut it down next. I've done so for many years and have never experienced loss in any application until this change to the viewer's shutdown. I'm not going to leave it on... that wastes energy and costs money even if shut down if it's still on at the wall switch (about 5W on most I've measured from just being connected to the mains). I'm still in the habit of quickly shutting down when what I have been using has visibly finished. Did it again last night, swore as I did it in realisation, and remembered to delete my chache this morning before starting FS. Note that I do this regardless of what I have been using... photoshop, blender, altium, chemoffice... anything. Nothing else has ever had a problem.

Operating systems have a mechanism (e.g. WM_QUERYENDSESSION) for pausing shutdown while critical tasks finish. OSs send 'tentative terminate' signals to running processes and there is a handshake; the OS doesn't just kill all running processes the instant you say shut down unless those processes allow it to. Otherwise... how could MS Word ask if you want to save that minimised document you forgot about? How could any task save its state for next time? I can only guess that the viewer isn't taking advantage of these mechanisms, and by 'pretending' to shut down more quickly than it really does (because it looks nicer - a trend started by MS themselves in Windows) at the same time... it's a recipe for disaster. If I had the time I would be tempted to look at the code myself, but I don't (and I'm hardly 'current' in my knowledge so it would take some getting back into).

It's not a case of a timing issue between the application, cache, SSD, Windows, whatever... it's a case of an application not managing its shutdown properly and the user being blamed for 'rushing' or shutting down when they shouldn't instead of fixing what is actually the problem.

 

Edited October 24, 2022 by Rick Daylight

[Love Zhaoying]

Why in the world is the application closing if there are still active threads?

That is not proper behavior for a well-behaved application!!

 

[Jaylinbridges]

8 hours ago, Love Zhaoying said:

Why in the world is the application closing if there are still active threads?

You mean Windows?  It used to take minutes for FS to shutdown with large flat inventories, as I said earlier.  If the user shuts down Windows before FS is finished, you corrupt the .gz file.  It would really help if someone familiar with the viewer (SL and FS) shutdown code would explain why the new performance viewers appear to shut down instantly, instead of minutes for some inventories, as they did in prior viewer versions.

 

Edited October 24, 2022 by Jaylinbridges

[Jaylinbridges]

8 hours ago, Rick Daylight said:

The trick is... how on Earth is the average user supposed to know when an application that has apparently closed, has actually stopped all its processing? Do you wait five seconds? Ten? What if something else goes on (that you can't see) and slows things down more?

Just like they did before the "instant shutdown". You did not see the WIndows desktop return when closing down the viewer until the application was finished. If you shut down your computer when FS was still closing, as the shutdown window told you, the corruption was on you.

The problem is the fake instant shutdown. Submit a JIRA to LL and to FS if you are affected.  They might just close your JIRA, telling you to simply wait a minute before shutting down Windows.

 

[Love Zhaoying]

16 minutes ago, Jaylinbridges said:

You mean Windows? 

No, I mean Firestorm. 

What I meant was: I assume from your earlier posts that Firestorm - while still actively saving data - closed the user window without leaving any clue "I'm still saving data".  That is bad.  The application should leave the window open until completely done.

 

[Jaylinbridges]

5 minutes ago, Love Zhaoying said:

The application should leave the window open until completely done.

It is in both the SL Viewer and FS code, which copied the SL code.  I am not affected since I do not shut down my computers seconds after I log out of SL. Those who are affected should file a JIRA.  Being retired, I really don't have such a tight time schedule anymore that I can not wait 30 secs before turning off my computer, "to save energy". 

[Love Zhaoying]

11 minutes ago, Jaylinbridges said:

It is in both the SL Viewer and FS code, which copied the SL code.  I am not affected since I do not shut down my computers seconds after I log out of SL. Those who are affected should file a JIRA.  Being retired, I really don't have such a tight time schedule anymore that I can not wait 30 secs before turning off my computer, "to save energy". 

I guess that I am not affected because my inventory is not very flat.

[Jaylinbridges]

2 minutes ago, Love Zhaoying said:

I guess that I am not affected because my inventory is not very flat

Neither is mine.  My shutdown time before the "performance" viewers was about 10 secs.  I have 85K of non flat inventory.  But if I insisted on shutting off my computer to save energy, 5 secs after I logged off, then I might be affected.  Good luck convincing the SL coders that this is an important issue.

One of the praises from the masses about the new performance viewer is that it "shuts down so quickly".  You think this might be a reason why it was included?  

[Rick Nightingale]

We'll have to agree to disagree on this.

My point is that the situation doesn't need to arise in the first place. Plenty of applications are capable of telling Windows "Wait a moment" when it sends the shutdown message/query. They don't lose data if I tell my PC to shut down after closing the program, or even if I forget to close them because they are minimised and shut down Windows first. They stop the shutdown process and tell me!

A user should not have to guess that an application is still processing in the background or for how long, after it looks like it has closed. That is simply poor programming, and a poor excuse for it. They should be able to tell the OS to shut down immediately, even if the application is still running! I've been involved in computers for 40 years... developed things like multitasking/user networking and database software for systems before Windows even existed, to bespoke whole-plant manufacturing control and management suites. I've done some development in Windows applications. I'm not trying to boast (plenty of people on this forum know more than me), just saying I do know a bit about software development even if I'm out of practice now.

I think it's important to design robust software that doesn't depend on a user knowing that shutting Windows down immediatly after an application apparently closes is bad. Because it should not be bad. It isn't for Photoshop, Blender, etc. etc.

Final points, since this is starting to be flogging a dead horse:

1. My <70k inventory is not flat, very far from it.

2. Just because 'the masses' like a feature, doesn't make it a good feature if it is basically a potentially data-losing 'cheat'. Even if only for some. Perhaps if it was done right...

Edited October 24, 2022 by Rick Daylight
Spelling - no spell chicker here

[Aishagain]

This is quite interesting. I must say that I believe that your practice of shutting off your PC moments after logging off makes you a bit of an "edge-case" in user activity, Rick, but the current 'BAM shutdown' is just too darn fast to feel healthy to anyone with a modicum of sense.  For a few days I was convinced that the new FS was crashing at log out, but since my settings and my inventory remained intact, I simply got used to it.  I never shut down my PC immediately after logging off SL simply because I do other things with it.

That LL did it because "they were asked to" by a small but seemingly strongly influential group of users smacks of LL doing its usual consumer research (ie none).  FS were, I assume, obliged to  use that new code, but in view of several other bugs, around right-clicking, texture editing and the friends list, I am inclined to think that whatever QA LL did on the viewer, it was not very exhaustive.

Edited October 24, 2022 by Aishagain

[Rick Nightingale]

5 minutes ago, Aishagain said:

...For a few days I was convinced that the new FS was crashing at log out...

Yes, so was I at first; it felt uncomfortable.

[Jaylinbridges]

12 hours ago, Rick Daylight said:

My point is that the situation doesn't need to arise in the first place.

I agree with you Rick.  But until they change the shutdown process so people won't go shutting down their computers too quickly, you just have to adapt.  LL and the SL coding has never been known for writing robust software.  And the old programmers who might have been able to keep the code up to date have retired or left LL. Timing and race conditions have caused problems for years, as hardware outpaces the software design in LL.  My point is they might think there are higher priority bugs to fix than this one.

Their early answer to shutdown was to prevent a user seeing the Window desktop until the viewer was finished updating the logout state.  Now they hide that, for what reason I have no idea.   Maybe it's a feature Marketing requested.

Logins were also a problem with large flat inventories, and the viewer often just timed out and you had to start over. 

 

Edited October 25, 2022 by Jaylinbridges

[LeggsLamore]

I had the same issue with having to clear cache and relog to regain my inventor after installing the lasterst FIrestorm update.  It would have every 2nd log in or so, got way way old.  Alos experiencing my group settings changing and the LMs i have saved at the top of my screen changing postions from the way they are set.  Also, my voice stopped working for inworld.  I can use voice in a call inworld, but not unless I was in a call.  I dealt with this for about a month until it just got way to irritating and time consuming to be resetting major items every day or two.  My solution ?  I went back to the previous upload. My inventory has loaded properly every time.  I suppose I will be staying with this 6.5.6.66221 version until I know the bugs are fixed.

[Tabitha Hawksby]

I've been having the same problem, and just since the update.  I have to clear inventory cache sometimes 2-3 times per week.  Never had this.  I even did a clean install.  Really think it's an update problem.  Hopefully they look into it since it's more than just one person having the problem. (Hint, Hint Firestorm) 

[suzyst]

Had the same problem with the last 2 updates,  gone back to the oldest working version and everything is fine.

[Kyrie Deka]

This plagues me now on Release Candidate (and the beta before it).   I began to suspect it was due to my relogging and not waiting long enough to reopen the viewer (= I receive the [FS already running...] notice).  As in something is interrupting cache finals at logout, so I log back in to only a partial inventory.  My viewer goes poof instantly at logoff, but takes another 2+ minutes to finish whatever it does after that.  And because I cannot tell when FS is done, I frequently try a bit too soon to reopen the viewer.  I never really shutdown Win11 fast after FS logout.  But I receive that [already running] reject notice often. Way above my paygrade, but I better figure out something pretty quick because if not then reloading a large inventory at every single login appears to be in my future now

And I just intentionally logged out of FS/didn't wait and immediately shutdown the pc, restarted, logged in, and yup orange cloud+ 1/6 my inventory only again.  

Edited November 14, 2022 by Kyrie Deka

[Ardy Lay]

34 minutes ago, Kyrie Deka said:

This plagues me now on Release Candidate (and the beta before it).   I began to suspect it was due to my relogging and not waiting long enough to reopen the viewer (= I receive the [FS already running...] notice).  As in something is interrupting cache finals at logout, so I log back in to only a partial inventory.  My viewer goes poof instantly at logoff, but takes another 2+ minutes to finish whatever it does after that.  And because I cannot tell when FS is done, I frequently try a bit too soon to reopen the viewer.  I never really shutdown Win11 fast after FS logout.  But I receive that [already running] reject notice often. Way above my paygrade, but I better figure out something pretty quick because if not then reloading a large inventory at every single login appears to be in my future now

And I just intentionally logged out of FS/didn't wait and immediately shutdown the pc, restarted, logged in, and yup orange cloud+ 1/6 my inventory only again.  

Just curious, why do you log in again soon after logging out?  I recently logged right back in because someone that was in local chat range sent an IM to me after I told them I was logging out for the night.  I didn't have a stopwatch on the process.  I also didn't have missing inventory or clouded avatars.  I use LL's viewer.  One oddity I did notice in my inventory was that the link to an Item I was wearing was in Trash.  

"2+ minutes" is what I would call "far too long."

[Jaylinbridges]

22 hours ago, Ardy Lay said:

Just curious, why do you log in again soon after logging out?  I recently logged right back in because someone that was in local chat range sent an IM to me after I told them I was logging out for the night.  I didn't have a stopwatch on the process.  I also didn't have missing inventory or clouded avatars.  I use LL's viewer.  One oddity I did notice in my inventory was that the link to an Item I was wearing was in Trash.  

"2+ minutes" is what I would call "far too long."

This incomplete inventory issue because the shutdown process fails is an age old problem.  Nothing is new here.  What is new is:

a) modern viewers have more to do when they shut down, especially FS and probably are not optimized for minimun shutdown times, and

b) large and flat inventories, which always take "far too long" to shut down, and finally write the .gz inventory to the local drive.  

I explained in a prior post here how an old partner would easily take 2 mins or more to shut down.  It was her huge and mostly flat inventory that was her problem.

The solution, other than spending 3 months adding subfolders and deleting 150K of duplicates, was to tell her to stop turning off her computer until the FS logout screen was gone.  The solution was to WAIT, and not be so damn impatient.  

But now, LL and FS using the rapid logoff code, has removed the only indication that you are not safely logged out.  The Windows desktop returns instantly upon logout.  Who's bright idea was this?   Let's remove the oil lights and gasoline gauge and speedometer from autos too, since any experienced auto engineer should just know what is happening to the engine by listening to it.  That seems to be the justification of the programmers in this fake shutdown code.  They know better, they have optimized inventories, and they have flawless internet connections, so why should they care, they never see missing inventory. 

The Firestorm help group has another answer to missing inventory.  Whitelist your anti virus, and if you already did, do it again, and if that didn't work then do it again.  Mention the shutdown issue to them and you get a blank stare and rude comments.  And so the issue goes on forever.

 

 

Edited November 15, 2022 by Jaylinbridges

[NiranV Dean]

I've gotten a couple reports that this is also happening for a couple people on Black Dragon, sometimes only on specific accounts, sometimes on all accounts.

[Kyrie Deka]

I should have also included that for many many months consistently, on the (current) FS Release, not once have I had any inventory not loading issues at login.  Not one time. I am generally inworld daily all year, at least one login and commonly more than once daily.  So my inventory being "flat" (if it was, and it is not) has not been the issue on the Release (and Releases before it), and nothing else here on this pc or my habits has been the issue for the FS Release (and prior Releases) for a very long time.  It seems before the Release Candidate my inventory etc. was situated to work very well at login.  

 

On 11/14/2022 at 1:33 PM, Ardy Lay said:

Just curious, why do you log in again soon after logging out? 

Oh and wanted to share on this too.  There are many good practice reasons or forced reasons for relogging.  I test using my alts.  I log out of one account and back in quickly to an alt to test etc.  On the Release (and prior Releases) this worked fine for a long long time.  Too, I change skins on the FS viewer, sometimes to test other times to see if the contrast works better for me:  It forces one to log out/back in to take effect.  There are several choices in FS settings that force one to log out/in after selecting.  There are updates to video cards and other software etc, that require or are best with a reboot after  I am commonly on 13-14 hour days, and updates arrive during long days.  There end up being a lot of reasons that I log out and back in quickly.  I have 11 years of long days of habits inworld, so once I see the viewer go poof out of the corner of my eye, I automatically click to re-open the viewer.  Habits can be redirected, but it would help retrain me if I could actually "see" when the viewer was done.  

 

And today I actually put this to test.  Since forcing myself not to shut the pc down, and not to jump to reopen the viewer, both for a full 5 minutes after viewer close, I have not had one inventory fail.  5 minutes pause is safest - if I jump any sooner and it is too soon, I have the inventory issue all over again.  5 minutes per login, shutdown, or leaving for a RL errand, will seem like a very long time.  But its certainly better than the 15 dedicated minutes per login spent reloading my inventory from scratch on a quiet sim unable to TP until done

[Rowan Amore]

I'm still confused as to why my experience is the exact opposite of everyone else.  In previous releases, I'd have inventory not loading issues weekly.  Since the latest viewer with the fast log out, not one single time.  Hmmm