Missing inventory (repetitively) in latest Firestorm

[Rick Nightingale]

This has happened now three times since I installed the latest FS 6.6.3.67470. I have never experienced this before. To fix it I have to delete my inventory cache and relog. If it was just once I wouldn't bother saying anything, but it's now the third time in a week. I have had successful logins between those three failures.

At login, I get a pop-up error message "Could not put on outfit. The outfit folder contains no clothing, body parts, or attachments." and my avatar stays as an orange cloud.

My inventory is not empty but is missing a lot (down from 70,000 ish to 37,000 ish - same number each time it happened (at least I recall the 37,xxx), and the same inventory seem missing in the same places every time this has happened.

All Outfits folders are empty, even though the clothing items themselves are where they should be and in bold as if I'm wearing them. If I look in the Worn tab in the Inventory window, everything I should be wearing is listed there.

All Objects content is missing, although subfolders (now empty) are still shown. Likewise the Favourites, Scripts, Settings and Sounds folders. Others are partially empty, for instance in Animations all the items in the 'root' Animations folder are there, but all subfolders (lots) are empty.

Thoughts, anyone?

(Edit: please forgive poor spelling; there's no spellchecker on this and I have dyspraxia (my brain can spell, my fingers are jokers).

Edited September 23, 2022 by Rick Daylight

[Aishagain]

What you describe sounds like inventory loss due to the local inventory gz file being corrupted but repeated issues might indicate an issue with your PC.  A scan might show issues.

I have experienced a similar issue, but only once.  The "worn" outfit was empty, I got that "this outfit contains no items" when I looked at it.

The items WERE worn if I looked in the "worn" tab of my inventory and I was rezzed to myself.  The viewer then locked up and I could not close it.

I then realised that my entire PC was locking up and the main drive light was flickering brightly as if the processor was fully occupied.

I could not shut it down via the windows menu so I did a "hard stop".

It restarted normally and after a scan (which showed no issues) I launched Firestom and all seemed normal and the previous empty outfit folder was populated as it should have been.

I ran for several sessions without any further issues and later I cleared the viewer cache after a couple of incidents were my house simply didn't rez (it looked weird with all the furniture and stuff floating in mid-air!).

So far the lock up has not repeated and I have successfully created several new outfits.

Whether all this was a viewer bug or just a PC glitch I don't know, but it sounds similar.

Edited September 23, 2022 by Aishagain

[Rick Nightingale]

That does sound somewhat similar. I have a dedicated SSD for SL... when I get back later I'll scan it, and maybe run a memory scan as well if I can find my memtest stick. Just in case.

[Rick Nightingale]

This is still happening; maybe 12 times now. It's just happened again. It also happens on other alts logged in on this PC.

I've moved my cache to a different SSD, so it's not that failing. Memtest86 didn't find any memory faults.

The cache file [avatar key].inv.llsd.gz is clearly truncated going off the file size. I've just logged in with LL viewer and copied the file over from that, although I doubt that is going to fix the issue since even completely wiping my cache folder hasn't helped. It's as if FS is truncating the cache file when it saves it at log off.

Something is going wrong here, and it seems restricted to just the inventory cache.

Edited October 2, 2022 by Rick Daylight

[Henri Beauchamp]

7 hours ago, Rick Daylight said:

The cache file [avatar key].inv.llsd.gz is clearly truncated going off the file size. I've just logged in with LL viewer and copied the file over from that, although I doubt that is going to fix the issue since even completely wiping my cache folder hasn't helped. It's as if FS is truncating the cache file when it saves it at log off.

Something is going wrong here, and it seems restricted to just the inventory cache.

If you cannot reproduce the same issue with other viewers, then it might indeed be a Firestorm bug: I'd suggest you verify, by using another viewer for a while (i.e. enough time that you can be confident the bug does not happen at all with it), then, once you are sure it is indeed Firestorm-specific, report to the bug on Firestorm's JIRA. I would recommend using LL's official viewer for that check, since in case you could reproduce the bug with the latter, you would then be well founded to report it on LL's JIRA instead...

Edited October 2, 2022 by Henri Beauchamp
typos

[Rick Nightingale]

Yeah, guess I'll have to use LL's viewer for a while.

One of the things I noticed after installing the latest FS was that it closed down more quickly, when I closed it. Probably a silly thought but I did wonder if it's sometimes closing all its threads before the cache has been fully written to disk. Edit: although if so, would the file even be finalised by the OS / NTFS, or just be dropped? Not done any deep Windows programming for a long, long time. Edit again: In retrospect, I could extract the inner file from the truncated, gzipped cache file without any complaints so I guess it's being written like that.

Edited October 2, 2022 by Rick Daylight

[Rick Nightingale]

After about 25 successful logins with the LL viewer, and no issue, I went back to Firestorm and it's happening every few logins.

Since no-one else is complaining about this, and it's pretty obvious if it happens, I must be a fringe case. Again. I'll head off to file a JIRA in a few minutes.

I've done everything I can think of here, down to changing out my cache SSD, multiple passes of Memtest86 to check for memory errors and a clean install of Firestorm (all folders wiped manually after uninstalling via control panel).

Edit: You have to laugh don't you... I'm trying to get it to do its trick so I can add before and after logs to the JIRA. Of course, it won't do it now I'm watching.

Edited October 10, 2022 by Rick Daylight

[Paulsian]

A few weeks ago with the latest version of the official sl viewer my inventory was not loading any of the official content (the green folders in inventory) and because they were not loading it was making the world water gray because the viewer could not locate the texture file for the water in the inventory? It took a few weeks to fix, and another user I talk with noticed it too. 

I noticed if you try to uninstall SL it does not fully remove everything SL related on computer, have to do search for SL files and maually delete in order to get a fresh install. Might need to do that with FS? Doing so will reset all your settings ( at least it does for the official viewer ).

[Rick Nightingale]

Thanks - yeah, I did that. Wiped the cache folder and the appdata folder. Uninstall wiped the program files folder; I checked. I restored settings from a backup. I doubt the issue is in there. It seems to be a failure when exiting, as if it fails to re-write the new inventory cache file. Sometimes.

I've even tried the latest FS beta (to try out the local mesh facility); that does it too.

I'm a bit tempted to wipe again and revert to the previous release version, but I'm enjoying the much improved rendering right now.

Edit: Oh nuts, I'm going to bite the bullet and revert to the previous. If that stops the issue, I know it's this relase of FS and not my PC.

Edited October 11, 2022 by Rick Daylight

[Natasha Bloodrose]

I am having the same issue.  Been a few weeks.  I am using the latest Firestorm update.  I have clean installed twice and it works for a few days then suddenly my inventory will not load, plus I lose all my landmarks.  I am about to go back to the previous version as this only started when I updated.  

 

[Rick Nightingale]

@Natasha BloodroseInteresting... please may I ask what operating system you are using? I have a theory...

[Gabriele Graves]

If you are on Windows, have you tried turning off your anti-virus or excluding the viewer data folders from being scanned?  False positives can sometimes quarantine files or stop them from being written to.

[Rick Nightingale]

Good idea, but I don't run an antivirus; not live scanning anyway. I have it disabled and have done so for many years*. Just do occasional out-of-band system scans (which I have also done for this issue, just in case). I do have a very good personal firewall though.

*The last time I got a malware infection (attempted, anyway) was many years ago. My AV (an expensive commercial one) didn't spot it even when I pointed it stright at the infected file. It was my firewall that alerted me and killed the attempt. Given how often I found an AV to be causing problems with something, I finally gave up on them and practice good system hygeine instead.

[adimay]

On 9/23/2022 at 7:01 AM, Rick Daylight said:

This has happened now three times since I installed the latest FS 6.6.3.67470. I have never experienced this before. To fix it I have to delete my inventory cache and relog. If it was just once I wouldn't bother saying anything, but it's now the third time in a week. I have had successful logins between those three failures.

At login, I get a pop-up error message "Could not put on outfit. The outfit folder contains no clothing, body parts, or attachments." and my avatar stays as an orange cloud.

My inventory is not empty but is missing a lot (down from 70,000 ish to 37,000 ish - same number each time it happened (at least I recall the 37,xxx), and the same inventory seem missing in the same places every time this has happened.

All Outfits folders are empty, even though the clothing items themselves are where they should be and in bold as if I'm wearing them. If I look in the Worn tab in the Inventory window, everything I should be wearing is listed there.

All Objects content is missing, although subfolders (now empty) are still shown. Likewise the Favourites, Scripts, Settings and Sounds folders. Others are partially empty, for instance in Animations all the items in the 'root' Animations folder are there, but all subfolders (lots) are empty.

Thoughts, anyone?

(Edit: please forgive poor spelling; there's no spellchecker on this and I have dyspraxia (my brain can spell, my fingers are jokers).

I AM HAVING THE exact same problem   but for me its happening almost ever single time...  and for it was when i downloaded the new firestorm too 

 

[Rick Nightingale]

It's happening more than I realised. The missing outfit error message is not always triggered, so there are times I'm missing inventory and have not noticed.

Today, before logging on I checked the size of my inv cache file; it was smaller than it should be. When I logged in. my inventory was 20k items short of the 70k it had when I logged off last night. Things I was using last night had vanished.

It really seems that the cache file is not being written properly when I log off. I have watched the files being written as I log off, as a temporary cache file is created and then gzipped. When I get this problem, it is notable that the size of file I get if I manually extract the gzipped cache is much smaller than the temporary file which is being gzipped in the first place. At least it looks like that if I'm interpreting what I see correctly. Has the gzip library been changed? Could it be dodgy?

[Jaylinbridges]

How did you turn off your antivirus program?  You realize that the Microsoft Defender Security software starts up again when you log back into Windows?  Turning it off the usual way is only temporary, until you relog or reboot.  It is a part of Windows, and always runs unless you have another active antivirus program running.

I think using the MS firewall is a waste of time.  But since you usually get your full FS inventory, that is not your problem.  The built in MS antivirus program will protect you from any serious virus.  What happened to you years ago is irrelevant.  You could also use MalawareBytes which will catch a few more mostly harmless but resource using programs from installing.

Chrome also has security options that annoy by blocking web pages without an up to date certificate, or have been reported as having viruses attached to downloaded files. But an active antivirus program also will catch these files and block them if they exist.  Your approach of thinking you have no antivirus and depending on a firewall will not stop an imbedded virus program.  

So did you really disable the MS Defender AntiVirus?  Because if you did not, then you need to whitelist all the FS files they suggest in their Wiki.

Here are the many obscure steps you need to do to disable all antivirus and file protection for Windows 10/11 (a foolish thing to do imo):

https://www.windowscentral.com/how-permanently-disable-windows-defender-windows-10

 

 

 

Edited October 13, 2022 by Jaylinbridges

[Jaylinbridges]

18 hours ago, Rick Daylight said:

Today, before logging on I checked the size of my inv cache file; it was smaller than it should be. When I logged in. my inventory was 20k items short of the 70k it had when I logged off last night. Things I was using last night had vanished.

It really seems that the cache file is not being written properly when I log off. I have watched the files being written as I log off, as a temporary cache file is created and then gzipped. When I get this problem, it is notable that the size of file I get if I manually extract the gzipped cache is much smaller than the temporary file which is being gzipped in the first place. At least it looks like that if I'm interpreting what I see correctly. Has the gzip library been changed? Could it be dodgy?

I sort of gave you the standard FS support answer in my other post.  This missing inventory problem with the new Performance FS is the most common complaint in the Firestorm support group.  But I've also seen that many who are told to whitelist the FS folders and files, and do it again if they were whitelisted before, but are not whitelisting all the folders, though they claim they are.  When asked to show a screen shot of their whitelisted files/folders something is usually missing.

The FS whitelisting Wiki is poorly written for anyone but an experienced techie that pays attention to detail.  The only complete procedure is for MS Defender, and even that assumes you remember the filenames that they list two pages above of scrolling before that.  I found many were not whitelisting the main FS .exe file, because the procedure is not explicit enough for a recipe follower.

Partial inventory loading has been an issue for those with poor internet connections anywhere in the path, and for wireless users, and cable users with too many TVs and gadgets attached. It is worse if your inventory is too large, or is too flat.  That has been the case for years.  The new performance FS seems to have made it worse for marginal setups.  Maybe the common use of SSD's has a timing issue if your connection is not always rock solid.  The FS support group can not explain why whitelisting the antivirus is now so necessary, when it was not before.   Their best answer is the antivirus programs are getting smarter, eg hand waving.

For the record, I have a solid fiber optic connection at 100 Mbps (no packet loss), live 50  miles from the AWS servers, and use all wired connections for everything, including the mouse and keyboard.  I also use a separate WD Black label Hard Drive with an SL cache partitioned, no SSD for the cache.  I have never had a missing inventory problem with prior FS's or the newest one, and I log in and out every day a dozen times/day. My inventory is also full of sub folders so not flat, and only 80K after 10 years of daily use.  I also do not use any software firewall, just the MS security software and the usual default modem/router port settings.

 

 

Edited October 13, 2022 by Jaylinbridges

[Jaylinbridges]

18 hours ago, Rick Daylight said:

It really seems that the cache file is not being written properly when I log off. I have watched the files being written as I log off, as a temporary cache file is created and then gzipped. When I get this problem, it is notable that the size of file I get if I manually extract the gzipped cache is much smaller than the temporary file which is being gzipped in the first place.

I didn't comment on this, even though I quoted it in the last post.  Your .gz file being smaller is the same as saying your inventory did not load completely, so this might be a timing issue when logging off.  If you log off and then delete the .gz file, SL servers send you your complete inventory when logging back in because the .gz file was missing.  So it might be a timing bug in FS.  Maybe there is some comment in their JIRA about this, other than blaming your antivirus.  I do note the instant log off does not mean the SL servers are done closing  down your account.  

Edited October 13, 2022 by Jaylinbridges

[Rick Nightingale]

Thank you for the comprehensive replies.

This will probably shock people now - but I still use Windows 7. That's how I can disable the AV permanently and know it stays that way. That's also the sort of reason that I still use Windows 7.

The OS is still under ESU from Microsoft (most people think it is out of support - it isn't), so security updates still happen for it (yes, that takes some, erm... fiddling). My installation on that PC has been held static for a couple of years now; nothing has changed and I did not update the OS again in or near the time frame over which I've seen this issue. I'm very careful what I do with the PC and how I use it. I've found (live/active) AV programs to be nothing but a pain for as long as they have existed and long ago stopped using them. Instead I rely on upstream firewalling and content blocking (which is not being triggered by anything SL related) and a personal firewall which is the only one I ever tested that I could not break through myself. I broke every other one on the market.

I would have noticed if I had missing inventory issues before. My internet is very stable; I keep an eye on it. Back in the day I was a CCNP among other things. I'm on FTTC 80/20 and about 30m from the cabinet. I'm on a 'business' level connection with my own IPv4 routed subnet and an IPv6 allocation so huge I could give an IP address to every quark in the known universe. I did try the usual of rebooting modem and router; not that this has ever fixed any issue I've had. My SL PC even has its own, dedicated, static-NAT'd public IP address. I would bet a million $ it's not my network or internet (if I had a million $ to bet, lol!)

Anyway, like I said, FS does load the entire inventory once I clear the existing cache, and loads it quickly. My inventory is well organised in folders; I'm used to not leaving too much in root folders from the days of DOS, lol. It's between logging off and logging back on, definitely, that the problem occurs.

When I watched what was happening with the files as I logged off, it really gave me the feeling that something (a GZIP thread for instance) was closing too early, before it had finished. That doesn't really make sense and I doubt that is what is really happening, but that's what it seems like from the way I seem to get an incomplete gzipped file.

Edit to add: It's not any sort of congestion on my LAN either. There aren't that many devices on it and I keep an eye on that too. All IoT stuff (like Echos) are on an isolated VLAN away from everything else. My line certainly isn't saturated. The only problem I ever had was when a NIC decided to start failing and flood the LAN with bogus packets. Wireshark soon spotted that when I realised I had a local issue; funny that it only showed up when doing SL - everything else still worked as normal but it does show how much something else can cause problems. I've always kept that in mind and investigate everything I can think of locally before thinking it's a viewer or SL problem.

Edit 2: Out of curiosity, since you mention the SSD timing... I'll move the cache to some spinning rust later today and try that.

Edited October 13, 2022 by Rick Daylight

[Jaylinbridges]

1 hour ago, Rick Daylight said:

This will probably shock people now - but I still use Windows 7. That's how I can disable the AV permanently and know it stays that way. That's also the sort of reason that I still use Windows 7.

So you are the other one?  I went back to Win7 from Win10, so I don't have to look up Win10 tutorials from the web for you.  It's been so long since I have found anyone using Win7, and had any problems.  Another lesson, but MS profits come first.

My NAT router firewall stops any invasion from ports unless I approve, and is all the Firewall I need.  I have checked all the paranoia sites that try to break into your computer, and they cannot even see me.  I do not need anything more.  A software firewall is redundant, and will just slow your web transfers down.  Too eliminate your firewall messing with your files, I would turn it OFF.  Then install the Win7 virus program, Microsoft Security Essentials, and leave it on for real-time protection.  That is really all you need!  It has never caused any computer issue, but has caught plenty of viruses from downloaded files and programs.  I don't suggest you run without realtime virus protection, and your firewall is not giving you that.

My Drive I:/ contains subfolders for my SL caches, for Firestorm:

To set up the whitelist, add these folders and programs to Security Essentials.  And set the whitelist like this (red is your win7 acct name which I crossed out):

If you still have missing inventory, then It's a timing bug in FS or your system, or a network issue.  Maybe a FS developer will figure out, but if they cannot reproduce it, don't hold your breath.  

Edited October 13, 2022 by Jaylinbridges

[Rick Nightingale]

I've reverted to an older version on request from support, to see if it happens there.

Wow - it certainly makes me realise just what an improvement the latest FS version brought. My framerate has dropped by 75%!

Edited October 13, 2022 by Rick Daylight

[Lyndka Cochrane]

On 10/11/2022 at 2:24 AM, Rick Daylight said:

Thanks - yeah, I did that. Wiped the cache folder and the appdata folder. Uninstall wiped the program files folder; I checked. I restored settings from a backup. I doubt the issue is in there. It seems to be a failure when exiting, as if it fails to re-write the new inventory cache file. Sometimes.

I've even tried the latest FS beta (to try out the local mesh facility); that does it too.

I'm a bit tempted to wipe again and revert to the previous release version, but I'm enjoying the much improved rendering right now.

Edit: Oh nuts, I'm going to bite the bullet and revert to the previous. If that stops the issue, I know it's this relase of FS and not my PC.

I had some silly issue with Firestorm dropping it's settings every few days - after a long discussion on the JIRA - they thought it might be Firestorm crashing as it closed down and corrupting the settings files

What eventually "cured" it for me was turning off Windows Write Caching on my system drive - it seems that Firestorm was exiting, Windows was being asked to Shutdown by me & probably thought the cache was now invalid so not writing it...

Now - I get missing content in Inventory folders every few days necessitating a clear Inventory Cache & restart - which does solve things for a few days... I can't help wondering if this is also disk cache related and whether there's any Windows API calls that Firestorm could use to flush the caches as it closes down.

The irony is  - we all used to say "You need a decent computer to run SL" and now we have - the inner workings haven't caught up

[Rick Nightingale]

@Lyndka CochraneAll of that makes a lot of sense - they are the sort of things I've been thinking. It does seem as if the cache file is not finished writing, and I do often shut down the PC quickly after exiting FS.

I am now on the very latest beta and it has happened twice in four days. I've been trying to provoke it with a lot more relogs that usual. Support, at the moment, think it is a connection/network issue as I log on (thus not getting a full inventory from LL at that point???), but I really don't think so.

I'm 99% sure that I have observed the cache file being written when I log off, and it being written smaller than it should be. If the inventory loss happened at log on, I would have the missing wearables and truncated inventory immediately then, not have it happen at the next log on. I'm watching every time I log off now, trying to catch that again so I can be 100% sure and report that.

[Rick Nightingale]

Yes - reliable reproduction!

It is Windows shutting down before FS has finished writing files.

I felt it from the start. It's that faster (apparently) closing of the viewer. It used to take a good number of seconds to close the viewer. Since the latest release version, it closes almost instantly but clearly (from watching the cache folder) has certainly not finished what it needs to do.

No wonder I couldn't catch it happening by watching the cache. It needed me to shut down my PC quickly after closing the viewer. Even with Windows winding up a couple of sleeping, spinning rust drives (only to shut them down again - as it does) it still doesn't finish Firestorm's threads (or maybe the write cache doesn't get flushed - not even sure it's on or how it works with an SSD).

Whatever, it is the change in Firestorm's shutdown that has caused this. I have always shut down my PC quickly like this after logging off, after Firestorm had closed.

[Rowan Amore]

I've been watching this thread and for me, it has been exactly the opposite.  Before the update, when the viewer took several seconds to close out, I would all too often log back in with an incomplete inventory.  Since using the new viewer which closes instantly, I have not had one instance of a failed inventory load.   

I don't, however, shut down the computer quickly after closing Firestorm.  Never have.  

I've just found it interesting to have 2 completely opposite experiences.  🤔