Unusually high RAM consumption on Linux

[Hexatope]

Hello all!

Some backstory; I've tinkered around with ubuntu builds of linux here and there, tried running SL (Firestorm viewer) on it a few times over the years but never decided to seriously start using it as a primary OS until about a few weeks ago. I have a few other systems so I've been using ZorinOS 15.2 (based on Ubuntu 18.04) and I've been enjoying it quite a lot.

I have two desktop rigs, one's all AMD (R5 3600 / 16GB DDR4-3200 / Vega 64, using RADV/Kisak-Mesa 20.1.1) and the other is Intel/Nvidia (3570K, 16GB DDR3-1333, GTX 980 Ti, on Nvidia 440.82 Open Source drivers). I was using the AMD rig for Windows and the Intel one for linux and noticed that Intel was using approximately twice the amount of RAM (as reported by the System Monitor application under memory) when logged into another account in the same area. I initially thought this was an issue with linux, but after trying Zorin on the AMD machine it doesn't have this issue, using up half the memory as the intel/nvidia.

Earlier today I set up a 3rd desktop to test performance for a friend looking for some upgrades for his wife's computer, (Fx8350, 8GB DDR3, GTX 960) and noticed that it was also using twice the amount of RAM as my main AMD system. Curiously, I dug out an old Radeon HD 6950 just to see if it would still have this problem on that card. In the area I was in, consumption on the FX rig went from around 3GB of ram down to around 900MB or so. (Performance also suffered but that's to be expected with Terascale cards and OpenGL, not to mention the open source Caiman drivers.)

Can anyone advise me on if there's a setting I should change, maybe on the nvidia settings, or could someone test on their machine with other generations of Nvidia GPUs to see if they experience the same? I currently only have the two Maxwell cards, 960 2GB and 980 Ti 6GB, and I'm curious to see if Pascal or Turing have the same problem.

Any help would be greatly appreciated, I tried searching around but haven't seen anyone post about this. Thanks in advance!

[Anna Nova]

I have an Archlinux system that is my one and only computer right now (discounting the Pi3 and PiZero  ).  AMD Ryzen 2700 with nVidia GTX1080Tx, and 16Gb.  The memory footprint of my regular usage with SL is about 40% with Brave(web browser) and Firestorm, this is from 27% with just Brave.  It varies with the scene complexity, I think, which suggests that some is associated with the NVidia card.  Firestorm itself only adds 3% to the memory (i.e. 30%), but when logging in rises to 46% while all the initial rendering takes place, and then falls back to 40% or so.  Opening a second (alt avatar) Firestorm instance and logging in raises the memory footprint to about 55%, which then falls back after the rendering to 53%.

Not sure if this helps, but there it is, for what it's worth.  I'm interested in your conclusions.

[Whirly Fizzle]

Related JIRA discussion: FIRE-23299 - Memory bloat in Linux viewer since 5.1.7.55786

[Hexatope]

8 hours ago, Anna Nova said:

I have an Archlinux system that is my one and only computer right now (discounting the Pi3 and PiZero  ).  AMD Ryzen 2700 with nVidia GTX1080Tx, and 16Gb.  The memory footprint of my regular usage with SL is about 40% with Brave(web browser) and Firestorm, this is from 27% with just Brave.  It varies with the scene complexity, I think, which suggests that some is associated with the NVidia card.  Firestorm itself only adds 3% to the memory (i.e. 30%), but when logging in rises to 46% while all the initial rendering takes place, and then falls back to 40% or so.  Opening a second (alt avatar) Firestorm instance and logging in raises the memory footprint to about 55%, which then falls back after the rendering to 53%.

Not sure if this helps, but there it is, for what it's worth.  I'm interested in your conclusions.

I've been hearing a lot of people are using Arch lately, I'll download manjaro tonight when I get home from work and see if the ram issue persists with a different distro. Thanks for the input!

Also, love the raspberry pi. I have a 4b 4gb model 😄

11 minutes ago, Whirly Fizzle said:

Related JIRA discussion: FIRE-23299 - Memory bloat in Linux viewer since 5.1.7.55786

This isn't exactly what I'm experiencing. It's more like the data is being double cached because it's exactly double of what my other PC is using, that post's issue sounds like some kind of memory leak. When I teleport to a more simple area the ram consumption does drop, usually to a minimum of 2GB. I will head over to sl17b later and get some metrics and screenshots.

I should also clarify, the performance between the two machines are almost identical and it runs stable with no crashes. I'm using the latest current release on both machines with default nvidia settings.

[Nick0678]

3 hours ago, Hexatope said:

I've been hearing a lot of people are using Arch lately, I'll download manjaro tonight when I get home from work and see if the ram issue persists with a different distro. Thanks for the input!

Also, love the raspberry pi. I have a 4b 4gb model 😄

This isn't exactly what I'm experiencing. It's more like the data is being double cached because it's exactly double of what my other PC is using, that post's issue sounds like some kind of memory leak. When I teleport to a more simple area the ram consumption does drop, usually to a minimum of 2GB. I will head over to sl17b later and get some metrics and screenshots.

I should also clarify, the performance between the two machines are almost identical and it runs stable with no crashes. I'm using the latest current release on both machines with default nvidia settings.

I have Ubuntu on one laptop and Arch on another Laptop , more or less they act the same when it comes to SL and don't seem to have any issues.

Here's the memory usage from one of them using Firestorm (non crowded area). (Intel / Radeon)

I ll check later my desktop when i get home it has Ubuntu and Intel / NVidia something , but tbh don't remember having such issues. (rarely use that).

[Hexatope]

Okay so I've installed Manjaro xfce 20.0.3 on the 3570K / 980 Ti. Got the Nvidia 440.82.1 drivers installed and an alt in the same area as my AMD rig.. curiously enough, it is using slightly higher ram on the firestorm process but not twice as much, 1.7GB for the AMD GPU and 2.3GB for the Nvidia.

 

Manjaro 20.0.3 / Nvidia 440.82

 

Zorin 15.2 (Ubuntu 18.04) / Vega 64

(Off topic.. despite the ram issue I still think I prefer ZorinOS over Manjaro because I'm not as familiar with Arch. Seems like Zorin boots faster too.)

If you guys would like to keep this going, would anyone like to suggest a place we can use as a benchmark, with a preset graphics setting? (I should also point out I haven't changed the graphics settings much, 128m draw distance, everything else is on high / adv lighting on, shadows off.) I might give a newer distro of Ubuntu a try too, 18.04 is two years old now and 20.04 is available.

Edited June 22, 2020 by Hexatope

[animats]

No stats, but I have suspicions of a memory leak, or gradual memory growth, on Firestorm/Linux. The effect seems to be that visiting very cluttered areas runs up memory usage, and the space isn't recovered after leaving. Eventually paging out to disk starts. At that point, it's best to log out and back in again. Visiting a large number of regions does not cause this problem. It's the peak region that does it.

I only have 8GB of RAM, and probably need to upgrade to 16 GB.

[Anna Nova]

22 minutes ago, animats said:

No stats, but I have suspicions of a memory leak, or gradual memory growth, on Firestorm/Linux. The effect seems to be that visiting very cluttered areas runs up memory usage, and the space isn't recovered after leaving. Eventually paging out to disk starts. At that point, it's best to log out and back in again. Visiting a large number of regions does not cause this problem. It's the peak region that does it.

I only have 8GB of RAM, and probably need to upgrade to 16 GB.

Interesting.  I have long thought that I had a ration of sim crossings, but could never quite work out what it was,  somewhere around 20.  Then there is the 'arrival region' effect when I always get broken on the last sim crossing of a journey.  These may both be symptoms of some unreclaimed  memory growth that occurs on 'cluttered' sims - inevitably the target of a journey isn't an empty one!

[Solar Legion]

Manjaro user here, just freshly loaded in at a small getaway at my home and only using 1.6 GB RAM (pardon the lack of cropping):

It should be noted that I do not run "stock" Manjaro, my setup uses the Linux Zen kernel (main Arch Repo, not the AUR or Manjaro's customized Repo) along with a newer nVidia driver (DKMS subsystem, 440.82.2 again through the main Arch Repo).

CPU is an AMD Ryzen 7 2700X while the GPU is an nVidia 980 Ti.

Firestorm Settings, Ultra preset with Draw Distance dropped to 128 - 256 Meter range, Avatar Complexity just below Unlimited, eight to ten Avatar Imposters, Texture Memory Buffer maxed .... 

Edited June 22, 2020 by Solar Legion

[Nick0678]

Open your statistics panel and check your GL Image Data and Vertex buffers when you do your tests.

 

Those 2.4GB that you see are way too much unless of course you are on a non parceled sim and you are also rezzing tens of avatars and Mesh objects.

 

Fresh load my main avatar in one of my skyboxes with Ultra settings ~700MB.

(killed the FPS of course due to weak GPU but thats insignificant about  RAM usage)

Edited June 22, 2020 by Nick0678

[iceing Braveheart]

check out this over 9GB memory usage firestorm is so hungry for my system memory but refuses to touch the video cards memory allergic or what.

was with low graphics preset on windows doesn't look like a linux specific issue or problem

Firestorm 6.3.9 (58205) May 27 2020 01:20:51 (64bit) (Firestorm-Releasex64) with Havok support

https://ibb.co/nsrdzPP

<a href="https://ibb.co/nsrdzPP"><img src="https://i.ibb.co/nsrdzPP/firestorm-over-9-GB-ram.jpg" alt="firestorm-over-9-GB-ram" border="0"></a>

[Hexatope]

What the.. care to share your specs and driver versions? I don't think I would get even close to that if I tried even with one of my Nvidia cards unless I was in a fully loaded club with my max avatars slider all the way up.

[Nick0678]

What the...try another viewer and check memory usage. 9 GB's!

Try coolvl or singularity or whatever.

Starting with 1-2 gb is ok depending on the mesh body, clothes etc that you are loading but 9GB? Cmon..

I am on vacation now so here's the memory usage ~605MB loaded also pushed on Ultra on an old 2011 model Sony Vaio that i have with me.

(Xubuntu 18.04)

Quote

 

CPU: Intel(R) Core(TM) i5 CPU       M 480  @ 2.67GHz (2848.29 MHz)
Memory: 5795 MB
OS Version: Linux 5.7.7-050707-generic #202007010943 SMP Wed Jul 1 14:31:20 UTC 2020 x86_64
Graphics Card Vendor: X.Org
Graphics Card: AMD REDWOOD (DRM 2.50.0 / 5.7.7-050707-generic, LLVM 9.0.0)

OpenGL Version: 4.1 (Compatibility Profile) Mesa 19.2.8

 

Edited July 6, 2020 by Nick0678