Garbage collection and lists

[Samm Florian]

I'm having trouble figuring out how Garbage Collection works in MONO scripts.  If a script deletes items from a list, is the memory from those items released?  More specifically, if I am adding and deleting objects from a list so that the list's size remains relatively constant, am I risking a stack-heap collision?  I've seen some references suggest that heap memory, once used, is never reclaimed.

[Samm Florian]

I'm having trouble figuring out how Garbage Collection works in MONO scripts.  If a script deletes items from a list, is the memory from those items released?  More specifically, if I am adding and deleting objects from a list so that the list's size remains relatively constant, am I risking a stack-heap collision?  I've seen some references suggest that heap memory, once used, is never reclaimed.

[steph Arnott]

Well, i'm no expert, but as far as i understand untill the garbage collection occurs the memory is still allocated. I tend to put llSleep some where to allow it the collection to get in. Then the memory is reallocated.

[Samm Florian]

So if the list is changing pretty slowly, like once a day, then garbage collection should take care of it with no problem?

[steph Arnott]

LOL, i was thinking it was changing fast, once a day to the servers is like a billon years. If you want to satisfy curiosity put in a llGetFreeMemory and out put it over a few days.

ADDED: There are others in this forum that will give you a far better explaination than me, my lists are updated 4 or 5 times a day and i have never had a stack heap collistion or ran out of memory, actually i never ever had one on anything.

[Samm Florian]

Oh good!  I was worried: I'm working with large lists which could theoretically reach the limit if the cruft weren't cleared away, and it's for a sale item where I don't want to have to go and reset everybody's products after 6 months.  I will definitely add a memory check, but that bit I saw about "The heap never shrinks" scared me a little.

Thanks!

And you're right, I should do some testing…

[steph Arnott]

Well i suppose you could set a memory reset limit, i.e when the memory reaches a certain leve it auto resets the script, how i not sure, but i really do not know wht your list is doing so i a bit in the dark. I do suggest checking back to see what Rolig or LepreKhaun  says.

 ADDED:  I miis spelt a name.

[Nova Convair]

The heap never shrinks means the max heap pointer is never set down. That is meaningless and does not say anything about the free memory.

I have no info about the garbage collection though.

But I have an object that loads stuff from a notecard onto the heap, performs some operations and teh last command is to load the next notecard (endless loop). It's running for over a year now and sim restarts/updates and rollbacks couldn't stop it either. So the heap memory is released.

[LepreKhaun]

---

Samm Florian wrote:

Oh good!

 I was worried: I'm working with large lists which could theoretically reach the limit if the cruft weren't cleared away, and it's for a sale item where I don't want to have to go and reset everybody's products after 6 months.  I will definitely add a memory check, but that bit I saw about "The heap never shrinks" scared me a little.

Thanks!

And you're right, I should do some testing…

---

One way to test is to find an often visited code spot, such as a timer event if used or within the function that updates your lists, and insert the following:

llSetText((string)llGetFreeMemory(), <1.0, 0.0, 0.0>, 1.0);

 Then you'd artificially speed up the execution of your program. That is to say, if you expect these lists to be updated once a day, write a small harness routine that updates that list every 5 seconds. That way you can get a rough estimate of years of usage in a very short time and readily see if your code would eventually hit the dreaded stack-heap collision.

 

This approach to QA admittedly does not give precise figures which one might require under some circumstances but is easy to implement, can readily show when a problem with memory management is going to be an issue and can be modified to show other metrics as well (such as how many avatar keys can an existing application store locally and process etc).

 

As far as exactly what is happening under the hood in regards to memory management within the mono VM, I doubt you really want to go there.

• SL resident user documentation is filled with a lot of speculation, conflicting test results and is outdated almost as soon as it is written since LL is continuously tweaking its installation. This is evidenced by https://wiki.secondlife.com/wiki/User:Becky_Pippen/Measure_Script_Memory_Usage, https://wiki.secondlife.com/wiki/LSL_Script_Memory, http://massively.joystiq.com/2010/03/16/second-life-script-limitations-to-prejudice-against-mono/, https://wiki.secondlife.com/wiki/Mono to name a few prominent examples.
• The documentation for mono itself can only take you so far since SL uses a custom implementation of it. This was a unique one-of-a-kind customization required by the LSL scripting environment, where untrusted (meaning user written) code is restricted to 64k sandboxes and run within threaded time slices in an interlinked system of virtual simulators that make up our SL world. This was further complicated by having to ensure that the mono embedding did not break existing scripts written under the legacy LSO VM, which has been left intact.
• Linden Research has been somewhat opaque on its customization (perhaps for good reason). Though one can say for certain they were basing their customization on Mono1.2.6 in 2009-2010, it's uncertain whether their planned upgrade at that time to Mono2.6.1 has yet been implemented (references: http://mono.1490590.n4.nabble.com/System-Reflection-Assembly-Load-Performance-td1539105.html and http://mono.1490590.n4.nabble.com/Moonlight-2-In-Second-Life-td1549515.html).

 

With that said, there are a few certainties:

• One currently has a choice of two different garbage collection schemes under mono. They are both conservative, meaning they if there's any doubt as to what might be needed, it won't be removed.  Other than that, they differ considerably and one might, if they persevered and could devise the right tests, determine which LL currently uses. If you wish to go down that rabbit hole, here's the nitty gritty on that:

1. The original and default GC is the Boehm-Demers-Weiser implementation. Details of it can be found at http://www.hpl.hp.com/personal/Hans_Boehm/gc/ . It gets the job (mostly) done though somewhat crudely.
2. On some releases of mono one now has the option to enable and use the SGen GC, http://www.mono-project.com/Generational_GC. This is a more sophisticated, computationally heavier and more thorough garbage collecting scheme and, as such, is explicitly called rather than being automatically run at times.

Whichever GC it is, the implementation is (at best) squirrely, hard to predict and will give one a lot of head scratching if they look too closely.Regardless of what one thinks they understand or their grasp of the subject, it is best to always err on the side of caution and ensure you leave your scripts adequate "wriggle room".

 [ETA: There is one (apparent) certainty I failed to mention that I'm aware of: Before a stack-heap error kills your script, it seems that a last minute garbage collection attempt is made and, if successfull, things will go merrily along. This is enabled by the 64k limit under mono not being strictly enforced (as opposed to the 16k limit under LSO) and a momentary expansion beyond that limit is allowed if the script can successfully deflate back into bounds. I don't have the source for that information handy but I am aware of it somewhere in the literature.]

 

Anyway, that's the little I know on the subject at this time. And most of that might well be mistaken, with tomorrow finding me rethinking it all. :smileywink: I just hope that I least gave you some leads to investigate on your own into a very complex and fascinating subject.