General and Specific Help On Memory Usage

[GManB]

A script I am writing is getting pretty large and am running up against the 64K memory limit (which I think is the maximum for a script). I have some general questions and then an ask about a specific chunk of code.

• Any general patterns to keep programs under the 64K limit?
• I assume I could break the program up into separate, communicating, programs.
  • Any comments on general patterns for such a separation?
• Any general ideas for saving memory?

Here is the code that's taking up the last bits of memory. If totalNumAnswers is over about 600 I get the stack heap collision message. I could limit the number of answers to about 1K, I think, reasonably. I'm guessing storing strings is probably less memory intensive than integers, but any thoughts? The description of llListRandomize seems to indicate that it randomizes the elements in the list in place but anyone know any gotchas?

CreateScrambledAnswerList()
{
    integer i = 0;
    scrambledAnswerIndicies = [];
    for(i=0;i<totalNumAnswers;i++)
    {
        scrambledAnswerIndicies = scrambledAnswerIndicies + (string)i;
    }
    scrambledAnswerIndicies = llListRandomize(scrambledAnswerIndicies, 1);
}

 

 

Thanks,

G

[animats]

If you're just storing a large number of fixed strings, you can put them in a notecard, or multiple notecards. Notecards are random access; if you know the line number you want, you can get just that line.

 

[GManB]

12 minutes ago, animats said:

If you're just storing a large number of fixed strings, you can put them in a notecard, or multiple notecards. Notecards are random access; if you know the line number you want, you can get just that line.

 

That's what I am doing. The list holds just the randomized indexes into a notecard with the answers. I don't see any way other than a list. One could randomly generate the indexes but to avoid dups you would have to keep a list of the used values. Not efficient in time. 

G

[Mollymews]

with a indice list then will save a little under 1/2 space by storing the indices as integer rather than as string

in the range [0..999] most of the strings will be 8 bytes long. For example  the string "123". "1" = 2 bytes. "2" = 2 bytes. "3" = 2 bytes. "\0" end-of-string marker = 2 bytes. Total 8 bytes

integer is 4 bytes

 

a aside

if you want to produce a really really long arrangement of unique integer values of any length in constant space and time and then can do this with a feistel network algorithm. There is a LSL implementation of this algorithm here:

 

[Profaitchikenz Haiku]

A thought here based on something I did that whilst not similar, has a concept you could adapt.

Spread your answers through extra scripts: they communicate through link messages, each script has a number it recognises as "mine"

When you want a random question/answer, generate a single random number in the range 1 to (number of store scripts)

message that particular script (Message has the number set to specified script)

The receiving script generate a random number between 0 and number of stored answers and spits out the reply

[Mollymews]

6 hours ago, Profaitchikenz Haiku said:

A thought here based on something I did that whilst not similar, has a concept you could adapt.

Spread your answers through extra scripts: they communicate through link messages

 

i agree with this approach

a main script which handles the interactions with the user(s) and a data server script. Main script making requests of the data server script using link message

as far as the main script is concerned it doesn't care where or how the data is stored: notecards, website or KVP [ or hardcoded in the data script ]

with a question and answer data server then usually three request commands needed:

1) LIST - returns a delimted string of the names of the datasets available. Example: "General,Fauna,History,Geography,Math"
2) BEGIN NameOfDataset - where NameOfDataset is in LIST
3) NEXT - returns the next row (question/answer data) from the currently chosen/begun set, or returns EOF on which a new BEGIN is requested

other requests can be added depending on use case

like COUNT NameOfDataset returns the number of rows in the dataset

BEGIN can have a 2nd parameter. Like LINEAR|RANDOM|ARRANGEMENT. Where LINEAR returns the rows in linear order: 1,2,3,etc. RANDOM returns rows by llFrand(). ARRANGEMENT returns rows uniquely in some arranged order other than linearly

and so on

from a design pov. When there is too much for a single data server script to handle then can add additional data slave scripts.  Our main script only ever communicating/interfacing with the data server script. The data server script managing its data slave scripts

Edited January 5, 2021 by Mollymews
[ ]

[GManB]

I like both, the Feistel network and a helper script. (Thanks @Mollymews @Profaitchikenz Haiku) I slightly prefer the Feistel solution. I checked memory as the first thing in default, state_entry, i.e., before I initial any data structures and I have about 13K free memory. I am going to test to see how much memory adding the additional code for two Feistel networks will be and if not too much probably go with that solution.

Using a helper script would require some restructuring of the code. I access the list of indexes inline in four different places within, of course, the execution of code as the result of some event. The helper script solution would require moving the code after each of the four accesses to after receiving the reply in a link_message event. This potentially opens opportunities for concurrency errors to creep in. And the moved code would lose the local variable context of the code block from which it came (necessitating either creating global variables to hold that context or passing the values of the local variables to the helper script which would simply return them with the next index. Had I started this project using helper scripts it would be fine just a different structure.

G

[GManB]

The Feistel code works fine. Did not add much to memory. Can accommodate 1000 answers now w/o exceeding memory limit. It was easy to insert into the main script. I will have to do more extensive testing but the limited tests were perfect.

 

THANKS!!!!

 

G