LSL String compression

[Xiija]

just curious, could you use regionsay to send the URL of one prim, and send your data via http ?

Quote

The request body size (e.g., of POST and PUT requests) is limited only by available script memory. Scripts can hold at most 32k characters in a string, under Mono, as characters are two bytes each, so, scripts cannot upload over 32k UTF-8 characters

 

Edited February 16, 2023 by Xiija

[Quistess Alpha]

18 minutes ago, Xiija said:

send your data via http

IIRC, LSL http stuff still has occasionally frustrating low character limits.

Hmm, reading the wiki,

Quote

[http_request]  body is limited to 2048 bytes; anything longer will be truncated to 2048 bytes.

[http_response] There is no limit, other than script size, to the amount of data that can be sent by this function.

so, the messaging protocol for maximum data transfer is a little convoluted.

Edited February 16, 2023 by Quistess Alpha

[primerib1]

21 hours ago, Wulfie Reanimator said:

If the goal is absolutely minimal script memory usage, you could run it through @Sei Lisa's LSL PyOptimizer.

I'm not sure if there are any benefits in doing that since this is a user function, which should use a minimum 512 byte block of memory.

User functions no longer immediately consume 512 bytes of memory with Mono. That was the old behaviour of LSL. See https://wiki.secondlife.com/wiki/LSL_Script_Memory#Declaring_functions

Rather, the Mono runtime will allocate additional 512-byte blocks as needed from free space.

[Wulfie Reanimator]

44 minutes ago, primerib1 said:

User functions no longer immediately consume 512 bytes of memory with Mono. That was the old behaviour of LSL. See https://wiki.secondlife.com/wiki/LSL_Script_Memory#Declaring_functions

Rather, the Mono runtime will allocate additional 512-byte blocks as needed from free space.

That's what I meant, sorry for the shorthand. 😋

If you were to put Jenna's function into a script and check llGetUsedMemory/llGetFreeMemory (without calling the function at all), then put the function through the optimizer before checking llGetUsedMemory/llGetFreeMemory again, there will be no difference in output despite the memory optimizations. But once you call the optimized function as _("", 0); to minimize memory usage from the function call, you'll notice that 1024 bytes are reserved by the user-function.

While Mono won't reserve 512 bytes for every call, it will reserve at least 512 bytes if it has no pre-allocated memory block to use. That block will then stay in use for the script (not just by that user function I imagine), until it either fills up or gets cleaned up by garbage collection.

Edited February 16, 2023 by Wulfie Reanimator

[Lucia Nightfire]

Just a critique, but if it can be avoided, it is most likely best not to use functions inside the test component of a for/while loop if the return will always be the same.

In the examples I saw in the last page, every loop iteration executes llStringLength() when it didn't seem to actually be a variable in the loop.

Edited February 16, 2023 by Lucia Nightfire

[Jenna Huntsman]

So, did some more tweaking today, here's what I've got now.

@Quistess Alpha, I've managed to get rid of the Modulo operation, but I can't see a way around getting the division operation out while not adding to the memory impact of the function itself (so, no second loop); Unless someone has any ideas?

string MemComp(string sInput, integer iDir)
{ //Compress 2 UTF-8 chars into UTF-16, or decompress UTF-16 to UTF-8. Credit: Jenna Huntsman, Mollymews, Quistess Alpha
    string sOut;
    integer iLen;
    for (iLen = llStringLength(sInput)*2; iDir < iLen; iDir = iDir + 2)
    {
        if(iDir & 1) //If iDir is a en even number (including 0), we're encoding - any odd number will decode.
        { //Decoding
            integer c = llOrd(sInput,iDir/2);
            sOut += llChar((c >> 8) & 255) + llChar(c & 255);
        }
        else
        { //Encoding
            sOut += llChar((llOrd(sInput,iDir) << 8) | llOrd(sInput,iDir+1));
        }
    }
    return sOut;
}

 

[Love Zhaoying]

Have you considered using "shift" instead of mult / divide? Since it is "by 2"? (LSL has << and >> I thought.)

[Wulfie Reanimator]

iDir = iDir >> 1; // divide by two
iDir = iDir >> 2; // divide by four...

The same works for multiplication.

iLen = llStringLength(sInput) << 1; // multiply by two
iLen = llStringLength(sInput) << 2; // multiply by four...

 

Edited February 16, 2023 by Wulfie Reanimator

[Love Zhaoying]

1 minute ago, Wulfie Reanimator said:

iDir = iDir >> 1; // divide by two
iDir = iDir >> 2; // divide by four...

 

Yeah, that. 

[Jenna Huntsman]

12 hours ago, Wulfie Reanimator said:

iDir = iDir >> 1; // divide by two
iDir = iDir >> 2; // divide by four...

 

Ah, that's it! My brain got stuck, so here's the new optimized function, with the Modulo and division removed and the string length stored in a variable:

string MemComp(string sInput, integer iDir)
{ //Compress 2 UTF-8 chars into 1 UTF-16, or decompress 1 UTF-16 to 2 UTF-8. Credit: Jenna Huntsman, Mollymews, Quistess Alpha, Wulfie Reanimator
    string sOut;
    integer iLen;
    for (iLen = llStringLength(sInput)*2; iDir < iLen; iDir += 2)
    {
        if(iDir & 1) //If iDir is an even number (including 0), we're encoding - any odd number will decode.
        { //Decoding
            integer c = llOrd(sInput,iDir >> 1);
            sOut += llChar((c >> 8) & 255) + llChar(c & 255);
        }
        else
        { //Encoding
            sOut += llChar((llOrd(sInput,iDir) << 8) | llOrd(sInput,iDir+1));
        }
    }
    return sOut;
}

 

Edited February 17, 2023 by Jenna Huntsman