Packing a Key into a String

[Love Zhaoying]

13 minutes ago, elleevelyn said:

is unclear to me why you are not using ordinal order to construct unique identifiers. eg. getNext(0) getNext(1) ... getNext(MAX) , getNext(0), ...

Ultimately, because the LSD is "neutral" and can be used as shared storage for multiple scripts.

One example: Instead of passing full JSON to one another, they can just pass the UUID of the data in LSD.

Another example: multiple processes hosted by separate "host" scripts can share "child" / "helper" scripts and each process needs to pass unique UUID's (so only it's data is professed). The second example is more "real". 

The normal use-case is creating a "data instance", and having a process pick up the "wrong" data would be very bad (since it would also modify and dispose of the wrong data). UUID's work fine so far.

Edited March 17, 2023 by Love Zhaoying

[elleevelyn]

1 hour ago, Love Zhaoying said:

Another example: multiple processes hosted by separate "host" scripts can share "child" / "helper" scripts and each process needs to pass unique UUID's (so only it's data is professed). The second example is more "real". 

 

a way to do the multiple processes case is with a broker script that manages the allocation of unique key/codes. Each client process requests a block of codes for its own use. When the block is exhausted then either roll over the block or request a new block from the broker

the number of processes and size of the assigned block can be quite large

for example: integer 32 bits: 1st 8 bits is the unique process prefix: 256 processes and 2*24 unique codes for each process. Key = (string)Prefix + (string)blockcounter

the bit split can be other. Like 7 bits for 128 processes. 9 bits for 512 processes with the block size adjusting accordingly

scaling up to a 2 integer broker then it can be 2^32 processes each with a 2^32 block. Or 2^16 processes, 2^48 block. etc

 

edit add. The 2 integer method is the simplest as the broker need only assign the process a unique integer identifier. The process managing its own 32 bit counter

 

Edited March 17, 2023 by elleevelyn
typo

[Love Zhaoying]

7 hours ago, elleevelyn said:

a way to do the multiple processes case is with a broker script that manages the allocation of unique key/codes.

I can't use / it would be too inefficient to use a separate script for that. Thanks for the suggestion, though! Basically, each script is completely independent. Unless it uses a client helper script to "do things". Since each script has the potential of creating its own keys, having a separate script for creating the keys would slow things down too much. (Only the "client" scripts actually create keys, and the "helper" scripts use whatever keys they are told to use.)

Luckily, most of these "keys" are very dynamic and don't need a lot of persistence. Now that LSD size is planned to double, it should matter less and less if the keys are "bigger".

[Profaitchikenz Haiku]

12 hours ago, Wulfie Reanimator said:

You'd have to show me what you're seeing,

I'm seeing what Quistessa has posted as an example of her shown name, and I think now I understand. 

 

12 hours ago, Quistess Alpha said:

When you pick a new name (by paying the 40$ fee) you may opt to change only your first name, and not your last name.

So that puts the kybosh on my idea of names,. as the last name which is immutable is also not going to be unique.

[Quistess Alpha]

2 hours ago, Profaitchikenz Haiku said:

the last name which is immutable

That's also changeable, I just didn't mention that it's changeable because that's the main reason people change names. if I wanted to be trendy I could change my name to "Tessa (Susan Sausage)" , assuming that's not taken yet. ('Samantha Sausage' is taken *shrugs*)

[Profaitchikenz Haiku]

OK, so back to the original question, is there a space-saving way of compacting s full-length key into something shorter?

I'd say yes, the safest way I can think of is therefore take every n-th character from the full key as string to create a 16-character version. There is a faint chance that it's going to clash. If that faint chance is too risky for the particular application, then one has to bite the bullet and use up more space.

2 hours ago, Quistess Alpha said:

('Samantha Sausage' is taken *shrugs*)

There are some very very strange people in the world, and for some reason, SL seems to grab more than it's fair share of them.

[Love Zhaoying]

It's safe to remove the hyphens! 
I'll bet you 1L$!

[Quistess Alpha]

1 hour ago, Profaitchikenz Haiku said:

space-saving way of compacting s full-length key into something shorter?

the OP asked about LSD specifically and that's been pretty well covered already. There's no "one method works best" you have to consider the use-case:

• bi-directional: If the 'packed' key needs to be turned back into a 'normal' key, you have to conserve all 128 bits of information. see all the specific examples in the script library.
• one-direction: If you only need to go 'one way' say for the script to remember 'have I seen this person before?' you have to consider how many keys you're storing at once, and how often your full list of keys might 'turn over'. rule of thumb I think is to take the number stored and square it, rounding up: so, If I need to remember say, 200 keys, 200*200 = 40,000 < 2^16 bits of information. 32 bits are easy to handle so might as well for safety. Try (integer)("0x"+(string)(<key>)); and convert to base 64 (llDeleteSubString(llIntegerToBase64(int),-2,-1);) if necessary.

ETA: for the above '200 keys' scenario, the 'just square it' heuristic says you need to keep at least 4 of the hex characters (4 bits per hex digit, *4 =16) but that might be a bit 'close to the line' if the list fills and empties frequently (Say, if you're remembering 200 (different) avatars every day, you'd expect one collision almost every 2 days! but assuming that the group of people your tracking is always different is also unrealistic. . . you can go very far down the rabbit hole.)

Edited March 17, 2023 by Quistess Alpha

[Love Zhaoying]

43 minutes ago, Quistess Alpha said:

the OP asked about LSD specifically and that's been pretty well covered already. There's no "one method works best" you have to consider the use-case:

• bi-directional: If the 'packed' key needs to be turned back into a 'normal' key, you have to conserve all 128 bits of information. see all the specific examples in the script library.
• one-direction: If you only need to go 'one way' say for the script to remember 'have I seen this person before?' you have to consider how many keys you're storing at once, and how often your full list of keys might 'turn over'. rule of thumb I think is to take the number stored and square it, rounding up: so, If I need to remember say, 200 keys, 200*200 = 40,000 < 2^16 bits of information. 32 bits are easy to handle so might as well for safety. Try (integer)("0x"+(string)(<key>)); and convert to base 64 (llDeleteSubString(llIntegerToBase64(int),-2,-1);) if necessary.

Great summary!

[elleevelyn]

13 hours ago, Love Zhaoying said:

I can't use / it would be too inefficient to use a separate script for that. Thanks for the suggestion, though! Basically, each script is completely independent. Unless it uses a client helper script to "do things". Since each script has the potential of creating its own keys, having a separate script for creating the keys would slow things down too much. (Only the "client" scripts actually create keys, and the "helper" scripts use whatever keys they are told to use.)

Luckily, most of these "keys" are very dynamic and don't need a lot of persistence. Now that LSD size is planned to double, it should matter less and less if the keys are "bigger".

yes that's a issue with brokers

a broker-less multi-vendors - multi-scripts - multi-script-internal processes scheme which produces unique codes less in length than (string)key

from the rule that scripts in object Contents must be uniquely named. A unique code can be obtained from linknumber + scriptname + counter

example: Say script has  same inventory name "A", then when dropped into the object with 3 links and 3 scripts in each link for 9 independent processes then prefixes: "0A", "0A 1", "0A 2", "1A", "1A 1", "1A 2", "2A", "2A 1", "2A 2"

naming convention is scripts to be named with an alphabet char in the 1st position. The underscore char "_" reserved and must be coded internally as the prefix delimiter. Creating the prefix code,  a one time occurrence in state_entry, edit [dynamically in changed event]

each script can then manage its own sub-code scheme as it prefers. Examples:

"0A 1_" + "P1" + (string)counter
"0A 1_ + "P2" + (string)counter
"1A 2_ + "0" + (string)counter
"1A 2_ + "1" + (string)counter
"2A_" + (string)counter

where "PI", "P2", "0", "1" is a sub-process code internal to the script. In the "2A_" case there is no sub-process code (script only having 1 process)
 
bit size of counter is up to the scripter, beyond 32 bits then a multiple integer type counter

Edited March 17, 2023 by elleevelyn
edit add [ ]

[Love Zhaoying]

1 hour ago, elleevelyn said:

yes that's a issue with brokers

a broker-less multi-vendors - multi-scripts - multi-script-internal processes scheme which produces unique codes less in length than (string)key

from the rule that scripts in object Contents must be uniquely named. A unique code can be obtained from linknumber + scriptname + counter

example: Say script has  same inventory name "A", then when dropped into the object with 3 links and 3 scripts in each link for 9 independent processes then prefixes: "0A", "0A 1", "0A 2", "1A", "1A 1", "1A 2", "2A", "2A 1", "2A 2"

naming convention is scripts to be named with an alphabet char in the 1st position. The underscore char "_" reserved and must be coded internally as the prefix delimiter. Creating the prefix code,  a one time occurrence in state_entry, edit [dynamically in changed event]

each script can then manage its own sub-code scheme as it prefers. Examples:

"0A 1_" + "P1" + (string)counter
"0A 1_ + "P2" + (string)counter
"1A 2_ + "0" + (string)counter
"1A 2_ + "1" + (string)counter
"2A_" + (string)counter

where "PI", "P2", "0", "1" is a sub-process code internal to the script. In the "2A_" case there is no sub-process code (script only having 1 process)
 
bit size of counter is up to the scripter, beyond 32 bits then a multiple integer type counter

Cool.

[primerib1]

23 hours ago, Profaitchikenz Haiku said:

OK, so back to the original question, is there a space-saving way of compacting s full-length key into something shorter?

I'd say yes, the safest way I can think of is therefore take every n-th character from the full key as string to create a 16-character version.

How many UUIDs do you expect to store?

I mean, even a less efficient packing like mine, into 22 characters and *very* fast, I'll have to store like 2500+ UUIDs before I'm in danger of running out of space.

And if LSD gets upgraded to 128 KiB, that means I can store 5000+ UUIDs before I'm in danger of running out of LSD space.