Jump to content

"Linkset Data" is coming.


Lucia Nightfire
 Share

You are about to reply to a thread that has been inactive for 238 days.

Please take a moment to consider if this thread is worth bumping.

Recommended Posts

8 hours ago, Rider Linden said:

*ATTENTION*

I gave llLinksetDataDelete and llLinksetDataDeleteProtected a return code so that you can know if they fail.  Unfortunately this means you will need to resave any script that uses those two functions.

(*this is why we never change functions in production.)

:) this is what development is all aobut - onward and upward!

interesting how the scripts errors out when you have to re save..good that it does that when the fuction changes, it makes you think aobut it.

Edited by Jenwen Walpole
  • Like 1
Link to comment
Share on other sites

This topic is still hot, so I thought maybe I add some more functions to the LSD family :)

First, not fully replaceable but it would help scripts to prepare the LSD store without resetting the whole thing. And when reducing the event load like in the BUG-232792 is out of the table it will also reduce the count of generated link messages:

integer llLinksetDataDeleteKeys(list keys, string password);

The function takes a list of keys to delete and deletes those it can. The function will delete unprotected, protected keys and public keys (I come back on them later.) When it could delete at least one key, it will trigger only one linkset_data event, with the CSV of the deleted keys in the name parameter and the CSV of the keys if failed to delete in the value parameter. The return could be either the number of deleted keys or more generally one of three constants meaning for full success, partial success, full failure.

When an unprotected key is to be deleted, the password parameter is ignored. A protected key is deleted if the password matches the one the key is protected with (and otherwise the deletion of this key fails.) Public keys are also protected with passwords, so they are handled the same as protected keys.

Public keys

Here the fun begins for all people who wait the access from outside the linkset:

integer llLinksetDataWritePublic(string name, string value, string password);
string  llLinksetDataReadPublic(string name, string password);
integer llLinksetDataDeletePublic(string name, string password);

These functions create, update, read and delete public keys in the LSD store attached to the linkset the script runs in. I.e. 'from inside' the linkset the keys behave quite similar to protected keys. But these keys are also available for the public access:

integer llLinksetDataWriteInto(key target, string name, string value, string password);
string  llLinksetDataReadFrom(key target, string name, string password);
integer llLinksetDataDeleteFrom(key target, string name, string password);
integer llLinksetDataResetInto(key target);
list    llLinksetDataListKeysFrom(key target, integer first, integer count);
list    llLinksetDataFindKeysFrom(key target, string regex, integer first, integer count);
list    llGetObjectDetails(key target, [OBJECT_DATA_AVAILABLE, OBJECT_KEYS_COUNT]);

The functions take the key of an object and operate on the LSD store attached to the root prim of it. The naming is not important and using llGetObjectDetails also not copyrighted :) However, the functions operate only on the public keys in that LSD store.

Especially the resetting only wipes out the public keys and leaves the protected and unprotected keys untouched, so the scripts inside the target not loose any data. Listing keys and the number of them also ignores protected and unprotected keys. Only DATA_AVAILABLE returns the free space, because the script can not know which of it will be used for comming public keys.

Writing access 'from outside' is only available when the script has modify permission to the target linkset (in the next post I introduced a pin which lifts this restriction.) The functions trigger appropriate linkset_data events to be handled by the scripts running within the target linkset. The accessing script will not need any event (I think return value is sufficient.)

Read access from outside will have the same restrictions llGetObjectDetails function has.

Finally, issuing llLinksetDataWriteInto creates a key in the LSD store, when it was not there yet, and this key can be read 'from inside' the linkset via the llLinksetDataReadPublic function as if it was created via llLinksetDataWritePublic.

This way scripts can provide data for free or secure access from other objects on the same region without using chat messages.  We can use scripted and unscripted objects to create environments or experiences by providing required data in the LSD stores of these objects, in a way the object creators were not aware of. Even automatically, e.g. via scene rezzers.

Edited by Jenna Felton
mentioned pin, typo
Link to comment
Share on other sites

I tried to keep the design of new functions close to the current ones, so that extending the system wont be too difficult.

However I realized that scripts might want to close access to the LSD store even to scripts which owner has modify right to their linkset. Then the script would need to set a pin (not my idea actually but I can not find the post)

llLinksetDataSetPublicAccessPin(integer pin); 
// when pin = FALSE, updating into LSD from outside is only open 
// when the linkset is modifyable, otherwise requires same pin.

Edit: Actually the pin can be also a regular parameter for all public access  functions that update the LSD store from outside. When the pin is not set to the LSD store, then access functions will use FALSE as pin and have success when the linkset is modifyable. Otherwise they use a pin number and have success when it is the correct one, regardless if the linkset is modify or not. This way no additional write, delete, reset functions are necessary..

And one more thing. Also not my idea, but I would also like to see the action parameter in the linkset_data event built of flags instead of being just a number. This way you can combine the actions WRITE, DELETE, RESET with additional parameters like PROTECTED (protected slot of LSD was updated), PUBLIC, CREATED (the key was created upon writing, and not just updated), etc. and so you can get a vast number of possible actions, and still be able to check them all easily.

Edited by Jenna Felton
Link to comment
Share on other sites

I've got a a couple of questions that maybe someone here who's been doing some extensive testing might be able to answer. Specifically:

  • Do the keys stay in the same order in LSD store when added to the store?
    • ie: can I think of the LSD store as a stack and depending on which way I approach the index it can be FIFO or LIFO?
  • Providing the stack is "static", if I update a key with a new value does it retain it's position in the stack or does it move to the top of the stack?

We've got two function which seems to say yes to the first point.  llLinksetDataListKeys and llLinksetDataFindKeys.  Both use an index into the LSD store to determine where to start on getting the listed return.

But the second point is ambiguous.  The cases where this is going to be pertinent is going to be narrow, but it might be important in some cases, especially for things like traffic statistics where you want to pop events, IDs, or ?? off the bottom of stack after a set time.  I"m sure if I can get enough time to test I'll probably figure this out but if someone already has an insight, I figured no harm in asking. 

 

Edited by Anna Salyx
fix typos
Link to comment
Share on other sites

It looks like some sort of sort maybe going on:

Stored 10 keys named "0", "1", "2", ... "9".

listed keys by index number 0 to 9 and got expected results. Key names "0" through "9" listed in the order they were saved.

deleted key named "4".

listed keys by index number and "4" is indeed gone.

added key with name "4"

listed keys by index 0 through 9.  Expected key named "4" to be at the last position, instead it's in position between key names "3" and "5".

expanded the data writes for keys named "0" though "19"

listed keys by index 0 - 19.

keys were stored apparently by the listed output:  "0", "1", "10", "11", "12", "13", "14", "15", "16", "17", "18", "19", "2", etc

so... it appears to me that the storage is not a static stack, but being dynamically sorted somehow/someway as the keys are added.

 

Link to comment
Share on other sites

1 hour ago, Anna Salyx said:

Do the keys stay in the same order in LSD store when added to the store?

They do not. I created a quick test script in which I add keys in the following order: "0,Z,2,A,4" and then immediately try to list said keys individually by index. The keys returned indicate their indies are not the same order in which they were entered: "0,2,4,A,Z". However, I did also try updating various key values, and that does not appear to alter their index as far as I can tell.

default
{
    state_entry()
    {
        llLinksetDataReset();
        
        llLinksetDataWrite("0","zero");
        llLinksetDataWrite("Z","one");
        llLinksetDataWrite("2","two");
        llLinksetDataWrite("A","three");
        llLinksetDataWrite("4","four");
            
        llOwnerSay("0: "+llList2CSV(llLinksetDataListKeys(0,1)));
        llOwnerSay("1: "+llList2CSV(llLinksetDataListKeys(1,1)));
        llOwnerSay("2: "+llList2CSV(llLinksetDataListKeys(2,1)));
        llOwnerSay("3: "+llList2CSV(llLinksetDataListKeys(3,1)));
        llOwnerSay("4: "+llList2CSV(llLinksetDataListKeys(4,1)));
    }

    touch_start(integer total_number)
    {
        llLinksetDataWrite("Z","one111");
        llLinksetDataWrite("2","two222");
        
        llOwnerSay("0: "+llList2CSV(llLinksetDataListKeys(0,1)));
        llOwnerSay("1: "+llList2CSV(llLinksetDataListKeys(1,1)));
        llOwnerSay("2: "+llList2CSV(llLinksetDataListKeys(2,1)));
        llOwnerSay("3: "+llList2CSV(llLinksetDataListKeys(3,1)));
        llOwnerSay("4: "+llList2CSV(llLinksetDataListKeys(4,1)));
    }
}

This is reinforced by a comment Rider made on BUG-232751 :

Quote

When two linksets are merged the simulator will make a best effort to move the datastore from the old root into the new.
If there is insufficient space, the simulator will discard key/value pairs that do not fit.

The internal sequence of the map used for Linkset data is not ordered, so if you have a keyname of "B" a key with the name "C" is not guaranteed to be the next item in the sequence. (nor is the store guaranteed to start with "A". )
The simulator will attempt to keep as much as it can however.

Suppose you have 10K left in the store. And you merge in a prim with the following keys/sizes:
[ "A": 11K, "B": 5K, "C": 1K ... "G":1K]
The simulator will skip "A" since there is not enough space, but keep "B" at 5K and "C" through "G" at 1K each, since there is enough space left in the store for all of those.

One other note. If both linksets have data with the same key, the root's key value pair will not be replaced.

The llLinksetDataListKeys and llLinksetDataFindKeys functions both sort the list of keys before returning them to your script.

 

Edited by Fenix Eldritch
added emphasis to Rider's quote for relevant area
Link to comment
Share on other sites

I was doing the same.  pulling by single index along.
 

while (L < maxCount) {
	llOwnerSay(llList2String(llLinksetDataListKeys(L, 1), 0));
    ++L;
}

So I was pulling one indexed position at a time, but still it's being sorted. I'm thinking for my needs it really doesn't matter but again for some tracking traffic projects I had in mind a FIFO stack would be nice.  I can still do what I've got in mind, but it'll take a bit of extra thought.

Link to comment
Share on other sites

On 10/24/2022 at 5:29 PM, Rider Linden said:

Which of those two would people prefer?  

  1. linkset_data is thrown even on a "trivially" successful operation. 
  2. linkset_data is thrown only when there is a change to KVP data. 

I'd prefer #2

if some usecase needs the event on all commits, that KVP really should be using a nonce.

On 10/25/2022 at 12:01 AM, Love Zhaoying said:

Ok, had to read back: "no change" meaning, an LSD KVP update resulted in no update because the KVP didn't change? Well then, if that's the case, we need to tell everyone, "HEY!! Hey hey heyyyy! One of the links tried to update this here KVP, but the value did NOT change!" Because that way, the script that needs to know can stop doing whatever it would have done if the value HAD changed. 

Use a nonce.

On 10/25/2022 at 3:41 PM, Lucia Nightfire said:

Except when Oz ordered llSetPrimMediaParams() & llSetLinkMedia() changed to add url schema enforcement. 😉

I still haven't fully recovered from that... At least I got them to setup a temp region without that change so people could recover data that got locked out.

Link to comment
Share on other sites

Do not depend on LSD key index order as if it was experience key index order.

LSD key order is never guaranteed to be the same even if you reset the data, re-enter each key name and value in the same order each time, resetting and repeating.

It can be different each time.

The only thing stable is the ascending alpha numeric sort order done with all key names, before any index inputs are handled with llLinksetDataListKeys().

I'm thinking of filing a request for a sort specified version of both LSD List and LSD Find.

Link to comment
Share on other sites

On 11/3/2022 at 4:45 AM, Gabriele Graves said:

I expect that this is because the key/values are being stored as a dictionary/hashmap/hashtable which often do not guarantee any particular order and are designed mainly for fast random access.

Except for Python >= 3.7 😄

But that's neither here nor now. So best bet is to grab the keys and sort them.

Link to comment
Share on other sites

Depending on the whole external accessing of LSD this might not be so necessary, but a function I'd like to see would be
 

llRezObjectWithData(object,pos,vel,rot,param, ["key1","val1","key2","val2"]);

 

Would be useful to rez a configured object, rather than the current, rez, wait for event, and handle messages to pump config into rezzed item.

Also, that maybe read write and delete should have been overloaded with a list instead of adding protected versions of the functions, allowing for easy future expansion without more functions requiring multiple variations, e.g.

llLinksetDataWrite(key, value, [LSD_PASS,"secret", LSD_PUBLIC, TRUE]);

The same for read/delete obviously... (though some options may not apply for all)

Instead of exponentially more new functions in the future.... with the current scheme if the scope were to be increased.

llLinksetDataWrite(key, value);
llLinksetDataWritePublic(key, value);
llLinksetDataWriteProtected(key, value, pass);
llLinksetDataWriteProtectedPublic(key, value, pass);

Could also be further expanded to accept a parameter like LSD_TARGET_ID to allow reading a remote object in the future, with or without pins or whatever is decided (but pins seem superfluous to me given we have passwords already), though some form of secondary functions for that may be more on form with the key being the first proper argument, not a parameter in the list.

Edited by WolfGang Senizen
clarity
  • Like 5
Link to comment
Share on other sites

Not sure if this has been asked before. But is linkset data reliable in the event of a sim crash? I. E, if I write data within the 15 minute cache window and the sim crashes, will the data be rolled back like everything else? I'd like to store the avatars on a sim in the event of a crash, but if the data gets rolled back too... Then everything written before cache commit is lost 

Edited by Skyler Ghostly
Link to comment
Share on other sites

1 hour ago, Skyler Ghostly said:

Not sure if this has been asked before. But is linkset data reliable in the event of a sim crash? I. E, if I write data within the 15 minute cache window and the sim crashes, will the data be rolled back like everything else? I'd like to store the avatars on a sim in the event of a crash, but if the data gets rolled back too... Then everything written before cache commit is lost 

It hasn't been asked, but I'd assume LSD gets rolled back too.

Link to comment
Share on other sites

Assuming there's no crash: At what point does the LSD get propagated so that it is available on any sim you go to (if the object with the script is worn), or upon the next region / simulator restart (for example, if the script is not worn)?

1. If wearing the object with the LSD script:

- When the object with the LSD script crosses region boundaries?

- When you log out? 

- etc.

2. If not wearing the object with the LSD script:

- After some LSD commit "every X# of LSD transactions"?

- etc.

3. Regardless of wearing the object or not with the LSD script:

- After some LSD commit "every X# of LSD transactions"?

- When the script exits gracefully?

- etc.

 

Link to comment
Share on other sites

1 hour ago, Love Zhaoying said:

Assuming there's no crash: At what point does the LSD get propagated so that it is available on any sim you go to (if the object with the script is worn), or upon the next region / simulator restart (for example, if the script is not worn)?

Quote

... unlike some other functions if you move an object containing Linkset Data (or teleport wearing an object containing Linkset Data) from a region that supports it to a region that does not, all Linkset Data stored with the object will be lost, even if you go back to a region that supports the feature. This limitation will no longer exist once all regions on the grid understand how to parse Linkset Data

That is from the the deploy plans last week.  I'm parsing it to mean that the LSD store is a prim property that moves with the object as it crosses region boundaries.  It's less about it being propagated and 'made available' to the different sims, and more that the new sim will read the properties of the prim when it enters and acts accordingly.  So traveling about should, in my mind, not pose any problems from the moment the data is written/stored.  I'd be interested in finding out from an official source if this understanding is correct, or where I"m mistaken.

Also, I think the overall limitation of how it gets saved to the asset database in the event of crashes will still come into play.  That said, I still wonder why it is that I can make a change or changes (like setting clothing textures, or updating body alphas via HUD) and then teleport to a half dozen locations, spend 2 or 3 hours chilling somewhere with friends, and then if i crash during a teleport at the end of all that, all those changes revert back as if I had never done them at all.

  • Like 1
Link to comment
Share on other sites

5 hours ago, Skyler Ghostly said:

Not sure if this has been asked before. But is linkset data reliable in the event of a sim crash? I. E, if I write data within the 15 minute cache window and the sim crashes, will the data be rolled back like everything else? I'd like to store the avatars on a sim in the event of a crash, but if the data gets rolled back too... Then everything written before cache commit is lost 

It has been asked before. LSD is a prim property and subject to rollbacks like anything else.

BTW, rollbacks can be less than a minute at best and over an hour at worst.

  • Like 3
  • Thanks 1
Link to comment
Share on other sites

25 minutes ago, Anna Salyx said:

That is from the the deploy plans last week.  I'm parsing it to mean that the LSD store is a prim property that moves with the object as it crosses region boundaries.  It's less about it being propagated and 'made available' to the different sims, and more that the new sim will read the properties of the prim when it enters and acts accordingly.  So traveling about should, in my mind, not pose any problems from the moment the data is written/stored.  I'd be interested in finding out from an official source if this understanding is correct, or where I"m mistaken.

Yes,  I remembered the quote about "if you move an object containing Linkset Data..from a region that supports it to a region that does not".  When I first saw the quote, I thought, "Ok..but when WILL it work vs. "not work" - what's the persistence?

Regarding being a "prim property"..the part in the quote about "all Linkset Data stored with the object will be lost, even if you go back to a region that supports the feature" implies that as you cross region boundaries / teleport, prim properties are "written" even if you do not change them.  This part was surprising to me (kind of scary, actually).  

29 minutes ago, Anna Salyx said:

Also, I think the overall limitation of how it gets saved to the asset database in the event of crashes will still come into play.  That said, I still wonder why it is that I can make a change or changes (like setting clothing textures, or updating body alphas via HUD) and then teleport to a half dozen locations, spend 2 or 3 hours chilling somewhere with friends, and then if i crash during a teleport at the end of all that, all those changes revert back as if I had never done them at all.

I guess your experience addresses my question - hopefully it will be less "fragile" after full implementation.  Otherwise, "any old crash" (you, region) or region restart could result in loss of the LSD.  (Not an optimal situation.)  If other, existing "prim parameters" were like this, we'd have all kinds of issues that I think we do not have today.

 

Link to comment
Share on other sites

23 minutes ago, Love Zhaoying said:

Regarding being a "prim property"..the part in the quote about "all Linkset Data stored with the object will be lost, even if you go back to a region that supports the feature" implies that as you cross region boundaries / teleport, prim properties are "written" even if you do not change them.  This part was surprising to me (kind of scary, actually). 

I don't read it that way, and maybe that's incorrect on my part. No, I see it more that when you enter a region that doesn't support a particular prim property, the simulator sees it, but considers it 'invalid data' and so a 'clean up' is triggered silently removing what is seen as corruption and that then commits the new 'clean' prim values to the asset database.  I don't think there is the overhead of "enter a region > read prim properties > save prim properties (just because)." 

Well I'd like to think that that final step of overhead is not being done *every-time* I teleport around the grid, or rez objects.

  • Like 1
Link to comment
Share on other sites

1 hour ago, Anna Salyx said:

That said, I still wonder why it is that I can make a change or changes (like setting clothing textures, or updating body alphas via HUD) and then teleport to a half dozen locations, spend 2 or 3 hours chilling somewhere with friends, and then if i crash during a teleport at the end of all that, all those changes revert back as if I had never done them at all.

Changes made to attachments are not committed until they are detached, so when you crash hard enough (not just logging out or getting disconnected), the normal "save on unrez" process doesn't take place.

You can, for example, change the texture on an attachment and then make copy the inventory item you're currently wearing. The copy you've just created won't have the changed texture, it's a copy of the attachment before you put it on (including script state).

It should follow that LSD will behave exactly the same.

  • Like 2
  • Thanks 1
Link to comment
Share on other sites

You are about to reply to a thread that has been inactive for 238 days.

Please take a moment to consider if this thread is worth bumping.

Please sign in to comment

You will be able to leave a comment after signing in



Sign In Now
 Share

×
×
  • Create New...