"Linkset Data" is coming.

[primerib1]

15 hours ago, Lucia Nightfire said:

This just in...

128 KiB Linkset data allowance is coming.

[12:52:24] Linkset Data(LSD) Test: llLinksetDataAvailable() = 131072

That is all.

This is awesome! Lots more things can be stowed away in LSD

But...

With larger space, means more keys ...

I do hope they implement https://jira.secondlife.com/browse/BUG-233195 before 128 KiB LSD is rolled out.

Or we'll all be facing with even slower scripts when doing cleanup...

[Love Zhaoying]

29 minutes ago, primerib1 said:

This is awesome! Lots more things can be stowed away in LSD

But...

With larger space, means more keys ...

I do hope they implement https://jira.secondlife.com/browse/BUG-233195 before 128 KiB LSD is rolled out.

Or we'll all be facing with even slower scripts when doing cleanup...

It's also up to the user to use "smart" keys!

I recently had to go the "more entries" route so that I'd be loading smaller data sets per entry. If I make the decision to use poorly designed keys, that's on me - in most of the current cases the user will never see the key. Using meaningful key values means I can tell at a glance what each LSD entry is for, but later I'll probably go back and make the keys as tiny as possible to gobble less of the total LSD.

 

Edited March 8, 2023 by Love Zhaoying

[primerib1]

20 minutes ago, Love Zhaoying said:

It's also up to the user to use "smart" keys!

I recently had to go the "more entries" route so that I'd be loading smaller data sets per entry. If I make the decision to use poorly designed keys, that's on me - in most of the current cases the user will never see the key. Using meaningful key values means I can tell at a glance what each LSD entry is for, but later I'll probably go back and make the keys as tiny as possible to gobble less of the total LSD.

 

Well, sometimes you can't avoid ending up with hundreds, even thousands of keys.

Like @Lucia Nightfireexplained in this comment to BUG-233195

If LSD space gets doubled, the number of keys to delete potentially doubles as well.

[Love Zhaoying]

23 minutes ago, primerib1 said:

Well, sometimes you can't avoid ending up with hundreds, even thousands of keys.

Like @Lucia Nightfireexplained in this comment to BUG-233195

If LSD space gets doubled, the number of keys to delete potentially doubles as well.

That's fine, I was more referring to the "key size". Big keys will take up more space than smaller keys, I assume.

[Fenix Eldritch]

9 hours ago, primerib1 said:

Some questions regarding the "Protected" variant of LSD KVP:

1. If a KVP is not originally protected, what will happen if I use llLinksetDataRead() on it? Will it fail? Silently or not?
2. Conversely, what will happen if I use llLinksetDataWriteProtected() on an unprotected KVP? Will it fail? Silently or not?
3. Similarly, what will happen if I use llLinksetDataDeleteProtected() on an unprotected KVP? Will it fail? Silently or not?
4. How does one "convert" between Protected and Unprotected variant? Is the only way is to Read-Delete-WriteDifferently ?

1. You are describing the normal use case. llLinksetDataRead will obviously read an unprotected KVP without issue.
2. The protected functions will behave similarly to the standard ones when the password parameter is empty. So you can read/write/delete unprotected keys with the protected functions so long as the password is empty. Else it behaves as if you supplied an incorrect password as per each functions' caveats information.
3. see #2.
4. You don't. You must delete and recreate the entry as protected or unprotected. If you don't have that entry's password, then the only option left for removal is the reset the entire datastore.

Edited March 8, 2023 by Fenix Eldritch

[primerib1]

1 hour ago, Fenix Eldritch said:

You are describing the normal use case. llLinksetDataRead will obviously read an unprotected KVP without issue.

Whoops sorry, I meant what will happen if I use llLinksetDataReadProtected() (with a supplied password) on an unprotected KVP.

[Love Zhaoying]

13 minutes ago, primerib1 said:

Whoops sorry, I meant what will happen if I use llLinksetDataReadProtected() (with a supplied password) on an unprotected KVP.

I hope it either "silently works" (as if you gave the right info), or "fails with error" (Hey! This isn't protected!)

 

[Lucia Nightfire]

3 hours ago, primerib1 said:

Whoops sorry, I meant what will happen if I use llLinksetDataReadProtected() (with a supplied password) on an unprotected KVP.

4 hours ago, Fenix Eldritch said:

The protected functions will behave similarly to the standard ones when the password parameter is empty. So you can read/write/delete unprotected keys with the protected functions so long as the password is empty. Else it behaves as if you supplied an incorrect password as per each functions' caveats information.

[Love Zhaoying]

Hi all, has the llLinksetDataFindKeys() sorting issue been resolved?  It would make my current work much easier.

Edit: I see "https://jira.secondlife.com/browse/BUG-232895" has been "Accepted"..

Thanks,

Love

Edited March 27, 2023 by Love Zhaoying

[Rider Linden]

4 hours ago, Love Zhaoying said:

Hi all, has the llLinksetDataFindKeys() sorting issue been resolved?  It would make my current work much easier.

Edit: I see "https://jira.secondlife.com/browse/BUG-232895" has been "Accepted"..

Thanks,

Love

The fix for that should be on all RC channels after Wednesdays roll.

[Love Zhaoying]

I was trying to research where we are with linking and unlinking.  The wiki only said this: 

"When merging two linksets, both with linkset data, the new root's data is preserved and the simulator makes an effort to move as many of the key value pairs from the old root to the new. If there is insufficient space for all data the simulator will discard some data from the old root. The data selected for preservation may be arbitrary."

Here's my question: If you link a prim WITH LSD with a prim WITHOUT any LSD, then unlink the prims, do both unlinked prims then contain a copy of the original data?  

I did a search on the thread and of course, did not find the answer. 

I guess that I am thinking about leveraging the behavior to "copy" LSD (without using other methods).

[Lucia Nightfire]

1 hour ago, Love Zhaoying said:

I was trying to research where we are with linking and unlinking.  The wiki only said this: 

"When merging two linksets, both with linkset data, the new root's data is preserved and the simulator makes an effort to move as many of the key value pairs from the old root to the new. If there is insufficient space for all data the simulator will discard some data from the old root. The data selected for preservation may be arbitrary."

Here's my question: If you link a prim WITH LSD with a prim WITHOUT any LSD, then unlink the prims, do both unlinked prims then contain a copy of the original data?  

I did a search on the thread and of course, did not find the answer. 

I guess that I am thinking about leveraging the behavior to "copy" LSD (without using other methods).

Only the prim that was once root will ever maintain linkset data.

You could use linking and delinking as a means of moving/populating data without setting up communications.

[Love Zhaoying]

2 minutes ago, Lucia Nightfire said:

You could use linking and delinking as a means of moving/populating data without setting up communications.

Yes, that was my goal.  The "child" ("not the original root") could have a script to delete everything from LSD it doesn't want to keep, after the unlink.  

This seems (on the surface) a lot simpler than other schemes I had been planning.

Thanks! 

[Lucia Nightfire]

1 hour ago, Love Zhaoying said:

Yes, that was my goal.  The "child" ("not the original root") could have a script to delete everything from LSD it doesn't want to keep, after the unlink.  

This seems (on the surface) a lot simpler than other schemes I had been planning.

Thanks! 

If it wasn't the original root, then it shouldn't have any LSD after delink.

[Love Zhaoying]

1 minute ago, Lucia Nightfire said:

If it wasn't the original root, then it shouldn't have any LSD after delink.

Ok, then could you explain this statement?  "You could use linking and delinking as a means of moving/populating data without setting up communications."

 

[Lucia Nightfire]

1 hour ago, Love Zhaoying said:

Ok, then could you explain this statement?  "You could use linking and delinking as a means of moving/populating data without setting up communications."

 

If I had an Object A with no LSD and an Object B with LSD and wanted to "move" the LSD from Object B to Object A, I would link them together making Object A the root, then delink.

[Love Zhaoying]

3 minutes ago, Lucia Nightfire said:

10 minutes ago, Love Zhaoying said:

Ok, then could you explain this statement?  "You could use linking and delinking as a means of moving/populating data without setting up communications."

 

If I had an Object A with no LSD and an Object B with LSD and wanted to "move" the LSD from Object B to Object A, I would link them together making Object A the root, then delink.

Got it, missed the point that Object A would need to "become the root".  Is this determined by which object initiates the link? I don't recall linking objects via LSL script myself yet, just attaching to the HUD via script.

 

[Lucia Nightfire]

1 hour ago, Love Zhaoying said:

Got it, missed the point that Object A would need to "become the root".  Is this determined by which object initiates the link? I don't recall linking objects via LSL script myself yet, just attaching to the HUD via script.

 

Doesn't matter who or what creates/changes the linkset.

If the root is delinked, LSD is going with it.

If a new root is linked to the linkset, LSD will move to it.

[Love Zhaoying]

Saw this post and my basic reaction was, "Well, duh."

 

[Lucia Nightfire]

So llLinksetDataCountFound(string search) & llLinksetDataDeleteFound(string search, string pass) are coming.

You can currently test them in the region Cloud Sandbox 2 on aditi.

Both behave similar to llLinksetDataListKeys() in regards to regex input.

The first returns an integer that is the regex return key count.

Applications are primarily ones that involve pagination.

The second returns a two-integer element list of the number of found keys deleted and the number of found keys not deleted (primarily if the password did not match).

Applications are primarily ones that involve cleanup without the need for repeated calls to llLinksetDataListKeys().

One concern is that llLinksetDataDeleteFound() will trigger the linkset_data() event for EVERY found key deleted.

This has the potential to event flood every script in the linkset that has the linkset_data() event present in state.

I'd like to ask everyone's opinion on what you think would be a better or alternative approach.

IMO, I think a third integer input called event_handling could be offered where an input of 0 would not trigger linkset_data() at all, 1 would trigger linkset_data() every delete, 2 would trigger a generic 'deleted' message, similar to Experience KVP deleting and 3 could return a CSV list of key names (how many, IDK).

What are your opinions?

[Love Zhaoying]

4 minutes ago, Lucia Nightfire said:

So llLinksetDataCountFound(string search) & llLinksetDataDeleteFound(string search, string pass) are coming.

You can currently test them in the region Cloud Sandbox 2 on aditi.

Both behave similar to llLinksetDataListKeys() in regards to regex input.

The first returns an integer that is the regex return key count.

Applications are primarily ones that involve pagination.

The second returns a two-integer element list of the number of found keys deleted and the number of found keys not deleted (primarily if the password did not match).

Applications are primarily ones that involve cleanup without the need for repeated calls to llLinksetDataListKeys().

One concern is that llLinksetDataDeleteFound() will trigger the linkset_data() event for EVERY found key deleted.

This has the potential to event flood every script in the linkset that has the linkset_data() event present in state.

I'd like to ask everyone's opinion on what you think would be a better or alternative approach.

IMO, I think a third integer input called event_handling could be offered where an input of 0 would not trigger linkset_data() at all, 1 would trigger linkset_data() every delete, 2 would trigger a generic 'deleted' message, similar to Experience KVP deleting and 3 could return a CSV list of key names (how many, IDK).

What are your opinions?

Thank you for this! 

Luckily, I don't use the  linkset_data() event, at all. (Maybe someday.)

But, every major script I write still uses LinksetData..

LinksetDataDeleteFound() sounds much better than the loop I have to use now whenever I want to delete a RegEx matched set of keys..granted, that loop becomes a simple function. But, adding that function to every script is silly.

Please post an update when these new functions are "live"!

[Lucia Nightfire]

1 hour ago, Love Zhaoying said:

Thank you for this! 

Luckily, I don't use the  linkset_data() event, at all. (Maybe someday.)

But, every major script I write still uses LinksetData..

LinksetDataDeleteFound() sounds much better than the loop I have to use now whenever I want to delete a RegEx matched set of keys..granted, that loop becomes a simple function. But, adding that function to every script is silly.

Please post an update when these new functions are "live"!

I don't use it either, but I was doing "hopefully" thorough testing and noticed it.

I know there are some that do rely on it, though.

[Quistess Alpha]

I think if someone wants to rely on the linkset_data event to keep track of multiple deleted keys. they should stagger the deletion similar to how a usual notecard read is structured (see code example). I don't think there's a good general case to handle that by back-end convention (a CSV in the linkset_data event argument could be equally problematic for large operations)

string delete_regex = ".some regex.";
default
{   state_entry()
    {   llLinksetDataDelete((string)llLinksetDataFindKeys(delete_regex,0,1));
    }
    linkset_data(integer type, string key,string value)
    {   if(type==LINKSETDATA_DELETE)
        {   // does llLinksetDataDelete(""); generate an event? if it does, do a sanity test here.
            llLinksetDataDelete((string)llLinksetDataFindKeys(delete_regex,0,1));
        }
    }
} // untested.

I'd be happy with llLinksetDataDeleteFound() generating a special kind of linkset_data event with the integer parameter as a new constant LINKSET_DATA_DELETE_MULTIPLE (==3) , and perhaps overloading the name and value parameters with the number of deleted keys and number of found but not deleted respectively (same as list return values, converted to strings). Or some other useful info, like the regex used in the first parameter and the # of keys as a CSV in the second.

Edited July 13, 2023 by Quistess Alpha

[Bleuhazenfurfle]

And this is more of why I was asking for an iterative approach way back early on, or at the very least, handle it more along the lines of the llDetected functions.  (Why they insist on returning lists all the time, given a scripts woeful limited memory…)  I get they don't want scripts tying up simulator memory, but there are better ways, and this way hobbles their ability to do better, as well.

Two alternatives come to mind:

The obvious alternative, of course, is to "page" the deletion — the same way as everything else (sigh).  I suspect the typical approach would be to delete an initial batch (as per your danger threshold), and then delete one more for every event that rolls in, possibly with a timer running in case some went missing at some point.  (Being able to specify a negative count to remove them from the end instead of the start could also be useful … to someone.)

However, since you can already do that right now by doing a search, and then deleting the items you found…  I'm all for llLinksetDataDeleteFound doing a "bulk deletion", with a LINKSET_DATA_BULK_DELETE action that just reports the numbers — and perhaps using a handle (I'd think an integer, llListen style, rather than a key) to link the two.  There is existing precedent for returning several values as CSV, and returning a handle gets around having to pass the regex itself in the event (which won't work with CSV — the next option would be JSON, but that's horrible and a handle should do fine).

Such a bulk delete event could additionally be combined with the llDetected method to allow the script to query the key names (and maybe even the removed values) if needed — saving the greedy conversion of key name strings, if nothing else.  That could also be bolted on later, too.  The combination of paged deletion for when you care (be it manual as it is now, or by paginating the new function), and bulk delete for when you don't (also, a page size of 0, if it goes that way), seem to provide the best coverage to me.

(Pssst, LL, not giving us these options, actually doesn't save you very much, it just hobbles your ability to do it better, and forces us to burn more of that precious memory and script time working around the limitations.)

Edited July 15, 2023 by Bleuhazenfurfle

[Lucia Nightfire]

Apparently, the method with potential for event flood was replaced with a method with potential for memory stack heap.

"I can't decide which is worse." - Tom Petty