"Linkset Data" is coming.

[Coffee Pancake]

2 hours ago, Casper Warden said:

This is an absolutely outstanding feature, and something that has been long overdue. CasperVend and CasperLet will be updated to make use of the feature on launch - no longer any need to store keys in the description of the object

Weak sauce!

Come back when you're storing compressed data in hovertext

[Jenwen Walpole]

 

2 hours ago, Tapple Gao said:

@Rider LindenI moved this batch notification proposal to https://jira.secondlife.com/browse/BUG-232792

Because you can have so many keys, it be nice if you could just enable/disable events on specific keys. 

Also along those lines perhaps the abilty to enable / disable all linkset_data events so that when your doing mass updates to many keys your can disable generating events.

Thoughts?

Edited October 22, 2022 by Jenwen Walpole

[Tapple Gao]

1 hour ago, Jenwen Walpole said:

 

Because you can have so many keys, it be nice if you could just enable/disable events on specific keys. 

Also along those lines perhaps the abilty to enable / disable all linkset_data events so that when your doing mass updates to many keys your can disable generating events.

Thoughts?

the second paragraph is exactly what I wrote a proposal for

The first seems like a lot more work, and people would also want link message filters then

[Jenwen Walpole]

My view is that having an all or noting event structure for keys would be a bit to broad if have 1000's of keys.   Your scipt could spend a lot of time ignoring events it dosnt need and pootentaly shortening your event queue un nessiarly.

i would invision having both the first and second methods, but there use would be for diffrent reasons.  you may need an event handler for just one key or a small number of keys,  Also being able to diable all linkd date events for situations like mass updates to prevent event queue overrun. 

i would agree you would not want to use tirning off single key events in a methodilogy to disable all the key events by disabling them one at a time. 

 

[Qie Niangao]

49 minutes ago, Tapple Gao said:

the second paragraph is exactly what I wrote a proposal for

The first seems like a lot more work, and people would also want link message filters then

Might be my fault for raising the link_message filter topic at the user group; I was reminded of that being a sought-after feature in the past. Although, as I recall folks mostly wanted to be able to subscribe to specific values of the integer num parameter, to avoid spreading scripts around the linkset and targeting links specifically, to reduce the number of scripts waking up for irrelevant messages.

A somewhat apt precedent for linkset_data batching is llListenControl() that can suspend receipt (not transmission) of all listen events on a channel (and lacking any analog of the handy LINKSETDATA_BATCH_UPDATE count).

Thinking about it more, there's specific virtue in the batching approach operating from the transmitting script, especially when there are multiple recipient scripts that could be flooded by an operation in the transmitter. It's the transmitter that knows when it's about to perform such an operation.

(That said though, I anticipate only rarely handling the linkset_data message at all, favoring a link_message with more specifics from the sender about where to find changes in the linkset data store.)

[Tapple Gao]

I made another minor feature request for linkset_data event, to not fire it if a value is updated redundantly: https://jira.secondlife.com/browse/BUG-232795

[Anna Salyx]

One thing I noticed while testing and I'm not sure if it's a big enough deal to worry with but....

Quote

Function: integer llLinksetDataWrite( string name, string value );
Creates or updates a key/value pair in the linksets datastore.
Returns an integer indicating success or failure. 0 is success, a positive number indicates failure.

Caveats

If value is an empty string, the key is deleted from the linkset data.

This makes testing for success counter intuitive.  That is:

 if (llLinksetDataWrite("SomeKey", "SomeValue")
 { 
	llOwnerSay("Data successfully written");
 } else {
	llOwenerSay("Error, data not written");
 }

Will cause the script to respond with the error/failure block even though the data was properly written to the key store. This is the reverse of what one would logically expect out of the if structure.

Likewise, if an empty value is given and the key deleted this also returns a 0 success which evaluates in the "if" statement as "false, error, didn't work."

 

 

[Nika Talaj]

I just stumbled across this in the LSL portal, had no idea it was coming so soon.  I haven't read this whole fascinating thread, but have a stupid question: is there any indication in the build menu that an object is already using LSD?  

I see it as a quick way to warn amateur scripters/builders like myself that they might want to take a a little care re-linking or adding more LSD usage.  [Yes, I know, one could always dump in a little script to check available space.]

And, congrats Rider et. al. on a terrific addition to LSL!

[Jenwen Walpole]

@Rider Linden

@Tapple Gao

and anyone who is interested

i've had a think on the whole linkset_data event overflow situation on mass updates..

what i think would be an easier to implement and more flexable soltuon is to have write and delete fuctions that do not generate events. so you can simply use those fuction when nessisary and still have events when you want / need them.

 

SUMMARY.

Becasue linkset_data event is for any keys updated or deleted. unwanted or events not needed can be generated. in example, if updating a large nubmer of keys at once you could over run the event queue. or if you only need events for specific keys,  So there are times when you want to update or delete keys but do not want to generate events. 
the llLinksetDataWriteNE, llLinksetDataWriteProtectedNE, llLinksetDataDeleteNE, llLinksetDataDeleteProtectedNE functions work the same as llLinksetDataWrite, llLinksetDataWriteProtected, llLinksetDataDelete, llLinksetDataDeleteProtected except they would NOT generate events.

Function: integer llLinksetDataWriteNE( string name, string value );

Function: integer llLinksetDataWriteProtectedNE( string name, string value, string pass );

Function: llLinksetDataDeleteNE( string name );

Function: llLinksetDataDeleteProtectedNE( string name, string pass );

 

Thoughts?

 

(edits were typo fixes)

 

 

Edited October 23, 2022 by Jenwen Walpole

[Kadah Coba]

On 10/22/2022 at 10:22 AM, Jenwen Walpole said:

Because you can have so many keys, it be nice if you could just enable/disable events on specific keys. 

Also along those lines perhaps the abilty to enable / disable all linkset_data events so that when your doing mass updates to many keys your can disable generating events.

Thoughts?

I was actually thinking similarly. But instead of the mess of duplicate functional, maybe a flag param, which would also allow for additional functionality in the future.

On 10/22/2022 at 1:09 PM, Qie Niangao said:

Although, as I recall folks mostly wanted to be able to subscribe to specific values of the integer num parameter, to avoid spreading scripts around the linkset and targeting links specifically, to reduce the number of scripts waking up for irrelevant messages.

That would be extremely nice to have. If num could be a range or better, a bitmask, that would be cool. Gimme gimme~

On 10/22/2022 at 1:09 PM, Qie Niangao said:

(That said though, I anticipate only rarely handling the linkset_data message at all, favoring a link_message with more specifics from the sender about where to find changes in the linkset data store.)

Mostly ditto here. I don't see using the LSD events much, but far less likely if all scripts using it are going to every LSD event.

4 hours ago, Nika Talaj said:

is there any indication in the build menu that an object is already using LSD?  

I see it as a quick way to warn amateur scripters/builders like myself that they might want to take a a little care re-linking or adding more LSD usage.  [Yes, I know, one could always dump in a little script to check available space.]

Forget the "amateur" part, you'd have to already know the object has an LSD store or explicitly check for it via a script.

I second this as a "would be nice to have". Could be added conditionally to the contextual selection_count field that shows num objects selected, land impact, etc. Could display something like "X LSD keys in Y objects" where X is the sum across the selection if >0.

13 minutes ago, Jenwen Walpole said:

what i think would be an easier to implement and more flexable soltuon is to have write and delete fuctions that do not generate events. so you can simply use those fuction when nessisary and still have events when you want / need them.

 

[Wulfie Reanimator]

5 hours ago, Anna Salyx said:

One thing I noticed while testing and I'm not sure if it's a big enough deal to worry with but....

This makes testing for success counter intuitive.  That is:

 if (llLinksetDataWrite("SomeKey", "SomeValue")
 { 
	llOwnerSay("Data successfully written");
 } else {
	llOwenerSay("Error, data not written");
 }

Will cause the script to respond with the error/failure block even though the data was properly written to the key store. This is the reverse of what one would logically expect out of the if structure.

Likewise, if an empty value is given and the key deleted this also returns a 0 success which evaluates in the "if" statement as "false, error, didn't work."

"Zero on success" is common practice in software development when there can be multiple reasons for failure and a more specific reason is wanted. Return codes from programs work this way under Windows/Linux/Mac too.

59 minutes ago, Jenwen Walpole said:

Thoughts?

There's already a feature request about it. https://jira.secondlife.com/browse/BUG-232792

Edited October 23, 2022 by Wulfie Reanimator

[Jenwen Walpole]

22 minutes ago, Kadah Coba said:

I was actually thinking similarly. But instead of the mess of duplicate functional, maybe a flag param, which would also allow for additional functionality in the future.

i had a re-think on this, see the proposed  llLinksetDataWriteNE, llLinksetDataWriteProtectedNE, llLinksetDataDeleteNE, llLinksetDataDeleteProtectedNE fuctions.

the reson i am thinking this is better is that it likely saves a read of the flag which is likely stored with the key and would ahve overhead. alternatly it could be a flag on the fuction to indicte generating and event or not. like

Function: integer llLinksetDataWrite( string name, string value, integer flag );

but that would be hard to implement later, and would need to be done before release otherwise people ahve to re-write code.  the NE fuctions i proposed above could be added now or later without much fuss.

 

[Anna Salyx]

I get that.  And had there been any indicator that there would be significance in a return code other than zero I'd not have raised this as a question.  But as it stands right now, the way the wiki entry is written, it's going to just simply be a binary success/failure condition. So why not make it so that it's a logically testable binary success/failure? IE: return TRUE if data written/stored successfully.

If there are plans in the pipe to give us a range of return codes to test against on failure, so that the return can be captured and acted on, then certainly I withdraw my question about and talking point about it.

edit:( and yes, I realize there can be an inline (if <function> == 0) to do the same, I'm just looking for simplified elegance is all.  which is also why I prefaced my initial comments with if it was a big deal or not).

 

edit edit:

Okay, I had forgotten, since they are not on the WIKI yet, that there are the Protected versions of the  Write/Read/Delete commands that can cause extended, codes to be returned on failure that can be checked against for reasons. ("LINKSETDATA_EPROTECTED", etc). 

So.... well.  Nevermind. 

Apologies for adding useless noise to the signal.

Edited October 24, 2022 by Anna Salyx

[Jenwen Walpole]

54 minutes ago, Wulfie Reanimator said:

There's already a feature request about it. https://jira.secondlife.com/browse/BUG-232792

added it as an aternate proposal to the jara

[Quistess Alpha]

https://jira.secondlife.com/browse/BUG-232800

In case nobody else thought about it (I can't pay exact attention to 10 pages, sorry), What about making the value parameter a list?

[Fenix Eldritch]

The regex portion of llLinksetDataFindKeys wiki page has been updated and in trying some of the examples, I'm confused with what I'm seeing...

default
{
    state_entry()
    {
        llLinksetDataReset();
        llLinksetDataWrite("AAACC","testdata");
        llLinksetDataWrite("A0A0C","testdata");
        llLinksetDataWrite("AAC","testdata");
        llLinksetDataWrite("ABC","testdata");
        llLinksetDataWrite("ACC","testdata");
        llLinksetDataWrite("ADC","testdata");
        llLinksetDataWrite("A0C","testdata");
        llLinksetDataWrite("A9C","testdata");
        llLinksetDataWrite("AaC","testdata");
        llLinksetDataWrite("AbC","testdata");
        llLinksetDataWrite("AcC","testdata");
        llLinksetDataWrite("AdC","testdata");
        llLinksetDataWrite("A[C","testdata");
        
        string regex = "A[[:digit:]]C";
        llOwnerSay("regex result: '"+llList2CSV(llLinksetDataFindKeys(regex, 0, 10))+"'"); 
    }
}

Firstly, are the shorthand for the character classes supposed to work? I mean the \d, \l, \u, \s, \w, etc.. I am able to get their named versions to work :decimal:, :lower:, :upper:, :space:, etc but not the shorthands... Also, why is it that I must enclose the named versions in double braces?

Secondly, I would have expected the regex of "A[[:digit]]C" would just return the two keys with one digit between A and C : "A0C" and "A9C". But it also captures "A0A0C". Why is that? Doesn't the [] thingy only match a single character? Does it something to do with the double [[]]?

Edited October 24, 2022 by Fenix Eldritch
typo

[Love Zhaoying]

"RegEx is stinky doodoo".  You can quote me on that.

[Love Zhaoying]

3 minutes ago, Fenix Eldritch said:

The regex portion of llLinksetDataFindKeys wiki page has been updated and in trying some of the examples, I'm confused with what I'm seeing...

default
{
    state_entry()
    {
        llLinksetDataReset();
        llLinksetDataWrite("AAACC","testdata");
        llLinksetDataWrite("A0A0C","testdata");
        llLinksetDataWrite("AAC","testdata");
        llLinksetDataWrite("ABC","testdata");
        llLinksetDataWrite("ACC","testdata");
        llLinksetDataWrite("ADC","testdata");
        llLinksetDataWrite("A0C","testdata");
        llLinksetDataWrite("A9C","testdata");
        llLinksetDataWrite("AaC","testdata");
        llLinksetDataWrite("AbC","testdata");
        llLinksetDataWrite("AcC","testdata");
        llLinksetDataWrite("AdC","testdata");
        llLinksetDataWrite("A[C","testdata");
        
        string regex = "A[[:digit:]]C";
        llOwnerSay("regex result: '"+llList2CSV(llLinksetDataFindKeys(regex, 0, 10))+"'"); 
    }
}

Firstly, are the shorthand for the character classes supposed to work? I mean the \d, \l, \u, \s, \w, etc.. I am able to get their named versions to work :decimal:, :lower:, :upper:, :space:, etc but not the shorthands... Also, why is it that I must enclose the named versions in double braces?

Secondly, I would have expected the regex of "A[[:digit]]" would just return the two keys with one digit between A and C : "A0C" and "A9C". But it also captures "A0A0C". Why is that? Doesn't the [] thingy only match a single character? Does it something to do with the double [[]]?

I hate to ask, since I am not participating in the testing, but apparently I have no filter.

Did you happen to try your use-cases on a "known good" standard RegEx testing platform, such as "https://www.regextester.com/"? In my experience, testing RegEx on another platform is a good idea when you are having unexpected behavior.  

[Fenix Eldritch]

Regextester doesn't even recognize the [[:digit:]] format, so it's already different from SL's implementation.

I may have figured out why "A[[:digit:]]C" was capturing what I thought was additional hits. It's matching the A0C at the end of "A0A0C". This is surprising to me because I thought the regex is only matching strings of 3 characters in total length. If I try the regex of "A.C" it only finds the keys A[C, AdC, AcC, AbC, A0C, ADC, AaC, ACC, ABC, AAC as I would expect.

... but that same "A.C" regex also finds "A0A0C" in regextester. So why is it seemingly returning different results in SL?

[Love Zhaoying]

32 minutes ago, Fenix Eldritch said:

Regextester doesn't even recognize the [[:digit:]] format, so it's already different from SL's implementation.

I may have figured out why "A[[:digit:]]C" was capturing what I thought was additional hits. It's matching the A0C at the end of "A0A0C". This is surprising to me because I thought the regex is only matching strings of 3 characters in total length. If I try the regex of "A.C" it only finds the keys A[C, AdC, AcC, AbC, A0C, ADC, AaC, ACC, ABC, AAC as I would expect.

... but that same "A.C" regex also finds "A0A0C" in regextester. So why is it seemingly returning different results in SL?

Without understanding the SL implementation of RegEx (I missed the main posts about it awhile back), I assume one or more of these are true:

1. The extra "[]" in the SL methods are required for whatever juju magick the SL gurus used to get a RegEx implementation.

2. This "[]" and other introductions / differences are making SL RegEx behave differently than "standard" RegEx.

3. The specific use-case you seek doesn't work in the SL version.

4. The "match end" part is probably working as expected / or not, RegEx is stinky doodoo.

It sounds on the surface like testing RegEx stuffs is a moving target if you don't have a baseline to compare it with..

My own dim memory was that "[ ]" was used for "sets".. I don't recall if "[::digit:]" makes sense at all (like, what kind of constant is "::digit::"?!?!?) or if it should really be "[0123456789]". Unless "::digit::" is an SL juju magick version of the RegEx constants. Inconsistent constants cause incontinence on many continents.

And the extra surrounding "[ ]"?  Now I am just sad and want to get large "WHY" tattoos. Because RegEx.

 

 

[Rider Linden]

2 hours ago, Fenix Eldritch said:

Secondly, I would have expected the regex of "A[[:digit]]C" would just return the two keys with one digit between A and C : "A0C" and "A9C". But it also captures "A0A0C". Why is that? Doesn't the [] thingy only match a single character? Does it something to do with the double [[]]?

"A[[:digit:]]C" matches "A0A0C".  To only match the 3 character ones try: "^A[[:digit:]]C$" (^ == beginning and $ = ending)

When I'm working with regexs I make heavy use of https://regex101.com/ (because, yes, regex is voodoo, but it is very very powerful voodoo.)

[Wulfie Reanimator]

9 hours ago, Fenix Eldritch said:

Firstly, are the shorthand for the character classes supposed to work? I mean the \d, \l, \u, \s, \w, etc.. I am able to get their named versions to work :decimal:, :lower:, :upper:, :space:, etc but not the shorthands...

'\' is an escape character in LSL strings. If you have '\u' in a string, the backslash is trying to escape the 'u' and doesn't end up in the final string.

To use the shorthands, you have to write them out as "\\u". (The first '\' escapes the second, and the pair becomes a single literal character for the string.)

9 hours ago, Fenix Eldritch said:

Also, why is it that I must enclose the named versions in double braces?

Single brackets have a specific meaning in regex. "[abc]" is a set of characters 'a', 'b', and 'c', and any one of them will be matched.

LSL is using a POSIX-compliant parser (not necessarily PCRE2 as I originally thought), you can read about the specification here. It's actually more interesting than I thought! *

The RegexTester website has two standards you can choose from; ECMAScript (default) and PCRE. There are more implementations, another site I like to use is Regex101.

9 hours ago, Fenix Eldritch said:

Regextester doesn't even recognize the [[:digit:]] format, so it's already different from SL's implementation.

I may have figured out why "A[[:digit:]]C" was capturing what I thought was additional hits. It's matching the A0C at the end of "A0A0C". This is surprising to me because I thought the regex is only matching strings of 3 characters in total length. If I try the regex of "A.C" it only finds the keys A[C, AdC, AcC, AbC, A0C, ADC, AaC, ACC, ABC, AAC as I would expect.

... but that same "A.C" regex also finds "A0A0C" in regextester. So why is it seemingly returning different results in SL?

"A[[:digit:]]C" finds the same (subset) of keys as "A.C". They both match "A0A0C".

@Quistess Alpha pointed out earlier in this thread that keys were not found based on substring matches. The implementation has changed and now "A" matches "CAT". This is a good change, and you can still match "only strings of 3 characters in total length" if you wrap your pattern as "^A.C$".🙂 ('^' meaning line start, and '$' meaning line end.)

 

 

-------------------

* To ramble about the double bracket thing a bit more... from the PCRE2 specification:

Quote

There are two different sets of metacharacters: those that are recognized anywhere in the pattern except within square brackets, and those that are recognized within square brackets.

Part of a pattern that is in square brackets is called a "character class".

So the first '[' symbol begins a character-class ("a set") definition.
The second '[' symbol begins a POSIX character-class (if followed by POSIX syntax "[:" and ":]").

Quote

Within a bracket expression, the name of a character class enclosed in "[:" and ":]" stands for the list of all characters belonging to that class.

So that's why they're usually wrapped in double brackets. POSIX just defined it that way.

But interestingly, you can use the POSIX syntax on its own in LSL, outside of any other brackets. We can even use the POSIX "collating elements" as described on that manual page (which would NOT work in PCRE2!):

Quote

Within a bracket expression, a collating element (a character, a multicharacter sequence that collates as if it were a single character, or a collating-sequence name for either) enclosed in "[." and ".]" stands for the sequence of characters of that collating element.

The sequence is a single element of the bracket expression's list.

A bracket expression containing a multicharacter collating element can thus match more than one character, for example, if the collating sequence includes a "ch" collating element, then the RE "[[.ch.]]*c" matches the first five characters of "chchcc".

Fancy.

Edited October 24, 2022 by Wulfie Reanimator

[Rider Linden]

I'll have to check out regextester.

Here is @Fenix Eldritch's search at RegEx101:

https://regex101.com/r/ameaZx/1

[Fenix Eldritch]

Oh now I feel silly. It didn't even occur to me to escape the shorhands themselves - but sure enough that works for me now. And I also completely missed the parser switcher button regextester. Setting that as suggested now recognizes the aliases. I'll add regex101 to my list of tools too.

Thank you for explaining this to me Wilfie & Rider!

32 minutes ago, Wulfie Reanimator said:

Quistess pointed out earlier in this thread that keys were not found based on substring matches. The implementation has changed and now "A" matches "CAT".

Ahhh ok. I wasn't aware of that change so my expectations were based on outdated observations. Thanks!

[Rider Linden]

I'm going to bring this up at tomorrow's user group, but I wanted to throw it out for discussion.

Right now if you update a key-value pair with an identical value ("foo:bar" and you reassign the value "bar" to the key "foo"), or when you delete a non existent key I will fire the respective event.  The logic there is that the operation was successful, even if trivially so. "bar" is now the value of "foo" and the key you deleted is not in the store.

The other school of thought is that the event should indicate a change in the KVP store. No key in the database? then no event on a delete.  If the value at a key is the same before and after then the KVP store has not changed so do not fire an event.

Which of those two would people prefer?  

1. linkset_data is thrown even on a "trivially" successful operation. 
2. linkset_data is thrown only when there is a change to KVP data.