LSL-only, grid-wide communication

[Wulfie Reanimator]

It's that complicated problem again.

Objects in SL can request to have an URL assigned to them for HTTP communication. Objects can also send HTTP messages. This makes it possible for objects to talk to each other using HTTP, grid-wide, bypassing region restrictions.

But there's a problem. Almost any interruption to the script's existence causes the requested URL to die. Script reset, rezzing, region change, and region restart will all cause all URLs for the object to be released.

One solution to this is to use an external service, which is definitely the most reliable way but not the only one. What about solutions that are completely in-world and LSL-based?

I've deliberately gone out of my way to not read up on it before trying to come up (and creating) my own solution. Now that I've done that, I'd like to ask for feedback and suggestions on it. Some "plain English" first though:

This is for two objects maintaining a bridge between each other, not a multi-object network. Both objects use identical scripts, so there is no dedicated "server" or "receiver." Though, since this script only has the bridging code, it can very easily be changed into something more asymmetrical. (Like sending commands from A to B, but B will only act on commands and not send them.)

How it works:

1. On state_entry, on_rez, and changed, the object requests for a new URL.
2. If the URL is given, it is saved into memory. (Because you cannot recall it otherwise.)
3. The object will then check if it has a stored URL for the other object.
   
   If it doesn't (usually the first time the script runs), it will only send its URL to an avatar to handle the linking by hand. This is done by going to the other object and saying the link on a chat channel.
   
   If it does, it will send its own URL to the stored URL (other object) before also sending the URL to an avatar. (The latter is currently just a safety-measure.)
    
4. From the other object's perspective now, when it receives a HTTP message with an object URL, it will store that URL as an invisible hovertext. This is done for relatively safe persistence in case the object is reset (go back to 1).

This works and the objects will be able to update each other with new URLs when one of them has to get a new URL for any reason.

You can test this by placing the script into an object on the ground, copying its URL from chat, placing another copy of the script in an attachment while in another region, and giving it the first object's URL. Teleporting between sims will show new URLs being requested for your attachment and sent to the first object, which sends you a message (via llInstantMessage) confirming that it has stored it.

One problem that puzzles me though is.. if there's any way at all for the objects to create a bridge by themselves if they have both gained a new URL and failed to communicate it to the other object. One case where this happens is when both regions are down at the same time, or if both objects are worn as attachments and the avatar(s) log off.

One solution I'm thinking about adding is a confirmation that the URL was actually received, similar to TCP packets. This way, if the URL wasn't received because the other object was offline, it can be sent again after some time. But there is no way to test if the other object's URL has changed, meaning that it can never receive the new URL.

For the sake of a catch-all test-case, let's say that these objects will never be on the same region at the same time and are guaranteed to break their bridge regularly because both objects will go "offline" simultaneously for some indeterminate period of time.

key avatar = "please-use-your-own-uuid";

integer chatChannel = 445;

key urlRequest;
list urlParams = [HTTP_METHOD, "POST"];
string urlPrefix = "http://sim";

string thisObjectUrl;

GetNewUrl()
{
    llOwnerSay("requesting new url");

    if(llGetFreeURLs())
        urlRequest = llRequestURL();
    else
        llOwnerSay("no free urls"); // todo: retry later
}

LinkWithExternalUrl()
{
    llOwnerSay("checking for stored url");

    string url = llList2String(llGetPrimitiveParams([PRIM_TEXT]), 0);
    if(llGetSubString(url, 0, 9) == urlPrefix)
        llHTTPRequest(url, urlParams, thisObjectUrl);
}

default
{
    state_entry()
    {
        GetNewUrl();

        // manual external url override
        llListen(chatChannel, "", "", "");
    }

    changed(integer change)
    {
        if(change & CHANGED_REGION
        || change & CHANGED_REGION_START)
        {
            GetNewUrl();
        }
    }

    on_rez(integer param)
    {
        GetNewUrl();
    }

    http_request(key id, string method, string body)
    {
        if(method == URL_REQUEST_GRANTED)
        {
            thisObjectUrl = body;

            // check for stored url, send if possible
            LinkWithExternalUrl();

            // send own url to avatar
            llInstantMessage(avatar, "new url: " + thisObjectUrl);
        }
        else if(llGetSubString(body, 0, 9) == urlPrefix)
        {
            // store remote url 
            llSetPrimitiveParams([PRIM_TEXT, body, ZERO_VECTOR, 0]);
            llInstantMessage(avatar, "new url stored");
        }
        else if(method == URL_REQUEST_DENIED)
        {
            llOwnerSay("URL_REQUEST_DENIED");
            // todo: retry later
        }
        else
        {
            // undefined
            llOwnerSay(llList2CSV([
                "id", id,
                "method", method,
                "body", body])
            );
        }
    }

    listen(integer c, string n, key id, string m)
    {
        if(llGetOwnerKey(id) != avatar) return;

        if(llGetSubString(m, 0, 9) == urlPrefix)
        {
            llSetPrimitiveParams([PRIM_TEXT, m, ZERO_VECTOR, 0]);
            LinkWithExternalUrl();
        }
    }
}

It should be apparent that this script is still a kind of proof-of-concept and in-progress. It does not handle denied URL requests or retry anything, for example.

Edited January 8, 2019 by Wulfie Reanimator

[Wandering Soulstar]

Trying to think this through, and about to board a flight, so will have more thinking time, but for now, without introducing a third 'server' prim, that does not go offline, I cannot figure out the solution to them both going offline at the same time and changing URL at the same time.

You could have third object, just used as a dumb storage system/registry ... though have to think through how to maintain constant comms there as well .. very interesting exercise you have given @Wulfie Reanimator

[KT Kingsley]

An experience key value pair will do this on land where that experience is enabled. Hopefully, sometime soon, when grid-wide experiences come along, it'll work anywhere unless that experience is specifically disabled.

Most of the objects I'm using this for are static and on one of my own parcels. My HUD goes with me, so if it loses touch (crashes or gets reset) I have to TP to one of those parcels to read the URL and reestablish the connection. No big deal, but I am really looking forward to grid-wide experiences, and hoping that access to key value pairs anywhere will be part of the deal.

Edited January 8, 2019 by KT Kingsley

[Wulfie Reanimator]

7 minutes ago, Wandering Soulstar said:

Trying to think this through, and about to board a flight, so will have more thinking time, but for now, without introducing a third 'server' prim, that does not go offline, I cannot figure out the solution to them both going offline at the same time and changing URL at the same time. 

You could have third object, just used as a dumb storage system/registry ... though have to think through how to maintain constant comms there as well .. very interesting exercise you have given @Wulfie Reanimator

This is what is generally called "redundancy." Basically just additional systems that aren't necessary until something breaks.

That said, even if you add a third object to work as a "middle man" or the dedicated server, that can go offline as well. Every region on SL restarts at some point, and rolling restarts can take out every sim with a redundant copy of the script, or even the whole grid can go down. Then what? You have to go to every copy and share the new URLs (even if you had a "circular system" where you only need to give one working URL to each object and they'll get every other URL from each other)

2 minutes ago, KT Kingsley said:

An experience key value pair will do this on land where that experience is enabled. Hopefully, sometime soon, when grid-wide experiences come along, it'll work anywhere unless that experience is specifically disabled.

Most of the objects I'm using this for are static and on one of my own parcels. My HUD goes with me, so if it loses touch (crashes or gets reset) I have to TP to one of those parcels to read the URL and reestablish the connection. No big deal, but I am really looking forward to grid-wide experiences, and hoping that access to key value pairs anywhere will be part of the deal.

I count Experiences as external services, mainly because you must have a sim dedicated to it. I might as well host my own simple web-server with nothing but PHP and a text file, but not everybody knows how to set that up, let alone those intimidating server installs. Currently, I've been using and testing this method by having one copy in my parcel and the other "moving around" on the grid.

[Qie Niangao]

28 minutes ago, Wulfie Reanimator said:

I count Experiences as external services, mainly because you must have a sim dedicated to it.

It doesn't need a dedicated sim at all, but indeed senders and receivers need to be on parcels that enable the Experience. This is how my various Traffic maps work. (They don't actually use HTTP between maps and telemetry servers because it's cheaper to just "broadcast" updates directly in the Experience persistent store, using it sorta like a shared-memory-based message queue.) But this wouldn't work for copies of your object that, as you say, move around the grid, except if they can reliably cross into parcels that enable the Experience... at least until we get the long-promised "grid-scope" Experiences.

For years and years we used llEmail to send updated URLs to fixed-UUID "registry servers" that could share those new URLs with interested parties. That centralized architecture isn't required of course; peer-to-peer updates would work just the same -- but with the same constraint: the peers would need to maintain the same UUID, whereas static UUID servers can support clients that come and go. (Even earlier, before LSL had http services at all, we'd use llEmail to do all the inter-sim communications, which was really painful.) I don't recommend this, though, because llEmail is still subject to a "stuck queue" condition that's rare and unpredictable enough that I've abandoned hope of it ever being debugged and fixed.

[chibiusa Ling]

3 hours ago, Wulfie Reanimator said:

Your stuffs....

Thats very interesting, and whilst not exactly the same as it uses an external service it reminds me of this that I found a while back when I was playing around with ideas for in world alternatives to databases or experience keys 

http://wiki.secondlife.com/wiki/Permanent_Primitive_URL

Edited January 8, 2019 by chibiusa Ling

[SeanMcDonald]

It's an external service, but what about using a Google spreadsheet to pass information among the objects?  Fixed URL, and so long as the script understands what it's reading, this should give grid-wide persistence to the data.

[Wulfie Reanimator]

16 minutes ago, SeanMcDonald said:

It's an external service, but what about using a Google spreadsheet to pass information among the objects?  Fixed URL, and so long as the script understands what it's reading, this should give grid-wide persistence to the data.

If it uses any kind of service outside of SL, it's out of scope for this discussion. That's absolutely the way to do it for any kind of real project/product, but the whole point is "what can I do besides that?" I realize that the nature of objects in-world means that there's no absolute way to keep cross-grid communications open, especially if the whole grid goes down, but that's an acceptable reality. I still want to brainstorm ways to maintain -- even if 1-on-1 -- a system like this as reliably as possible, assuming that the whole grid doesn't die.

A grid-wide, per-owner Experience or at least limited persistent storage would be the dream that would get me to pay for Premium on its own, but until then...

34 minutes ago, chibiusa Ling said:

Thats very interesting, and whilst not exactly the same as it uses an external service it reminds me of this that I found a while back when I was playing around with ideas for in world alternatives to databases or experience keys 

http://wiki.secondlife.com/wiki/Permanent_Primitive_URL

This is an interestingly clever solution even if it's not the solution, by the way.

Edited January 8, 2019 by Wulfie Reanimator
Forgot to quote Chib.

[KT Kingsley]

1 hour ago, Wulfie Reanimator said:

[...]

I count Experiences as external services, mainly because you must have a sim dedicated to it. I might as well host my own simple web-server with nothing but PHP and a text file, but not everybody knows how to set that up, let alone those intimidating server installs. Currently, I've been using and testing this method by having one copy in my parcel and the other "moving around" on the grid.

No, you don't need a dedicated sim. You do need to be a premium member to create an experience, and you do need to have the parcels where you use it allow it specifically (at least until grid-wide scope becomes a thing).

[Wulfie Reanimator]

25 minutes ago, KT Kingsley said:

No, you don't need a dedicated sim. You do need to be a premium member to create an experience, and you do need to have the parcels where you use it allow it specifically (at least until grid-wide scope becomes a thing).

I said "sim" but what I meant was "a piece of land on a sim." Slightly different, but hopefully you get the idea. (Unless the Experience database can be accessed outside of the land where it's enabled?) Experiences are a no-go because there's no guarantee at all that the object(s) will be on (or return to) Experience land. Even if you could guarantee that one of the objects will be on Experience land 100% of the time, it would be useless as the other object would have no Experience access.

That, and not everybody has an Experience. I don't want a solution that doesn't work for everybody, as in anybody without an Experience or access to any specific external service.

Edited January 8, 2019 by Wulfie Reanimator

[Innula Zenovka]

Restarting the region breaks the old URL.  What happens during rolling restarts?   If both regions restart simultaneously,  then when the regions come back up, do you have to reset the URLs manually?

[Qie Niangao]

1 hour ago, Wulfie Reanimator said:

That, and not everybody has an Experience. I don't want a solution that doesn't work for everybody, as in anybody without an Experience or access to any specific external service.

It's not necessary to have an Experience to be able to use an Experience. Take for example the AVsitter Experience that's used by (presumably) thousands of different owners who are barely aware it exists other than needing to enable it on their land. A generous Experience owner might well donate some Experience access to a noble cause (as long as they can be reasonably sure it won't be abused by a DDoS or something that gets their Experience terminated by the Lab). But of course that Experience owner must keep paying their Premium fees or the whole carriage turns into a pumpkin. And, unless it's one of those promised grid-scope Experiences, it won't meet these requirements anyway.

The only thing I can imagine working within these constraints is a large number of redundant, peer-to-peer http communicators that can sort of "self-heal" their little network when some subset of them goes down and can only reestablish communications because it knows enough other nodes that some will still be at their last-known URLs.

[steph Arnott]

Try using another form of com to re-establish the primary.

[Wandering Soulstar]

To Steph's point .. could you not use llEMail to update the URL should both be in regions that reset at the same time?

[steph Arnott]

1 minute ago, Wandering Soulstar said:

To Steph's point .. could you not use llEMail to update the URL should both be in regions that reset at the same time?

Yes, which was why i mentioned another form to kick it.

[Innula Zenovka]

5 hours ago, Qie Niangao said:

But of course that Experience owner must keep paying their Premium fees or the whole carriage turns into a pumpkin.

Are we sure about this?   I think I remember reading somewhere that, if the Experience owner stops being a premium member, existing scripts set to that experience keep on running but no new ones can be compiled using the experience, though my memory is by no means infallible, of course, and -- shockingly -- not everything I read online about SL turns out to be true.

It would be good to have some sort of official statement of what happens in these circumstances -- preferably in the wiki, I suppose.   

[steph Arnott]

5 minutes ago, Innula Zenovka said:

Are we sure about this?   I think I remember reading somewhere that, if the Experience owner stops being a premium member, existing scripts set to that experience keep on running but no new ones can be compiled using the experience, though my memory is by no means infallible, of course, and -- shockingly -- not everything I read online about SL turns out to be true.

It would be good to have some sort of official statement of what happens in these circumstances -- preferably in the wiki, I suppose.   

Do not know but a friend who was premium and did experiense stated that there is a grace period then it is removed.

[Wulfie Reanimator]

10 hours ago, Innula Zenovka said:

Restarting the region breaks the old URL.  What happens during rolling restarts?   If both regions restart simultaneously,  then when the regions come back up, do you have to reset the URLs manually?

Yes. If both objects go "offline" at the same time, when one of them returns, it tries to send its new URL to the other object's URL. The other object isn't there to receive it though, and even when the second of the two objects comes back, the stored URL is dead with no object to receive it.

 

10 hours ago, Qie Niangao said:

It's not necessary to have an Experience to be able to use an Experience. [...]

The only thing I can imagine working within these constraints is a large number of redundant, peer-to-peer http communicators that can sort of "self-heal" their little network when some subset of them goes down and can only reestablish communications because it knows enough other nodes that some will still be at their last-known URLs.

When I said "have an experience," I meant "have access to (anyone's) Experience..."

Redundancy may be the only realistic way that I can think of, but there's a a kind of critical mass of how many you'd want to use, because if and when the entire network does reach an unrecoverable state, someone must go to every single "node" to link it to some other node.

 

4 hours ago, Wandering Soulstar said:

To Steph's point .. could you not use llEMail to update the URL should both be in regions that reset at the same time?

llEmail is almost ideal, because it can send an email directly to an object UUID, but.. The UUID of an in-world object changes every time it is rezzed (e.g. worn) and after sim restarts, so that's the same issue and not any more reliable than sending URLs back and forth. You'd have to send UUIDs back and forth too. There are no other types of cross-region communication in LSL besides Email and HTTP.

Edited January 9, 2019 by Wulfie Reanimator

[steph Arnott]

3 minutes ago, Wulfie Reanimator said:

Yes. If both objects go "offline" at the same time, when one of them returns, it tries to send its new URL to the other object's URL. The other object isn't there to receive it though, and even when the second of the two objects comes back, the stored URL is dead with no object to receive it.

 

When I said "have an experience," I meant "have access to (anyone's) Experience..."

Redundancy may be the only realistic way that I can think of, but there's a a kind of critical mass of how many you'd want to use, because if and when the entire network does reach an unrecoverable state, someone must go to every single "node" to link it to some other node.

 

llEmail is almost ideal, because it can send an email directly to an object UUID, but.. The UUID of an in-world object changes every time it is rezzed (e.g. worn) and after sim restarts, so that's not any more reliable than sending URLs back and forth. You'd have to send UUIDs back and forth too. There is no other types of cross-region communication in LSL besides Email and HTTP.

The email can return that key at regular intervals.

[Wandering Soulstar]

-- misquoted --

Edited January 9, 2019 by Wandering Soulstar

[Wandering Soulstar]

4 hours ago, Wulfie Reanimator said:

The UUID of an in-world object changes every time it is rezzed (e.g. worn) and after sim restarts,

Forgot about that 😞

 

 

[steph Arnott]

8 hours ago, Wandering Soulstar said:

Forgot about that 😞

 

 

It can be done and has been done. Issue is that it is too unreliable.

[Qie Niangao]

13 hours ago, Wulfie Reanimator said:

llEmail is almost ideal, because it can send an email directly to an object UUID, but.. The UUID of an in-world object changes every time it is rezzed (e.g. worn) and after sim restarts, so that's the same issue and not any more reliable than sending URLs back and forth. You'd have to send UUIDs back and forth too. There are no other types of cross-region communication in LSL besides Email and HTTP.

In this application where the objects themselves are ephemeral, llEmail offers no benefit over HTTP: In both cases, you'd need enough redundant instances to be able to reestablish communications after some have had their addresses changed. That means each instance must know the address of enough peers that at least one of those peers survives at their known address when the instance tries to reconnect to the network. In other applications llEmail can get by with much less redundancy because in those applications some instances are known to stay put, so their UUIDs won't get changed out from under them the way URLs do. But as I mumbled earlier, even static UUIDs can stop getting email because of a very old bug which I finally bestirred myself to look up.

I still submit that if and when we get grid-scope Experiences, they'll be the natural solution to this problem.

[mikka Luik]

13 hours ago, Wulfie Reanimator said:

..

The UUID of an in-world object changes every time it is rezzed (e.g. worn) and after sim restarts, so that's the same issue and not any more reliable than sending URLs back and forth. You'd have to send UUIDs back and forth too. There are no other types of cross-region communication in LSL besides Email and HTTP.

Just quick caveat on that - have an inworld monitor thart emails me stuff and the UUID does not seem to change on sim restarts. Eg inworld email address generated by UUID via llEmail

09 01 2019 <2a988bd4-1d35-e0*f-f981-e788*ab8823c@lsl.secondlife.com>

09 12 2018 <2a988bd4-1d35-e0*f-f981-e788*ab8823c@lsl.secondlife.com>

And reckon main channel has restarted at least once during that month (and yes redacted a couple of the UUID chars just in case =^^=) Have used it for experimental comms (even inter-grid) but yes there does seem to be the 'stuck' issue sometimes so never pursued it.

 

[Qie Niangao]

2 minutes ago, mikka Luik said:

the UUID does not seem to change on sim restarts

Oh definitely not! URLs do change on sim restart, but UUIDs only when the object is rezzed. Indeed, the object can be Worn, carried across a sim border and back, and preserve its UUID when Dropped -- the trick for recovering from the stuck email queue bug I mentioned above.