LSL JSON Grammar Parsing - Progress!

[Love Zhaoying]

As part of my "too big to describe easily" project, I have created a baseline LSL Parser for JSON.

This gives me a "structured" format that I can use for any purpose (with different commands / P-Code / etc. as needed).  

Anyway, the point of this post is to show you:

a) An example "pseudocode" LSL function;

b) A working prototype JSON example, showing the grammar for implementing a)

c) The output that I get when running this using my LSL JSON Grammar Parser.

Example "Pseudocode" LSL Function

Note: The only thing in this example not supported in LSL is the "Switch/Case" commands.

Purpose: Demonstrate building output strings, incrementing a variable, If/Else, Switch/Case, and Looping.

string Name;
string X;

MyFunction() {

	llOwnerSay("Hello, " + Name + "!");

@Loop_Here;
	X = (string)((integer)X + 1);

	llOwnerSay("X=" + X);

	if ((integer)X <= 2) {
		llOwnerSay("If<=2 is TRUE!");
	}
	else if ((integer)X <= 4) {
		llOwnerSay("If<=4 is TRUE!");
	}

	if ((integer)X > 5)
		return;

	switch(X) {
		case "1": llOwnerSay("X=1!");
		case "3": llOwnerSay("X=3!");
		default: llOwnerSay("X is some other value!");
	}

	llOwnerSay("Looping..");

	jump Loop_Here;

}

 

Example JSON, Formatted (Comments added)

[

    ["^SAY","^`Hello,_","^@Name","!"],        // llOwnerSay("Hello, " + Name + "!");

    ["^^X",["^+","^@X",1]],                // X = X + 1;
    ["^SAY","^`X=","^@X"],                // llOwnerSay("X=" + X);
    [

        [["^CMP","^@X",2],"^?<=",        // if (x<=2) 
            ["^SAY","^`If X<=2 is TRUE!"],["^RTP"]        // llOwnerSay("If<=2 is TRUE!");
        ],
        [["^CMP","^@X",4],"^?<=",        // else if (x<=4)
            ["^SAY","^`If X<=4 is TRUE!"],["^RTP"]        // llOwnerSay("If<=4 is TRUE!");
        ]

    ],

    [["^CMP","^@X",5],["^?>",["^RTS"]]],        // if ((integer)X > 5) return;

    "^@X",                        // switch(X) - Note: This line just "gets" the value of X.
    {
        "1":["^SAY","^`X=1!"],            // case "1": llOwnerSay("X=1!");
        "3":["^SAY","^`X=3!"],            // case "3": llOwnerSay("X=3!");
        "DEFAULT":["^SAY","^`X is some other value!"]    // default: llOwnerSay("X is some other value!");
    },

    ["^SAY","^`Looping.."],                // llOwnerSay("Looping..");

    ["^GOE",1]                    // jump Loop_Here;

]

Example Output

Hello, Avatar!
X=1
If X<=2 is TRUE!
X=1!
Looping..
X=2
If X<=2 is TRUE!
X is some other value!
Looping..
X=3
If X<=4 is TRUE!
X=3!
Looping..
X=4
If X<=4 is TRUE!
X is some other value!
Looping..
X=5
X is some other value!
Looping..
X=6

A few notes: 

- The Grammar Parser is currently "recursive"; I will "unroll" it to be "iterative".  This will allow me to use it asynchronously (for events, etc).

- The code is less than "100" lines or so:

1) The "Grammar Parser" part of the code knows nothing about the "Commands" except the Command Prefix "^".  

2) The "Function Handler" part of the code handles all "functions" - in example , the JSON values starting with "^".

- The "Functions" are prototypes only, and will be replaced with my existing "P-Code" (from an earlier project).

- I will be using this initially for a "game engine" in LSL. (I have posted about that previously.)

- I don't plan to share the code for this, unless it becomes "perfected enough" for me to feel comfortable doing so.

- Due to how JSON is structured, JSON_OBJECT types are really only useful for "lookup" and for the "Switch/Case" statement.  Otherwise, all "code" is in JSON_ARRAY elements.

- This Parser is using both "data before function" and "data after function" processing - non-deterministic.   I have found this to be the most flexible.  What the heck does that mean?

Short answer: Parameters can be EITHER before OR after a "^Function".  Except for a few special cases, a ^Function is only executed / evaluated after its closing block "]" as in: "[^Function]".   So, these two statements are equivalent: ["^`Hello", "^SAY"] and ["^SAY", "^`Hello"].

Feel free to ask anything!

Thanks,

Love

Edited September 23, 2022 by Love Zhaoying

[CmpZ]

A word beginning with "^" means "do this"?  Like "^SAY" means call llOwnerSay?  "^+" means to use + in an expression?

And a word beginning with "^^" means assignment?

What's the `backtick` for?  Does it precede a string literal?

 

[Love Zhaoying]

1 hour ago, CmpZ said:

A word beginning with "^" means "do this"?  Like "^SAY" means call llOwnerSay?  "^+" means to use + in an expression?

Yes. I am using "^" as a "Prefix" to mean: "This JSON value is a Function" - and/or must be handled by my "Function handler".  

Anything that does not start with "^", the Parser treats as a "parameter" and adds to a "Parameter Stack".  The "Parameter Stack" is just a ";" separated string in my implementation; I avoided lists except where  I did not have a choice: the llJson() calls themselves all take a list as a parameter.

The "^" was chosen because that is the "Function prefix" in my "P-code Interpreter".  So, if everything else were to change when I integrate this Parser with my P-code Interpreter, the "^" would probably stay the same.

1 hour ago, CmpZ said:

And a word beginning with "^^" means assignment?

Yes, it means "the remainder after ^^ is a 'variable name', and any parameters following are the 'value' to assign".  I chose "^^" somewhat randomly, I am using "^@" as "get variable value". 

Originally "@" and a few others like the backtick "`" (see below) did not need the "^" prefix.  Then, I finally figured out that to make the Parser more "pure", I should use the "^" prefix for anything that is not a "Parameter" and let the "Function handler" do the rest.

1 hour ago, CmpZ said:

What's the `backtick` for?  Does it precede a string literal?

Yes, but only for "special cases". Originally, I was going to use the single quote ' for a string literal parameter.  Then, I figured out that "everything is a string literal parameter if it is not a Function".

The backtick "`" actually means:  This string literal gets an extra "Last String parameter" flag - similar to C-language null-terminated strings.

Using a "Last String parameter" ("end of string" / null / etc.) avoids needing to include a "# of Parameters Count" for Functions like "SAY".

When the backtick ` is encountered, the Function handler adds "000;" before that parameter to the "Parameter stack".

Example: `Hello becomes "Hello;000;" on the Parameter stack.

When the next parameters are needed for the same "SAY" command, they don't need the "000".

Example: adding ", avatar!" the Parameter stack now becomes:  ", avatar!;Hello;000".  

When the "SAY" function is finally run, it consumes all the Parameters "in reverse" until "000" is encountered.

So, ", avatar!;Hello;000" becomes "Hello, avatar!".

Note: The "_" is used to "add a space to the end of a string literal parameter", because JSON strips trailing spaces.

Thank you for reading and asking questions!

[Love Zhaoying]

I added "^JSR" - "Jump to Subroutine" today, and tested it with two functions:

- "SAYX" - Uses "^SAY" to show the value of variable X

- "INCX" - Performs "X=X+1".

These were added by name to a JSON_OBJECT, and the original function is now "MAIN".  Note that I left "^RTS" (Return from Subroutine) off the new calls "SAYX" and "INCX" due to some bug (the "^RTS" bubbles up, I will need to "kill" that flag when returning to the "Parent' caller).

So, the outer JSON is now a JSON_OBJECT containing each "Function call", while before it was a JSON_ARRAY.

Condensed JSON:

{"INCX":[["^SAY","^`INCX():"],["^^X",["^+","^@X",1]]],"Main":[["^SAY","^`Hello,_","^@Name","!"],["^JSR","INCX"],["^JSR","SAYX"],[[["^CMP","^@X",2],"^?<=",["^SAY","^`If X<=2 is TRUE!"],["^RTP"]],[["^CMP","^@X",4],"^?<=",["^SAY","^`If X<=4 is TRUE!"],["^RTP"]]],[["^CMP","^@X",5],["^?>",["^RTS"]]],"^@X",{"1":["^SAY","^`X=1!"],"3":["^SAY","^`X=3!"],"DEFAULT":["^SAY","^`X is some other value!"]},["^SAY","^`Looping.."],["^GOE",1]],"SAYX":[["^SAY","^`SAYX(): X=","^@X"]]}
Parser: - Init: gSlot1={"Vars":{"Name":"Avatar","X":0,"Y":1,"Z":2}}

Expanded and annotated:

{

"INCX":    // Function "INCX" follows

[["^SAY","^`INCX():"],["^^X",["^+","^@X",1]]],  // Say "INCX():", then X=X+1

"Main":   // Function "MAIN" follows

[["^SAY","^`Hello,_","^@Name","!"],

["^JSR","INCX"],    // Call "INCX"

["^JSR","SAYX"],   // Call "SAYX"

[[["^CMP","^@X",2],"^?<=",["^SAY","^`If X<=2 is TRUE!"],["^RTP"]],[["^CMP","^@X",4],"^?<=",["^SAY","^`If X<=4 is TRUE!"],["^RTP"]]],[["^CMP","^@X",5],["^?>",["^RTS"]]],"^@X",{"1":["^SAY","^`X=1!"],"3":["^SAY","^`X=3!"],"DEFAULT":["^SAY","^`X is some other value!"]},["^SAY","^`Looping.."],["^GOE",1]],

"SAYX":  // Function "SAYX" follow

[["^SAY","^`SAYX(): X=","^@X"]]    // Say "SAYX(): X=" + the value of X

}
 

[CmpZ]

For the backtick, you mentioned that `Hello becomes Hello;000 on the stack.  If I were to write "Hello", "000" in the JSON, does that turn into the same thing on that stack?

 

[Love Zhaoying]

29 minutes ago, CmpZ said:

For the backtick, you mentioned that `Hello becomes Hello;000 on the stack.  If I were to write "Hello", "000" in the JSON, does that turn into the same thing on that stack?

 

Almost - but backwards. The first parameter in JSON is added to the stack "first", and consumed "last".

So, "000" would need to be before "Hello" in the JSON data to replace the ` backtick behavior.

Today, I'm finally making changes to use "Iterative" processing instead of "Recursive" processing. 

[Love Zhaoying]

Update: I successfully converted the JSON Parser from "Recursion" to "Iteration".

This will enable me to use Asynchronous operation: exit and wait for response, then resume at the point where the JSON "structured code" was executing, etc.

That is the next goal - then add a few more features, do some code refactoring, and put the code to use!

Thanks,

Love