A Tool For Writing LSL Scripts In Another Programming Language

[Testicular Slingshot]

I would like the community's opinion on this subject to see if it's a worthwhile project to pursue.

For a while now, I've been thinking about creating a program that lets you write LSL scripts in another programming language, such as Python, or Java. With such a thing, you could introduce new abstractions to LSL like classes, dictionaries, switch statements, and shortcuts for tedious code. However, there's a lot of differences between these languages, so there would be some inescapable oddities like requiring typecasts on list items.

I'm confident that I would be able to create this tool by myself, but I would like to know what other people think of such an idea. Which source language would people prefer to use for writing LSL scripts?

Thanks for your time!

[Coffee Pancake]

LSL, operating in a tight 64k jail with a high bytecode overhead, would suffer from such an approach.

Any gains made by being able to make scripts in ______ will invariably come at the cost of script memory and execution efficacy.

 

 

[Madelaine McMasters]

The primary barrier to coding in LSL isn't the language, it's the workings of the underlying infrastructure. Java won't make llSetLinkPrimitiveParamsFast() or llHTTPRequest() any easier to understand. Niceties like libraries and #include are outside the scope of what's possible through translation. At the language level, LSL should be pretty easy to grasp for anyone with experience in another high level language.

[LoneWolfiNTj]

1 hour ago, Testicular Slingshot said:

I've been thinking about creating a program that lets you write LSL scripts in another programming language

If a script isn't written in LSL, then it isn't an LSL script.

I can see some advantages of making a cross-compiler to translate another language (say, C or Perl or Python) into valid LSL source that will compile to LSL bytecode under Mono. Perl data structures would be particularly nice. Python lambdas would be cool. C "structs" would be useful.

However, there would be some limitations: you couldn't write to files, because LSL doesn't allow that. You couldn't write to notecards, because LSL doesn't allow that. You couldn't load a 374295 byte file into memory and work with it, because LSL doesn't allow that. So your cross-compiler would have to be savvy as to LSL's IO & memory limitations, and would by necessity allow users to write, say, "PythonLSL" scripts which could use only a subset of the Python language.

But that being said, if you want to do it, go for it. It'll be a lot of work to create, though; compilers (including inter-language cross-compilers) always are.

[Mollymews]

If you are going down the path of make LSL look like other languages, I wouldn't mind being able to use named linkset properties.

for example there is a linkset with names say: Crate, Apples, Bananas, Carrots. Where Crate is the root prim. Apples is the first linked prim (link number 2)

then in code I can write

Apples.Texture = [1, "red", SELF, SELF, SELF];

the pre-processor knows to translate this to LSL:

llSetLinkPrimitiveParamsFast(LINK_SET, [PRIM_LINK_TARGET, 2, PRIM_TEXTURE, 1, "red", <repeats>, <offsets>, 'rot']);

SELF means get the existing <repeats> <offsets> and 'rot' from the prim and insert the values into llSetLink...

extended example:

Apples.Texture = [1, "red", SELF, SELF, SELF, 2, SELF, SELF,SELF, 90.0];

translation:

llSetLinkPrimitiveParamsFast(LINK_SET, [
    PRIM_LINK_TARGET, 2, PRIM_TEXTURE, 1, "red", <repeats>, <offsets>, <rot>,
        PRIM_LINK_TARGET, 2, PRIM_TEXTURE, 2, surface2name", <repeats>, <offsets>, 90.0]);

then the most complex:

Apples.Texture = [ALL_SIDES, "red", SELF, SELF, SELF];

llSetLinkPrimitiveParamsFast(LINK_SET, [
    PRIM_LINK_TARGET, 2, PRIM_TEXTURE, 1, "red", <repeats>, <offsets>, <rot>,
        PRIM_LINK_TARGET, 2, PRIM_TEXTURE, 2, "red", <repeats>, <offsets>, <rot>,
        
        .. for each surface of Apples 3, 4, 5, etc
]);

named properties for all the others also. Like Apples.Color = [...]  Apples.Text = [...] etc etc

then finally

Crate.Properties = [ Apples, PRIM_TEXTURE, ALL_SIDES, "green", SELF, SELF, SELF,
                     Bananas, PRIM_TEXTURE, 1, "yellow", SELF, SELF, 90.0,
                     Carrots, PRIM_COLOR, ALL_SIDES, <1.0,0.5,0.0>, SELF]
                   ];

the pre-processor flagging a error when a named link is not found

[Testicular Slingshot]

1 hour ago, Mollymews said:

If you are going down the path of make LSL look like other languages, I wouldn't mind being able to use named linkset properties.

for example there is a linkset with names say: Crate, Apples, Bananas, Carrots. Where Crate is the root prim. Apples is the first linked prim (link number 2)

then in code I can write

Apples.Texture = [1, "red", SELF, SELF, SELF];

the pre-processor knows to translate this to LSL:

llSetLinkPrimitiveParamsFast(LINK_SET, [PRIM_LINK_TARGET, 2, PRIM_TEXTURE, 1, "red", <repeats>, <offsets>, 'rot']);

SELF means get the existing <repeats> <offsets> and 'rot' from the prim and insert the values into llSetLink...

extended example:

Apples.Texture = [1, "red", SELF, SELF, SELF, 2, SELF, SELF,SELF, 90.0];

translation:

llSetLinkPrimitiveParamsFast(LINK_SET, [
    PRIM_LINK_TARGET, 2, PRIM_TEXTURE, 1, "red", <repeats>, <offsets>, <rot>,
        PRIM_LINK_TARGET, 2, PRIM_TEXTURE, 2, surface2name", <repeats>, <offsets>, 90.0]);

then the most complex:

Apples.Texture = [ALL_SIDES, "red", SELF, SELF, SELF];

llSetLinkPrimitiveParamsFast(LINK_SET, [
    PRIM_LINK_TARGET, 2, PRIM_TEXTURE, 1, "red", <repeats>, <offsets>, <rot>,
        PRIM_LINK_TARGET, 2, PRIM_TEXTURE, 2, "red", <repeats>, <offsets>, <rot>,
        
        .. for each surface of Apples 3, 4, 5, etc
]);

named properties for all the others also. Like Apples.Color = [...]  Apples.Text = [...] etc etc

then finally

Crate.Properties = [ Apples, PRIM_TEXTURE, ALL_SIDES, "green", SELF, SELF, SELF,
                     Bananas, PRIM_TEXTURE, 1, "yellow", SELF, SELF, 90.0,
                     Carrots, PRIM_COLOR, ALL_SIDES, <1.0,0.5,0.0>, SELF]
                   ];

the pre-processor flagging a error when a named link is not found

This is a pretty interesting idea, but I'm not sure if I want to mess around with the preprocessor. Still would be cool to have though.

[Mollymews]

6 minutes ago, Testicular Slingshot said:

This is a pretty interesting idea, but I'm not sure if I want to mess around with the preprocessor. Still would be cool to have though.

ah! I see now what you are intending. To write a compiler that outputs CIL which can be executed on the runtime server ? yes ?

if you were to go down this path then have a look at OpenSim if you haven't already, to at least have a project server to work with. There have been a number of attempts at different languages over the years, in various states of repair

link here: http://opensimulator.org/wiki/Scripting_Languages

 

[Testicular Slingshot]

5 minutes ago, Mollymews said:

ah! I see now what you are intending. To write a compiler that outputs CIL which can be executed on the runtime server ? yes ?

Not exactly. What I wanted to do was make a "transpiler" (https://en.wikipedia.org/wiki/Source-to-source_compiler ), which would just translate some source code into LSL that could then be copy-pasted into the viewer's editor.

[Mollymews]

 

1 hour ago, Testicular Slingshot said:

What I wanted to do was make a "transpiler" (https://en.wikipedia.org/wiki/Source-to-source_compiler ), which would just translate some source code into LSL that could then be copy-pasted into the viewer's editor.

A transpiler is a pre-processor in this sense. I.e. Java > LSL > CIL

languages like C#, VB, Object Pascal, Ruby, Python, Java etc all use named objects with properties and methods:  object.property   object.method

so if I write in Java

String s = "Hello world";

int len = s.length();

and this is transpiled to

integer len = llStringLength(s);

is the same method to get to

list texture = Apples.Texture(ALL_SIDES);

Apples.Texture = texture;

transpiled to the equivalent LSL llGetParams and llSetParams

 

Edited October 22, 2021 by Mollymews
LSL

[Profaitchikenz Haiku]

I think the only real benefit you could gain from this approach would be to have libraries of snippets that could either be inline-ed into the output LSL for speedier execution, or else grouped int functions for compactness. As others have said, the issue with LSL isn't so much the language as the underlying word it has to interface with.

What  think you could consider implementing is something like a preprocessor to take a switch-case construct and produce a set of if-else statements.

 

[Wulfie Reanimator]

10 hours ago, Madelaine McMasters said:

Niceties like libraries and #include are outside the scope of what's possible through translation.

11 hours ago, Testicular Slingshot said:

new abstractions to LSL like classes, dictionaries, switch statements, and shortcuts for tedious code.

It's worth mentioning that Firestorm's Preprocessor already does these things to some extent.

You can #include external files and #define constants/macros. It enables switch-statements, list[index] style access, magic constants (like current line), and other things.

https://wiki.firestormviewer.org/phoenix:lsl_preprocessor

46 minutes ago, Profaitchikenz Haiku said:

What think you could consider implementing is something like a preprocessor to take a switch-case construct and produce a set of if-else statements.

The current preprocessor generates jumps in order to support fall-through. 🙂

Edited October 22, 2021 by Wulfie Reanimator

[Profaitchikenz Haiku]

1 hour ago, Wulfie Reanimator said:

It's worth mentioning that Firestorm's Preprocessor already does these things to some extent.

Yes, I have played with it. To be honest I don't favour pre-processors myself, I feel more comfortable seeing the real code that gets given to the next stage. My suggestion to the OP was of implementing a standalone one as an exercise. 

And yes, jumps/gotos have had far too bad a reputation in the past, of you've ever programmed in assembler you'll know hat jump are as common if not more so than calls and returns.

[animats]

The Firestorm pre-processor is pretty good. The best feature is #include, which allows libraries. Importantly, anything from an #include that isn't referenced is removed. That allows libraries to contain lots of functions you may need, without memory cost if not used.

For a transpiler, LSL is a terrible target language. The immutable list system is the only data structure, and it's too hard to work with. If you translated arrays and structures into LSL, the resulting LSL code would be terribly slow.

The right solution, which Open Simulator has, is to let you write C# code. Internally, LSL (usually) compiles to Mono, which is a bytecode system that can run various languages. So the server side is almost C# ready. C# is an OK language, and we know it can be mapped to the LSL functions, so, why not?

What I had to go through to implement the maze solver for my NPCs in LSL was hell. All those global variables. I wrote that in Python first, to debug the algorithm.

Edited October 22, 2021 by animats

[Mollymews]

some thoughts on lists and arrays, and how might this be done with a pre-processor

i quite like the list type as it can store different types of data in the same structure

we can think of the list as a multi-column table also and code it up that way. LSL has some list strided functions to help with this

we can also think of a list as a traversable tree. I won't get into it here, just say that there is a example of how to do this way down in the Scripting Forum somehwere

 

the thing I find most tedious about lists is that we have to type functions to get the list element value

integer value = llList2Integer(myList, index);

i would like to be able to code:

integer value = myList[index];

the pre-processor inferring the appropriate function from the type of value. Translating to:

integer value = llList2Integer(myList, index);
string value = llList2String(myList, index);
key value = llList2Key(myList, index);

key value = (key)myList[index];

as this is preceded by a cast then the appropriate translation is

key value = (key)llList2String(myList, index);

Firestorm pre-processor does the reverse already:  myList[index] = value; Translating to llListReplaceList

once we have got to this point then can look at the strided list as a n-column array (a table)

#specify list myTable As Stride 2

list myTable;

myTable =  [123, 456];
myTable += [789, 100];

integer value = myTable[0, 0]; // == 123
value = myTable[0, 1]; // == 456
value = myTable[1, 0]; // == 789
value = myTable[1, 1]; // == 100

as myTable has been specified with a stride of 2 then the pre-processor can resolve the indexing

integer row = 1;
integer column = 0;

value = myTable[row, column];

translates to:

value = llList2Integer(myTable, 2 * row + column);  // == 789

a 3-column table goes:

#specify list myTable As Stride 3

list myTable;

myTable =  ["abc", 456, 789];
myTable += ["def", 100, 123];

integer row = 0;
integer column = 0;

string s = myTable[row, column];
integer i = myTable[row + 1, column + 2];

translate:

string s = llList2String(myTable, 3 * row + column);  // == "abc"
integer i = llList2Integer(myTable, 3 * (row + 1) + column + 2); // == 123

the pre-processor knowing that (row + 1) has to be wrapped in parentheses should the scripter not have provided them already

[Mollymews]

suppose we wanted our pre-processor to bring Object.Method into life

string s = "Hello World ";

integer len = s.length();

string chr = s.charAt(3);

string str = s.strAt(3, 7);

string trim = s.trim(STRING_TRIM);

translations:

integer len = llStringLength(s);

string chr = llChar(s, 3);

string str = llGetSubString(s, 3, 7);

string trim = llStringTrim(s, STRING_TRIM);

levelling up:

s.insert("One ", 6);
s.delete(6, 4);
s.replace("One", 7, 11);

integer i = s.find("World");

all translating to the appropriate LSL function

same with lists

list myList = [something,something];

myList.insert(["One"], 2);

translation: myList = llListInsertList(myList, ["One"], 2);
 

#specify myTable As Stride 2

list myTable = [some,thing,some,thing];

myTable.insert([this,thing], 1);  // insert as row 1
myTable.delete(1);  // delete row 1
myTable.replace([this, thing], 1); replace row 1, columns 0 and 1
myTable.replace([newthing], 1, 1); replace row 1 column 1

integer i = myTable.find([athing], 1);  // find a thing in column 1

// calculate row from i
row = i / 2;

 

ps. is all pretty interesting for me thinking about these kinds of things. I am not going to do it tho. I just like to think about how it might be done

i am wait for Rider Linden to make LSL++ in this kinda way.  Like one day maybe Rider Linden will go: gosh! I ran out of things to do, other than come to work and eat my lunch. So I better look busy or the boss might give me the sack. And go off and make LSL++ like before next Friday 😸

pps. Never know but LSL++ might come with a memory upgrade to 256K. Hope so

Edited October 23, 2021 by Mollymews

[Mollymews]

adding more thoughts

events. I wouldn't propose changing event names or the way in which they work

but as we are treating variables as objects then we can do this

touch_start(integer num)
{
    key id = num.detectedKey(0);
    string name = num.detectedName(0);
    vector pos = num.detectedPos(0);
}

the script would refer to itsef as self. Like

self.say(0, "hello");
self.regionSay(id, channel, "hello");
self.requestPermissions(id, ...);  // the script is requesting perms from id for itself

self.memoryLimit = 20000;
integer freemem = self.freeMemory();

listen(integer channel, string name, key id, string msg)
{
    if (id.owner() == self.owner())
    {
        if (msg.charAt(0) == "@")
           ... do @ stuff ...
    }  
}

thinking about linked objects more, the method I mentioned previously might be to much effort to implement. Alternatively then we put the onus on the scripter to specify the link numbers correctly. Same as it is now in LSL

#specify Apples As Link 2
#specify Bananas As Link 3

self.messageLinked(Apples, 0, "hello", id);

link_message(integer sender, integer num, string msg, key id)
{
   if (sender == Bananas)
   {
      if (msg == "change texture on surface 1")
      {
         self.texture = [id, 1];   // llSetTexture
         if (self.color(1) != <1.0, 1.0, 1.0>)   // llGetColor
             self.color = [<1.0,1.0,1.0>, 1.0] ; // llSetColor
      }
   }
}

with Get/Set Params then

list params = self.params([ list of parameters ]); // llGetP..
... do something with params ...
self.paramsFast = [ list of parameters ]; // llSetP..Fast

list params = Apples.params([list]);  // llGetLinkP...
Apples.paramsFast = params;            // llSetLinkP...
 

the thought behind me posting all of this is that I don't think we should model our language on a existing language. What we would be better to do is design a syntax for LSL++ which largely follows the existing LSL API naming conventions. The syntax also  reflecting the syntax conventions in languages like C#, Java, etc

as when we do this then people moving from LSL to LSL++ will find it easier. As will new people coming to LSL++ from object-oriented languages

LSL++ might be too ambitious as a name. LS+ might be more appropriate. Linden Scripting + a little bit. As all we have done here is skinned LSL. A skinning tho that lends itself to making a pre-processor that has a one-to-one mapping to the LSL API. A pre-processed code that in turn lends itself to interfacing in a fairly straightforward way with the existing LSL->CIL compiler

in sum if it went all the way then would just be a 3rd compiler option in the code editor: LSO LSL LS+

using the idea of a pre-processor: LS+ > LSL, as a syntax and API mapping tool to help design the LS+ language

 

[Profaitchikenz Haiku]

9 hours ago, Mollymews said:

the thing I find most tedious about lists is that we have to type functions to get the list element value

I think something could be done using llGtListEntryType if the return value was then used to actually produce the cast ?

ie, for a lst of varying types, you could produce a parallel list of the types of the contents of the first list. It would be all integers, obviously The resulting preprocessor code would appear to just do

integer myVal = listOfValues[4];

but the preprocessor would generate statements to determine the type of entry[4] and extract it by either inlining the statements or calling a function.

Being honest though, I can't see any performance advantage, and the coding advantage also seems trivial, it's cosmetic more then useful.

[Mollymews]

1 hour ago, Profaitchikenz Haiku said:

I can't see any performance advantage, and the coding advantage also seems trivial, it's cosmetic more then useful.

if there is an immediate advantage to LS+ it would be for people new to SL who have some experience with dev languages like Java, Ruby, Delphi, C#, VB, etc

i agree with your sentiment tho that any performance or extensibility gains would not come just with a pre-processor. It would come when the LS+ compiler was built. Things like structs for example would be a LS+ extension.  Introducing struct to LSL has always I think been problematic for Linden as it introduces a syntax addressing paradigm not included in LSL. Whereas struct is a natural language fit for LS+

LS+ example:
 

list People;  // is not specified As Stride so is a standard list

struct Person
{
   key id;
   string name;
   vector pos;
}

touch_start(integer num)
{
   if (num.detectedKey(0) == self.owner)
   {
      integer i = -People.length();
      for ( ; i; ++i)
      {
         Person this = People[i];
         self.ownerSay(this.id + " " + this.name + " " + (string)this.pos);
      }
   }
   else
   {
      Person thisPerson;

      thisPerson.id = num.detectedKey(0);
      thisPerson.name = num.detectedName(0);
      thisPerson.pos = num.detectedPos(0);

      People += [thisPerson];
   }
}

 

ps. just add that I kinda agree with not introducing new paradigms to LSL itself. As when so can easily end up butchering the naturalness of the language. Like butcher a language enough can end up with C++

pps. I have been having a play with Apple Swift language. Keeps all the advantages of Objective C. But adds a whole new level of usability, richness and elegance. Is really nice to work with. I got it running on Windows in Visual Studio 2019. I am never going to do anything in C++ ever again

Edited October 23, 2021 by Mollymews
when

[Profaitchikenz Haiku]

3 hours ago, Mollymews said:

just add that I kinda agree with not introducing new paradigms to LSL itself

^^^

This.

It's a nice little compact language suitable for newcomers to learn by reading examples and doing. Adding to it would just increase the swirl of names before a beginners' eyes.

 

Regarding structs, list come close but obviously fail on both the ability to include list within lists, and the ability to either have pointers to other lists or pointers to function addresses.

But in all of my time here, I've never yet threw up my hands and said "this language is impossible to work with". But then, I grew up programming on machines where 16K of working set memory was generous, and most variable or procedure names were limited to 6 characters. Looking through some C# code I got the impression that the names of many of the functions had more characters in them then there were characters in the actual body of the code.

Edited October 23, 2021 by Profaitchikenz Haiku

[Mollymews]

7 minutes ago, Profaitchikenz Haiku said:

list within lists

is not easy to implement this as is effectively a unstructured nested tree. And it might be better to create a new tree type for this with tree helper functions

otherwise

list a;
for (i = 0; i < 100; ++i)
{
    a += [[a]];
}

will take a lot of time when the traversal of this lists within lists  can only be coded in LSL. Whereas a tree type could have traverse functions

a thing is that we can use a list now as a tree already with slow traversal

this said most people I think, think of list within list as a 2-dimension thing as a substitute for a table array, which can be done already with strided list

start adding 3,4,5,...,10 dimensions then is going to crawl and use heaps of memory without traverse functions

 

[Profaitchikenz Haiku]

9 minutes ago, Mollymews said:

start adding 3,4,5,...,10 dimensions then is going to crawl and use heaps of memory without traverse functions

Yes, but from my Fortran days I would immediately look at sparse arrays as the best solution to this. In effect they were lists of lists.

[Mollymews]

12 hours ago, Profaitchikenz Haiku said:

Yes, but from my Fortran days I would immediately look at sparse arrays as the best solution to this. In effect they were lists of lists.

i looked it up, because I am nosey

from the Fortran examples I found, seems a sparse array is a n-dimensional matrix with memory only taken up by used elements

LSL equivalent is as you say lists within lists. Like:

list a =
[
  0,
  [1,[3,[7,8],4,[9,a]]],
  [2,[5,[b,c],6,[d,e]]]        
];

i thought it was more like a tree because 'list a' is a binary tree. But I can see now that a tree is only a subset view of a n-dimensional sparse matrix

i did read also that Fortran has quite a lot of helper functions/methods to quickly address/traverse the sparse array

a answer on stackoverflow, was the most helpful in my understanding of this

https://stackoverflow.com/questions/62255387/sparse-array-in-fortran

Fortran sparse arrays are pretty cool. And I can see how extremely useful they are to engineers and scientists who have heaps of variable data to model

[Testicular Slingshot]

15 hours ago, Profaitchikenz Haiku said:

Yes, but from my Fortran days I would immediately look at sparse arrays as the best solution to this. In effect they were lists of lists.

Multidimensional lists can be done with one list using math to get the true index of an item. They can't be jagged arrays though from what I can figure out, as that would require traversing the list, and keeping track of how many items is in each sub-list, which adds a lot of overhead. I hope that makes sense.

[Mollymews]

27 minutes ago, Testicular Slingshot said:

jagged arrays ... overhead

i dug up the post about using a list as a container for multi-dimensional structures. The post was in answer to a question about how to serialise/traverse a list structured as a unbalanced binary tree. The post shows a way to do this with a stack method which is non-recursive

is here:

as you mention tho, the bigger the tree (or any other unbalanced/jagged structure) gets the longer it takes to traverse and update. Is one the few blessings that comes from only having 64K memory for a LSL script. Our unbalanced structure is going to be relatively small so to fit it all into memory

 

ps.  Whenever topics like this come up I perk up and think ooo! thats interesting ! and I get lighty up eyes, itchy brain and even more itchy fingers. Type type type. And when I do then I can burble a whole lot. So I don't mean to run you all your thread tracking rainbow paw prints all over it

i would encourage you tho to have a go at writing your transpiler. As a thing in itself is pretty interesting.  And if you do get it to a point where the translation is close enough to good then  you could as a further exercise take it to a whole next level

have your new language compile to CIL

Sei Lisa has already made the compiler part, drawing on Linden open source code. Is here: https://github.com/Sei-Lisa/LSL-compiler

 

 

[Profaitchikenz Haiku]

4 hours ago, Mollymews said:

lists within lists.

Lists within lists is just about possible in LSL to one extra level, although horrible inefficient.

Consider a list of items describing the size, local position and local rotation of a child prim in a linkset

list details = [<size>,<lPos>,<lrot>];

dump that list to a string using the pipe character as the separator

string details_str = ["<size>|<lPos>|<lRot>"];

Now make up a list of such strings for every child in the linkset. List linksetDetails = [str1, str2,...];

To access the contents, read a particular string from the list, then either parse back to a list or simply slice substrings using the separator character

With numerical details it's straightforward, but if you start including strings you have to be sure that the actual strings will never contain your separator, or indeed a comma.

It's grossly inefficient because each character in the list is going to require 2 bytes . 

Edited October 24, 2021 by Profaitchikenz Haiku