Wiki note on precision

[NaomiLocket]

And type casts.

DumpList2String suggests we can use (string) to our hearts content and get an "identical" result. The typecast page tells us that precision is now at 5 places. A friend passed me a situation, and I did a simple dump of a quick mock of vectors showing that LSL is still doing 6 places of precision sometimes, and my friends if(~value) are failing when they should not (if precision were consistent).

Quote

Notes

Instead of using llDumpList2String(myList, "") you may wish to consider using the more efficient (string)myList as it produces an identical result with less memory usage due to eliminating a function-call. Each element of the list is converted to string format in the result, so floats expand to six digits of precision, rotations and vectors are represented with "<" and ">" characters, etc.

Can someone amend the wiki page to reflect the situation, instead of assuming people will go checking the Typecast page (when people are not doing typecasts themselves it still happens).

Object: 
[GetPrimitiveParams] // Raw llDumpList2String
<1.000000, 1.000000, 1.000000>
1.000000
Object: 
 <avector> // Typecast
<1.00000, 0.53654, 0.00000>

My friend related their recent experience, the precision worked as expected under LSO, but not under mono when pulling colour vectors from parameters and comparing to a hardcoded vector. For now they expect to limit precision themselves and round off, but it is an annoying surprise.

I'd have put this on a wiki discussion page, but you know why that isn't happening.

Edited June 1, 2020 by NaomiLocket
derpy grammar (there may be more)

[Wulfie Reanimator]

I could edit the wiki page but it definitely seems to work fine for me.

 singular vector typecast to string:

vector test = <0.12345678, 0, 0>;
llOwnerSay((string)test);

Output, 5 decimals as according to the wiki:

<0.12346, 0.00000, 0.00000>

 

Basic list test:

llOwnerSay((string)[<1, 2, 3.12345>, 0.12345678]);

Output, 6 decimals as according to the wiki:

<1.000000, 2.000000, 3.123450>0.123457

 

Display vs real value when comparing:

vector test = <0.12345678, 0, 0>;
float x = 0.123457;
llOwnerSay((string)[test.x, " == ", x, " returns ", test.x == x]);

Output, values are only identical in the string but not in memory, as expected:

0.123457 == 0.123457 returns 0

 

Edited June 1, 2020 by Wulfie Reanimator

[NaomiLocket]

They were still unable to get their script to pass under a typical ~ for the case.

ie: 0.123457 != 0.12345

Edited June 1, 2020 by NaomiLocket

[Wulfie Reanimator]

35 minutes ago, NaomiLocket said:

They were still unable to get their script to pass under a typical ~ for the case.

It's best to have your friend come here to talk about the specifics then. You can't use ~ with vectors or floats. It's an integer operator.

36 minutes ago, NaomiLocket said:

ie: 0.123457 != 0.12345

Regarding your edit, that should be obvious. 0.12345 is never 0.123457.

Edited June 1, 2020 by Wulfie Reanimator

[NaomiLocket]

Oh derp. I'm going to go peek at it again. (they just had a scripter group in world go over it and stuff)

[NaomiLocket]

2 minutes ago, Wulfie Reanimator said:

It's best to have your friend come here to talk about the specifics then. You can't use ~ with vectors or floats. It's an integer operator.

No that is fine. They ran the ~ on an integer result captured from the typical listfindlist.

[NaomiLocket]

What they did specifically, was take the vector from getlinkprimitiveparams into a list, take the value from the list into a vector (list2vector), capture the listfindlist passing the vector as a list as a direct argument [ variable ] to integer and do a ~.
They expected it to match their handwritten colour variable (that was 6 places).
The [ variable ] was spat out as 5.
I am guessing now they tried to set it with 6 and it was perhaps truncated.

[Wulfie Reanimator]

8 minutes ago, NaomiLocket said:

What they did specifically, was take the vector from getlinkprimitiveparams into a list, take the value from the list into a vector (list2vector), capture the listfindlist passing the vector as a list as a direct argument [ variable ] to integer and do a ~.
They expected it to match their handwritten colour variable (that was 6 places).
The [ variable ] was spat out as 5.
I am guessing now they tried to set it with 6 and it was perhaps truncated.

Then it's time to use this button in the color picker rather than copypasting from chat:

It'll copy the currently selected color vector in the same format as all the color functions use/return.

[NaomiLocket]

That just leaves me with the hard task of convincing them.

[Kyrah Abattoir]

3 hours ago, Wulfie Reanimator said:

It's best to have your friend come here to talk about the specifics then. You can't use ~ with vectors or floats. It's an integer operator.

Isn't it technically a binary operator?

[Wulfie Reanimator]

14 minutes ago, Kyrah Abattoir said:

Isn't it technically a binary operator?

No. The term you're looking for is unary -- as in one -- as it operates on one value.

~ is a unary integer operator in LSL. It operates one a single, integer value at a time.

+ for example is a binary operator in LSL. It operates on two values, not just with integers.

Some languages (like C) even have ternary operators, which have three parts.

Edited June 1, 2020 by Wulfie Reanimator

[Kyrah Abattoir]

Just now, Wulfie Reanimator said:

No. The term you're looking for is unary -- as in one -- as it operates on one value.

~ is a unary integer operator in LSL. It operates one a single, integer value at a time.

+ for example is a binary operator in LSL. It operates on two values, not just with integers.

Sorry i ment an operation at the binary level.

[Wulfie Reanimator]

3 minutes ago, Kyrah Abattoir said:

Sorry i ment an operation at the binary level.

By that logic you could make the argument that every operator is a binary operator, especially (and not only) all math-specific ones. That's not how operators are classified, unless you were thinking bitwise operators.

Edited June 1, 2020 by Wulfie Reanimator

[Profaitchikenz Haiku]

I'm guessing here that Kyrah was hinting at it being a bitwise operator?