Jump to content

Wiki note on precision


NaomiLocket
 Share

You are about to reply to a thread that has been inactive for 1514 days.

Please take a moment to consider if this thread is worth bumping.

Recommended Posts

And type casts.

DumpList2String suggests we can use (string) to our hearts content and get an "identical" result. The typecast page tells us that precision is now at 5 places. A friend passed me a situation, and I did a simple dump of a quick mock of vectors showing that LSL is still doing 6 places of precision sometimes, and my friends if(~value) are failing when they should not (if precision were consistent).

Quote

Notes

Instead of using llDumpList2String(myList, "") you may wish to consider using the more efficient (string)myList as it produces an identical result with less memory usage due to eliminating a function-call. Each element of the list is converted to string format in the result, so floats expand to six digits of precision, rotations and vectors are represented with "<" and ">" characters, etc.


Can someone amend the wiki page to reflect the situation, instead of assuming people will go checking the Typecast page (when people are not doing typecasts themselves it still happens).

Object: 
[GetPrimitiveParams] // Raw llDumpList2String
<1.000000, 1.000000, 1.000000>
1.000000
Object: 
 <avector> // Typecast
<1.00000, 0.53654, 0.00000>

My friend related their recent experience, the precision worked as expected under LSO, but not under mono when pulling colour vectors from parameters and comparing to a hardcoded vector. For now they expect to limit precision themselves and round off, but it is an annoying surprise.

I'd have put this on a wiki discussion page, but you know why that isn't happening.

Edited by NaomiLocket
derpy grammar (there may be more)
Link to comment
Share on other sites

I could edit the wiki page but it definitely seems to work fine for me.

 singular vector typecast to string:

vector test = <0.12345678, 0, 0>;
llOwnerSay((string)test);

Output, 5 decimals as according to the wiki:

<0.12346, 0.00000, 0.00000>

 

Basic list test:

llOwnerSay((string)[<1, 2, 3.12345>, 0.12345678]);

Output, 6 decimals as according to the wiki:

<1.000000, 2.000000, 3.123450>0.123457

 

Display vs real value when comparing:

vector test = <0.12345678, 0, 0>;
float x = 0.123457;
llOwnerSay((string)[test.x, " == ", x, " returns ", test.x == x]);

Output, values are only identical in the string but not in memory, as expected:

0.123457 == 0.123457 returns 0

 

Edited by Wulfie Reanimator
Link to comment
Share on other sites

35 minutes ago, NaomiLocket said:

They were still unable to get their script to pass under a typical ~ for the case.

It's best to have your friend come here to talk about the specifics then. You can't use ~ with vectors or floats. It's an integer operator.

36 minutes ago, NaomiLocket said:

ie: 0.123457 != 0.12345

Regarding your edit, that should be obvious. 0.12345 is never 0.123457.

Edited by Wulfie Reanimator
Link to comment
Share on other sites

2 minutes ago, Wulfie Reanimator said:

It's best to have your friend come here to talk about the specifics then. You can't use ~ with vectors or floats. It's an integer operator.

No that is fine. They ran the ~ on an integer result captured from the typical listfindlist.

Link to comment
Share on other sites

What they did specifically, was take the vector from getlinkprimitiveparams into a list, take the value from the list into a vector (list2vector), capture the listfindlist passing the vector as a list as a direct argument [ variable ] to integer and do a ~.
They expected it to match their handwritten colour variable (that was 6 places).
The [ variable ] was spat out as 5.
I am guessing now they tried to set it with 6 and it was perhaps truncated.

Link to comment
Share on other sites

8 minutes ago, NaomiLocket said:

What they did specifically, was take the vector from getlinkprimitiveparams into a list, take the value from the list into a vector (list2vector), capture the listfindlist passing the vector as a list as a direct argument [ variable ] to integer and do a ~.
They expected it to match their handwritten colour variable (that was 6 places).
The [ variable ] was spat out as 5.
I am guessing now they tried to set it with 6 and it was perhaps truncated.

Then it's time to use this button in the color picker rather than copypasting from chat:

99dbd9b309.png

It'll copy the currently selected color vector in the same format as all the color functions use/return.

  • Thanks 1
Link to comment
Share on other sites

14 minutes ago, Kyrah Abattoir said:

Isn't it technically a binary operator?

No. The term you're looking for is unary -- as in one -- as it operates on one value.

~ is a unary integer operator in LSL. It operates one a single, integer value at a time.

+ for example is a binary operator in LSL. It operates on two values, not just with integers.

Some languages (like C) even have ternary operators, which have three parts.

Edited by Wulfie Reanimator
  • Like 1
Link to comment
Share on other sites

Just now, Wulfie Reanimator said:

No. The term you're looking for is unary -- as in one -- as it operates on one value.

~ is a unary integer operator in LSL. It operates one a single, integer value at a time.

+ for example is a binary operator in LSL. It operates on two values, not just with integers.

Sorry i ment an operation at the binary level.

Link to comment
Share on other sites

3 minutes ago, Kyrah Abattoir said:

Sorry i ment an operation at the binary level.

By that logic you could make the argument that every operator is a binary operator, especially (and not only) all math-specific ones. That's not how operators are classified, unless you were thinking bitwise operators.

Edited by Wulfie Reanimator
Link to comment
Share on other sites

You are about to reply to a thread that has been inactive for 1514 days.

Please take a moment to consider if this thread is worth bumping.

Please sign in to comment

You will be able to leave a comment after signing in



Sign In Now
 Share

×
×
  • Create New...