Jump to content

Float cut Zeroes


thimblemunch
 Share

You are about to reply to a thread that has been inactive for 1609 days.

Please take a moment to consider if this thread is worth bumping.

Recommended Posts

There are several ways to do that.  Try this one:

How many places do you want after the decimal point?  Three, maybe?  Multiply your float by 1000 and  convert it to an integer.  Then convert that  to a string and split into two parts so that there are three digits in the right hand part, and put a decimal point at the spot.

Link to comment
Share on other sites

written off the top of my head, not in world to test it

string flrstr(float num, integer places)//the number to be cut, and how many decimal places you want{      string numstr = (string)num;//converts the float to a string      integer idx = llSubStringIndex(numstr, ".");//finds the decimal place      numstr = llGetSubString(numstr, 0, idx+places);//captures the string from the beginning to the the number of decimal places specified      return numstr;}
Link to comment
Share on other sites


thimblemunch wrote:

Hmm I would only want two spots after decimal place. I am lost at the slitting two parts so there are two/three digits area. Is there any examples of this kind of thing or just the LSL wiki keywords I'm going to need to do it?

Thanks

One, two, three ... the number doesn't make any difference. Use llDeleteSubString to get rid of the parts you don't want.  The mechanics of which functions to use are the trivial parts of scripting, so that's up to you.  The important part is the logic.  That's where you'll get the interesting suggestions.  As I said, there are many ways to approach this problem.

  • Like 1
Link to comment
Share on other sites

to cut off only the trailing zeros without specifiying the number of decimal places, you could use a while loop, starting at the end of the float cast to a string, find the first non-0, the capture the string to that place.
if it doesn't have any trailing zeros, it will never cut any off and still return the whole string

string fltstr(float num){string numstr = (string)num;integer len = llStringLength(numstr);integer i;while(len){len--;if(llGetSubString(numstr,len,len) != "0")//when we come to the first trailing number (or decimal point) that is not a 0{numstr = llGetSubString(numstr,0,len);//capture the string to the index we found the first non-zerolen = 0;//exit the loop by changing the len to zero(FALSE)}}return numstr;}

Edit: Fixed the line to capture the string length, had llStringLength(num);

It is now llStrengLength(numstr); as it should be 

Link to comment
Share on other sites

what Rolig said

+

if do only want 2 decimal places and rounding up/down then

 

string float2str(float f){  return llDeleteSubString((string)(f + 0.005), -4, -1);   }

 

eta. I just add a caveat here

before anybody use this as is, then is best to read the points discussed further down in the convo. What the limitations of LSL string casting are 

Link to comment
Share on other sites

we always get 6 trailing decimal points in the LSL format string

you are right tho about numbers that take more than 24 bits to store their values

the way LSL string casting out-the-box works, the upper bound is 8388607.0 . Anything above this and LSL casting will round the string to a whole number with 6 zeroed decimal places

when rounding to 2 decimal places then the effective positive lower bound is 0.005. So that will cast ok. Anything less than 0.005 and greater than 0.0 will round down to 0.0

and then if did want to deal with negative numbers then should typically subtract 0.005 instead of adding it like we do for positive values

to do a larger number thats stored in more than 24 bits (> 8388607.0) then we have to do what else Rolig said

Link to comment
Share on other sites

You are right, my mistake

Still it is a bit in sane to keep digits after the point that carry no value and to round small numbers to zero

float a = 1E38;float b = 0.0;float c = 1e-8;default{    state_entry()    {        llOwnerSay( "a = "+(string)a+" b = "+(string)b+" c = "+(string)c);    }}/*Output a = 100000000000000000000000000000000000000.000000 b = 0.000000 c = 0.000000*/

:smileysurprised::):smileyvery-happy:

Link to comment
Share on other sites

There's a "Fixed Precision" page on the wiki. I'm embarrassed to admit that, before this thread, I used that function for a long time without actually studying how it works.

string fixedPrecision(float input, integer precision){    if((precision = (precision - 7 - (precision < 1))) & 0x80000000)        return llGetSubString((string)input, 0, precision);    return (string)input;}
(It helps to know / realize that floats are always cast to string with exactly six digits precision, as mentioned above. Also that "=" isn't "==", that "<" isn't "<<", that it's using negative indexing for the llGetSubString, and that "& 0x80000000" guards against a non-negative index.)
Link to comment
Share on other sites


Madelaine McMasters wrote:

Now there's a clear example of the value of commenting your code. And the person who wrote it apparently can't be bothered to provide that value, and thereby earns my scorn.

;-).

jejejjeje (:

you need to go mud wrestling more often. Mud wrestlers get quite scornless after a few bouts

can get even more scornless when get into soapy bath wrestling

q: (: 

+

ps. Is about now that is time for another letter to LL:

dear LL,

please to give us a currency type ok

when you got like nothing else better to do one day, other than just turn up to eat your lunch. Or somebody elses bc they brought even more yummy stuff to the work shared lunch

tnks (:

Link to comment
Share on other sites

float a = 1.2345678E38;float b = 0.0;float c = 1.2345678e-8;string fixedPrecision(float input, integer precision){    if((precision = (precision - 7 - (precision < 1))) & 0x80000000)        return llGetSubString((string)input, 0, precision);    return (string)input;}default{    state_entry()    {        llOwnerSay( "a = "+fixedPrecision( a, 3)+" b = "+fixedPrecision( b, 3)+" c = "+fixedPrecision( c, 3));    }}/* Output: a = 123456800000000000000000000000000000000.000 b = 0.000 c = 0.000*/

I had to do this to figure out what might be the meaning of the word 'precision'.

The output indicates that precision is used for the number of decimal digits after the decimal point.

In my vocabulary a is presented with seven digits precision while c is presented with zero digits precision.

After all this routine is very useful for numbers with absolute values not too big and not too small and zero

:smileysurprised::):smileyvery-happy:

Link to comment
Share on other sites


Dora Gustafson wrote:

I had to do this to figure out what might be the meaning of the word 'precision'.

The output indicates that precision is used for the number of decimal digits after the decimal point.

In my vocabulary
a
is presented with seven digits precision while
c
is presented with zero digits precision.

I think of precision the same way you do, Dora. That function takes the spreadsheet (financial) approach, which I suppose I'd call "resolution".

Link to comment
Share on other sites

I agree. Way back when I was first taught such things, we were taught that precision is a measure of your confidence in a value.  So, it may be accurate to say that the distance from the street to the back of your garage is 50 m, but that value is only useful in a practical sense if you also know whether it's 50m +/- 10m or 50m +/- 1m.  Precision makes the difference if you hope to park a car without hitting the wall.

There is obvously a connection between the statistical definition of precision and the one that you are calling "resolution", Maddy.  If I report a value as 12.346, I am saying that I'm confident that it's between 12.3455 and 12.3465, so my implied precision is +/- 0.0005.  I am using the number of significant figures after the decimal point as a shorthand because we have an understanding that we round trailing digits (the "leftover" part) up or down to the nearest value after the decimal. That implied precision is your "resolution".  In fact, however, I might have done analyses that give me higher confidence than just +/- 0.0005.  My real precision might be +/- 0.0003, in which case "resolution" is underselling my statistical confidence.

Link to comment
Share on other sites


Rolig Loon wrote:

I agree. Way back when I was first taught such things, we were taught that precision is a measure of your confidence in a value.  So, it may be accurate to say that the distance from the street to the back of your garage is 50 m, but that value is only useful in a practical sense if you also know whether it's 50m +/- 10m or 50m +/- 1m.  Precision makes the difference if you hope to park a car without hitting the wall.

There is obvously a connection between the statistical definition of precision and the one that you are calling "resolution", Maddy.  If I report a value as 12.346, I am saying that I'm confident that it's between 12.3455 and 12.3465, so my implied precision is +/- 0.0005.  I am using the number of significant figures after the decimal point as a shorthand because we have an understanding that we round trailing digits (the "leftover" part) up or down to the nearest value after the decimal. That implied precision is your "resolution".  In fact, however, I might have done analyses that give me higher confidence than just +/- 0.0005.  My real precision might be +/- 0.0003, in which case "resolution" is underselling my statistical confidence.

Yep, and I suspect you are as annoyed as I am over the excessive precision you see in everyday discussions of things (Clinton ahead 3.1% - margin of error of 4%).

Here's how I learned the terms in engineering school: 

  • Resolution is the smallest change you can detect, real or not
  • Precision is the repeatibility or consistency of your measurement, real or not
  • Accuracy is how well your measurements reflect reality
  • Stability is the confidence you have that your accuracy won't vanish while you're on vacation.

If I could have only one of those to start, I'd take stability. I can build the rest from there... and enjoy my vacations.

Here's another take on all but stability...

https://phidgets.wordpress.com/2014/05/20/accuracy-precision-and-resolution-theyre-not-the-same/

;-).

Link to comment
Share on other sites


Rolig Loon wrote:

I agree. Way back when I was first taught such things, we were taught that precision is a measure of your confidence in a value.  So, it may be accurate to say that the distance from the street to the back of your garage is 50 m, but that value is only useful in a practical sense if you also know whether it's 50m +/- 10m or 50m +/- 1m.  Precision makes the difference if you hope to park a car without hitting the wall.

There is obvously a connection between the statistical definition of precision and the one that you are calling "resolution", Maddy.  If I report a value as 12.346, I am saying that I'm confident that it's between 12.3455 and 12.3465, so my implied precision is +/- 0.0005.  I am using the number of significant figures after the decimal point as a shorthand because we have an understanding that we round trailing digits (the "leftover" part) up or down to the nearest value after the decimal. That implied precision is your "resolution".  In fact, however, I might have done analyses that give me higher confidence than just +/- 0.0005.  My real precision might be +/- 0.0003, in which case "resolution" is underselling my statistical confidence.

I was just thinking how much less important the distinction between resolution and precision can be in binary, where I work. Each place holder is only 2x bigger than the one to the right. So, you can't undersell your statistical confidence by more than 2x by dropping a bit.

Link to comment
Share on other sites

i had a quick nosey to follow up on what Qie said about 6-digit decimal precision in LSL

+

am pretty sure the LSL float is compliant with IEEE754. At least 754-1985. Not sure about 754-2008 tho, given that this came out after SL started. Altho 2008 is a superset of 1985 so am not sure it would make a lot of difference in practice for LSL math

compliant meaning as the LSL math engine is implemented, then in the range [1 to 6] of significant decimal digits there are no gaps in the numbers. That all numbers of 6 significant decimal digits or less (regardless of where the decimal point is) can be reached

and any calculation on these will be accurate (precisely correct) provided that the sum, and all subsums of the workings, are also less than or equal to 6 significant decimal digits

+

is possible to get 7.22 decimal digit precision (24bits), however I think that without testing every LSL-produced value above 6 digits in a math suite test then probably best to stick with 6 when/if accuracy is required

where the 7th digit can be relied on tho to be always accurate is when the quotient is 0. eg. 0.000001, 0.999999

+

if we did want a larger precisely accurate range in our LSL app then we pretty much have to use some kinda LSL-written bignum float math module

+

i just also add a note about LSL rounding

LSL uses binary rounding out-the-box. Which makes sense given that most LSL apps float math involves vectors and rotations. Binary rounding is overall more closer to accurate for these kinda calcs

if we do want currency rounding then have to do the +/- 0.005 thing ourselves

Link to comment
Share on other sites


wherorangi wrote:

i had a quick nosey to follow up on what Qie said about 6-digit decimal precision in LSL

Oh eep! I did kinda say that, now that I look back, but I only meant the fixed 6-trailing-digit representation when a float is cast to a string, as had been mentioned before in this thread. I just used "precision" imprecisely, with precisely the same imprecision as the function being labeled "fixedPrecision()" which might more accurately be called "fixedPoint()".

Link to comment
Share on other sites

You are about to reply to a thread that has been inactive for 1609 days.

Please take a moment to consider if this thread is worth bumping.

Please sign in to comment

You will be able to leave a comment after signing in



Sign In Now
 Share

×
×
  • Create New...