llCastRay returning zero hits seemingly at random

[Judy Hynes]

In the course of developing a script that uses llCastRay, I saw a behavior that makes no sense to me based on the documentation for this function.  The short story is that in the course of debugging some other issues, I found that multiple calls to llCastRay with the same inputs would sometimes produce zero hits (that is, a status_code of 0) even though it just produced several hits casting the same ray a few seconds earlier.  I cannot detect a pattern of any kind; it seems to detect the correct number of hits many times in a row, but occasionally returns status_code 0 for no apparent reason.

My function call looks like this:

    list ray = llCastRay(start, end, [RC_MAX_HITS, 3, RC_DETECT_PHANTOM, FALSE]);
    integer status_code = llList2Integer(ray, -1);

I have verified

• start and end are always valid and reasonable (based on debugging print statements)
• status_code is never negative, i.e. no error codes are ever being returned
• nothing is changing about the test environment, i.e. the prims it should be detecting are present, not moving, and not changing any of their attributes
• it's not being called repeatedly super fast; there's always at least 2 seconds between calls and often more
• nothing else weird is going on on the sim at the time; I'm the only one on it and nothing is going on around me

There is nothing in the  llCastRay wiki page about this kind of behavior; as I read it, all errors return a negative status_code, and there are no caveats about "it will actually give status 0 under condition x" or similar.  In short, I can think of no reason that I shouldn't always get the same positive number in the status_code if I give the function the same inputs, nothing changes in the environment, and there are no errors (i.e. negative status codes) related to sim performance.

Does anyone know why llCastRay could return status_code 0 at seemingly random times?  Or does anyone have any suggestions on debugging further or reducing/eliminating the chance of this random 0 result?

[Judy Hynes]

After much additional experimentation, I've found what I think the issue is.  I am calculating the end point by projecting from the start point along the rotation of the script's prim.  However, this sometimes makes one or more of the coordinates of the endpoint negative.  When this happens, the failure can occur seemingly at random, although llCastRay does return valid and reasonable results most of the time even so.  When I move my test or reduce the distance out that end point is placed such that none of the end point vector's components are negative, I am unable to reproduce the issue.  I feel pretty good that this is indeed the problem.

Here is my new question.  What is a good way to "cut off" the calculated endpoint so that it never crosses the boundary of the region it's in?  Or put another way, I still want to project up to a fixed distance away from the start point based on the prim rotation, but I want to "truncate" the result if it hits the edge of the sim.  What's a good way to do this?

The end point calculation is simple:

    vector end = start + <100.0,0.0,0.0>*llGetRot();

How can I bound the result so that neither of the x,y values falls outside the range [0, 256], nor z allowed to be negative?

Edited May 10, 2022 by Judy Hynes

[Quistess Alpha]

5 hours ago, Judy Hynes said:

How can I bound the result so that neither of the x,y values falls outside the range [0, 256], nor z allowed to be negative?

Some early morning "don't wanna sleep" general sketch:
 

vector diff = (end-start);
float magDiff = llVecMag(diff);
// calculate where the ray would hit the floor:
if(end.z < start.z)
{
  float f = (0-start.z)/(diff.z); // the 0 is overkill, but makes the similarity to below clearer.
  vector floor_hit = start+(f*diff);
  if(llVecDist(start,floor_hit)<magDiff)
  {	end=floor_hit;
   	jump raycast;
  }
}
// repeat above with x and y.
// . .. 

// calc where would hit east wall:
if(end.x > start.x)
{  float f = (256.0-start.x)/(diff.x);
   vector east_hit = start+(f*diff);
   if(llVecDist(start,east_hit)<magDiff)
  {	end=east_hit;
   	jump raycast;
  }
}
// similarly for y.
// ...
@raycast;
// do the actual raycast form start to end.

Edit: You can make the distance comparisons more efficient by using quadrances instead (distance squared = (a-b)*(a-b)) but adding that optimization might obfuscate the intent a bit too much for the forum post.

Edited May 10, 2022 by Quistess Alpha

[Quistess Alpha]

I could actually make that a bit nicer looking. . .

vector uWallLimitRay(vector start, vector end, vector dimension, float limit)
{   // return the point the ray would hit on the wall.
    vector diff = (end-start);
    float f = (limit-(start*dimension))/(diff*dimension);
    return start+(f*diff);
}

vector uSimLimitRay(vector start, vector end)
{ // return end, or end truncated within the region if out of bounds.
  if(end.z<0)
  {  return uWallLimitRay(start,end,<0,0,1>,0);
  }if(end.y<0)
  {  return uWallLimitRay(start,end,<0,1,0>,0);
  }if(end.x<0)
  {  return uWallLimitRay(start,end,<1,0,0>,0);
  }/*if(end.z>4096)
  {  return uWallLimitRay(start,end,<0,0,1>,4096);
  }*/if(end.y>256)
  {  return uWallLimitRay(start,end,<0,1,0>,256);
  }if(end.x>256)
  {  return uWallLimitRay(start,end,<1,0,0>,256);
  }
  return end;
}

/// the actual raycast
vector start = llGetPos();
vector end = uSimLimitRay(start,start+<60,0,0>*llGetRot());
// do raycast.

still early morning, no guarantees against math or programming errors.

Edited May 10, 2022 by Quistess Alpha
Simplified logic, removing distance comparisons entirely.

[Qie Niangao]

3 hours ago, Quistess Alpha said:

still early morning, no guarantees against math or programming errors.

Not sure about return because a single ray could extend past multiple border planes (e.g., <-5,-5,-5>), so probably sequential tweaks to end through the conditionals. Possibly else clauses for each dimension's <0 and >256 conditions because it can't be both—although in this case, I bet the errors would disappear if checking only the <0 cases.

Incidentally, I'm thinking this is probably a bug I never noticed in a little raycasting script that I run all the time, so I'm grateful to @Judy Hynes for bringing this up.

[Judy Hynes]

Thanks for the ideas @Quistess Alpha.  I will give this concept a try and see how it goes.

@Qie NiangaoI too am surprised this has not be discussed or brought up before.  I remember back when I first starting learning how llCastRay works that I questioned whether I needed to take care to bound the start and end points to stay inside the same region.  But my initial experiments not doing so using some out of bounds points didn't seem to have any issues.  It's only now that I see it mostly works even with negative vector components, but I think this must be some kind of accident of the math somehow.  I say this because I attached the test prim, went into mouselook and triggered the script changing my look angle slightly each time.  I could get the failure to happen this way sometimes (I would estimate 1 in about 10 to 15 tries).

Then I added this code just to test the "is it the math" theory.

    rotation rot = llGetRot();
    vector start = llGetPos() + <1.0, 0.0, 0.0>*rot;
    vector end = start + <300.0,0.0,0.0>*rot;
    
    list ray;
    integer status_code=0;
    integer retry_count=3;
    while( (status_code == 0) && (retry_count > 0) )
    {
        ray = llCastRay(start, end, [RC_MAX_HITS, 3, RC_DETECT_PHANTOM, FALSE]);
        status_code = llList2Integer(ray, -1);
        if(status_code == 0)
        {
            llOwnerSay("  status 0, trying again");
            //end += <0.0001, 0.0001, 0.0001>;
            --retry_count;
        }
    }
    
    llOwnerSay("   status="+(string)status_code+" start="+(string)start+" end="+(string)end);

With the line end += <0.0001, 0.0001, 0.0001>; commented out, any time status_code was 0 it repeated this result for all the the loop's retries.  That is, the same inputs into llCastRay always produced the same output, so it's not some kind of timing or transient issue that clears up if there happens to be some time that passes between calls.  However, with that line active (NOT commented out), a failure is almost always followed by success upon the first retry.  (I only had one test I can remember where 2 retries were needed to get a non-zero result.) So changing the end point, and thus the math, even by a tiny amount causes a different result.  And of course as before, there should have been hits detected in all tests, so the zero result is definitely wrong.

I'd love to know what's going on here.  It seems like insisting that the start and end points to llCastRay must be bounded in some way would be a reasonable API restriction, so I'm actually a little surprised it isn't.  But clearly there's something wrong happening under the hood with out-of-bounds points, so it would be good to understand what's really going on here.

[Judy Hynes]

I was able to test the endpoint limiting suggestion from @Quistess Alpha.  The math all seems to work just as expected, so thank you for that.  I have been able to try my same experiment with the limiting in place and have not reproduced the llCastRay error.  Since I was able to get the error regularly before, I think this does in fact address the issue.  I did a little rewriting on the conditions of the limit function, but otherwise it's what you originally wrote.  Here is the code I used when testing in case others want to try this solution for the issue.

 

// returns the end point adjusted to the specified limit
vector uWallLimitRay(vector start, vector end, vector dimension, float limit)
{
    vector diff = (end-start);
    float f = (limit-(start*dimension))/(diff*dimension);
    return start+(f*diff);
}

// returns end point adjusted to remain inside the region if necessary
vector uSimLimitRay(vector start, vector end)
{
    if(end.z < 0.0)
    {
        end = uWallLimitRay(start, end, <0.0, 0.0, 1.0>, 0.0);
    }
    /* don't need an upper bound on vertical dimension
    else if(end.z > 4096.0)
    {
        end = uWallLimitRay(start, end, <0.0, 0.0, 1.0>, 4096);
    }
    */
    
    if(end.x < 0.0)
    {
        end = uWallLimitRay(start, end, <1.0, 0.0, 0.0>, 0.0);
    }
    else if(end.x > 256.0)
    {
        end = uWallLimitRay(start, end, <1.0, 0.0, 0.0>, 256.0);
    }
    
    if(end.y < 0.0)
    {
        end = uWallLimitRay(start, end, <0.0, 1.0, 0.0>, 0.0);
    }
    else if(end.y > 256.0)
    {
        end = uWallLimitRay(start, end, <0.0, 1.0, 0.0>, 256.0);
    }
    
    return end;
}

cast_ray_experiment()
{
    vector start = llGetPos();
    vector end = start + <300.0,0.0,0.0>*llGetRot();

    // bound the endpoint within the current region
    end = uSimLimitRay(start, end);

    // now do the ray and see if anything was hit
    list ray = llCastRay(start, end, [RC_MAX_HITS, 3, RC_DETECT_PHANTOM, FALSE]);
    integer status_code = llList2Integer(ray, -1);
    
    llOwnerSay("status_code="+(string)status_code+"  start="+(string)start+" end="+(string)end);
}