DJ Board Display Not Working Properly

[Emmerich Beeswing]

39 minutes ago, Oz Linden said:

The old way certainly won't work.  Does this test script with these URLs work on other simulator versions?

When I try those URLs with CURL locally, I get the same results you describe here, so I don't see why they should work from LSL.

The script works with the old syntax at the old Simulators. I only changed the way the http user agent gets addressed by the new syntax. That the old way does not work is ok its just hard that It doesnt give back some error or so the http request just goes to nirwana with no answer so I can't catch some error and decide if the script is running on an old or an new Server Version thats the problem.

[Qwalyphi Korpov]

It's interesting that requests that worked... back when this worked now fail in a variety of ways using the latest server version and the example HTTP_USER_AGENT settings.  So far return codes 404, 403 and 502.

Possibly due to variations in the particular Shoutcast servers involved.

[Oz Linden]

anything that worked before (including test scripts) should still work on any region that is not in the Magnum or Cake channel; if it doesn't, the problem is not our server change.

if you can find a simple test script that works on the older releases now and fails on the Magnum or Cake channel, let us know.

[Qwalyphi Korpov]

12 minutes ago, Oz Linden said:

anything that worked before (including test scripts) should still work on any region that is not in the Magnum or Cake channel; if it doesn't, the problem is not our server change.

if you can find a simple test script that works on the older releases now and fails on the Magnum or Cake channel, let us know.

Four hours ago I posted a simple script (& other details) that should work with the new server change but doesn't. 

Of course anything that worked on the old release fails under the new release, giving a Script Warning/Error/Debug message:  URL passed to llHTTPRequest is not valid.  So I'm confused.

I'll paste as simple a script I can that has just two lines that differ.  One needs to be commented to run in the old servers and the other needs to be commented to run in the new.  Works fine on the old.  Fails on the new.

--------------------------------

// v3 - to use new HTTP request HTTP_USER_AGENT parameter
// v3.1 - simpler
key kSentRequest;
string URL;
string stream_status;
string stream1;
string stream2;
integer toggle = 0;
default
{
    state_entry()
    {   
        stream1 = "http://37.59.14.77:8352";
        stream2 = "http://server3.luschaudio.com:10272";
    }
    touch_start (integer number)
    {
        if (toggle)URL = stream1;
        else URL = stream2;
        llOwnerSay("URL:"+URL);
        toggle = !toggle;
        kSentRequest = llHTTPRequest(URL + "/7.html"+" HTTP/1.0\nUser-Agent: XML Getter (Mozilla Compatible) \n\n", [], "");  // old code
        //kSentRequest = llHTTPRequest(URL + "/7.html",[HTTP_USER_AGENT, "Stream-Script/1.0 (Mozilla Compatible)"],"");  // new code
    }   
    http_response (key kRecRequest, integer intStatus, list lstMeta, string strBody)
    {
        llOwnerSay("Status:"+(string)intStatus+", ResponseBody:"+strBody);
    }
}

[Whirly Fizzle]

Just to note for those testing this, Magnum & Cake are being rolled back again.

https://status.secondlifegrid.net/incidents/z6c6kk0h07hz

 

 

[Seline Constantine]

Any update on this issue?

Edited June 23, 2017 by Seline Constantine

[Emmerich Beeswing]

Waiting for the next update still Seline

[Ambix Habana]

Awesome to see Linden Noobs working on problems they made by themselves, and need users to get their own crap fixed. Impressive! Another proof that they have NO *****ing clue what they are doing. Thanks for 14 years of entertainment the special way ...

[Emmerich Beeswing]

Hello Oz

Today I tried this script Korpov used before at RC Magnum Server Version 17.06.23.327348 :

key kSentRequest;
string URL;
string stream_status;

default
{
    state_entry()
    {   
        URL = "http://secondstream.de:10001";
    }
    touch_start (integer number)
    {
        llOwnerSay("URL:"+URL);
        kSentRequest = llHTTPRequest(URL + "/7.html",[HTTP_USER_AGENT, "Stream-Script/1.0 (Mozilla Compatible)"],"");
    }   
    http_response (key kRecRequest, integer intStatus, list lstMeta, string strBody)
    {
        llOwnerSay("Status:"+(string)intStatus+", ResponseBody:"+strBody);
    }

}

 

Response: <h1>ERROR</h1> <h2>The requested URL could not be retrieved</h2>

With the old Syntax:

kSentRequest = llHTTPRequest(URL + "/7.html"+" HTTP/1.0\nUser-Agent: XML Getter (Mozilla Compatible) \n\n", [], "");

I got URL passed to llHTTPRequest contains a control character.

Can you pls help us with the right syntax?

Thx in advance

Emmerich

 

[Qwalyphi Korpov]

Testing today looks like the same results as a week ago.

[Oz Linden]

Ok ... we're continuing to learn new things ... with the pain that sometimes brings. What follows is going to get a little technical, but bear with me. If you're feeling swamped, skip to the numbered steps at the bottom of this post.

The fix is now behaving the way we intended, and sending perfectly good HTTP requests. We've found that some stream servers (including the ones we were doing our QA with) work just fine with the new option. Unfortunately, some others don't, but now we believe we know why not.

The broken servers we've found (including those above) are failing because they won't accept requests as large as we send. If we artificially truncate our requests, they work. An HTTP request consists of the request line with the URL followed by a bunch of header lines. Most of the headers we send are the X-SecondLife-something headers that identify the requesting object, where it is in SL, and who owns it.  Lots of scripters use these in their web applications; they're very useful and also can help us to track down misbehaving objects.

So... why did these scripts work with these servers before?  We missed one other effect of the old hack:

llHTTPRequest(URL + "/7.html"+" HTTP/1.0\nUser-Agent: XML Getter (Mozilla Compatible) \n\n", [], "");

What everyone thought was important about this hack was that it added "Mozilla Compatible" in a User-Agent header. That may have been and may still be important to some servers, but our new option provides for that (we can see in our traces that your scripts are sending your new user agent value with that in it). Note, however the "\n\n" at the end of the hacked-up URL parameter. We didn't realize the importance of that until just a bit ago. It turns out that that is what's making these broken servers work, because it's prematurely terminating the request headers (in HTTP, the headers are terminated by a blank line) - truncating the header. The rest of the headers our server generates (including standard ones and those useful X-SecondLife-* headers) were being treated as part of the request body by the server, but because nothing in the truncated headers tells the server to expect a body and these primitive stream servers don't expect one, they were just ignored. Now that we're not cutting them off artificially, the stream server reads the full set of headers and eventually decides that the request is too big and returns an error. Experimentally, it looks as though our standard requests are a couple of hundred bytes over what the broken servers we've tested against this afternoon will accept.

So what will we do? The short answer is "we don't know yet"; I'm writing this just a few minutes after we've diagnosed the problem, and we've got a bit more experimenting to do to work out the limits of these broken servers. I know, I keep saying "broken servers" as though I'm trying to shift blame. Not really, or at least not exclusively.  We should have figured this out a little earlier, and I'll personally take most of the blame for that. We missed the importance of the extra newline, and since the servers we'd picked from the earliest reports of this problem happen not to have the extreme size limitation, our tests all worked. But limiting request sizes to the very low values these servers are, and then not returning an error that indicates why they are giving up is very poor behavior - there's no excuse for either, really.  Compared to what browsers send routinely these requests are not large at all. So "broken servers" is fair.

So what should you do?  

1. Test your scripts (use a Magnum or Cake region; they've got the new option).  Do make the change to use the new HTTP_USER_AGENT option; it's permanent and may be important.
2. See if you can update or locate server software that doesn't have the request size limitation, or increase the limit if that's possible.  Making the limit 2K bytes would leave lots of headroom.
3. If you find server software that works or how to configure it so that it does, please share that knowledge here as soon as possible.

We're going to experiment some more and think about the problem. Stay tuned for more updates.

Note Well: It is possible that we'll decide that because it has other changes that are important to Second Life as a whole (and it does), we'll need to promote this version to the rest of the grid, even though scripts in it won't work with some stream servers, so please treat the steps above as Very Important.

 

[Emmerich Beeswing]

Tested several version with and without \n or %20% or even without defining a user agent. You said you did see it working? WHat is the Syntax please? That did NOT work:

key kSentRequest;
string URL;
string stream_status;

default
{
    state_entry()
    {   
        URL = "http://secondstream.de:10001";
        llOwnerSay("URL:"+URL);
        kSentRequest = llHTTPRequest(URL + "/7.html",[HTTP_USER_AGENT, "XML Getter (Mozilla Compatible) \n\n"],"");
    }   
    http_response (key kRecRequest, integer intStatus, list lstMeta, string strBody)
    {
        llOwnerSay("Status:"+(string)intStatus+", ResponseBody:"+strBody);
    }
}

 

RESPONSE: 

[23:36] Object: HTTP_USER_AGENT 'XML Getter (Mozilla Compatible) 

' is invalid

Edited June 29, 2017 by Emmerich Beeswing

[Lyndka Cochrane]

Oz - just to say "Thank you" and the folks at the desks around you for your work on this - I'm sure you'll reach some resolution, your tenacity in trying to resolve this issue is impressive.

[alison Carroll]

All i know with all this talk about scripts and the like is that i STILL have two SHX Boards that are NOT working where in other SIMS they still work perfectly ???? why is it so hard not to have the same software running in all servers ?? I do not, like most other users of SHX Boards, know much about LSL/Scripting and anyway the scripts in the SHX are locked and not editable.

[Emmerich Beeswing]

Yes I agree with Lyndka..thanks for your help Oz. Hope we get that running soon because my customers on the RC Servers are disappointet too atm :-)

[Shuichi Shinji]

@Oz:

Thank you for explaining the issue in such detail, it really helps understanding the difficulties in resolving this in a sensible way...

Would be be a possible solution to add another option to llHTTPRequest like HTTP_SKIP_DEFAULT_HEADERS or HTTP_SKIP_SL_HEADERS so the user of the command can decide whether or not to send the X-SecondLife headers?

[Shuichi Shinji]

And another question: Would it be possible to parse the hack (in a very restrictive way to avoid security issues) and build a standard-conforming HTTP request out of it? That would have the advantage that old scripts incl. SHX would still work. The preferred way would of course be the one using the extra options for user agent and potentially to skip the default headers, but to not break old content, the hack parsing would really be helpful...

[Oz Linden]

6 hours ago, Emmerich Beeswing said:

Tested several version with and without \n or %20% or even without defining a user agent. You said you did see it working? WHat is the Syntax please? That did NOT work:

key kSentRequest;
string URL;
string stream_status;

default
{
    state_entry()
    {   
        URL = "http://secondstream.de:10001";
        llOwnerSay("URL:"+URL);
        kSentRequest = llHTTPRequest(URL + "/7.html",[HTTP_USER_AGENT, "XML Getter (Mozilla Compatible) \n\n"],"");
    }   
    http_response (key kRecRequest, integer intStatus, list lstMeta, string strBody)
    {
        llOwnerSay("Status:"+(string)intStatus+", ResponseBody:"+strBody);
    }
}

 

RESPONSE: 

[23:36] Object: HTTP_USER_AGENT 'XML Getter (Mozilla Compatible) 

' is invalid

Emmerich ... your server (secondstream.de) is one of those that is behaving badly.  At the moment, there is no change you could make to your script that would work with it.

We are experimenting with ways to modify our requests that might allow it to work with servers like yours that reject large requests, but I must emphasize that any change we make is liable to be fragile with such poor quality server software, so trying to find a way to improve the streaming server you use is strongly advised.

To be clear: it will never work to put any control characters in any header value or in the URL; that's what is invalid about the User Agent value you supplied. The fact that it used to be allowed was a bug (and an embarrassing one).

We'll try to put up a region on Aditi later today for more testing... watch this thread.

[Emmerich Beeswing]

45 minutes ago, Oz Linden said:

Emmerich ... your server (secondstream.de) is one of those that is behaving badly.  At the moment, there is no change you could make to your script that would work with it.

We are experimenting with ways to modify our requests that might allow it to work with servers like yours that reject large requests, but I must emphasize that any change we make is liable to be fragile with such poor quality server software, so trying to find a way to improve the streaming server you use is strongly advised.

To be clear: it will never work to put any control characters in any header value or in the URL; that's what is invalid about the User Agent value you supplied. The fact that it used to be allowed was a bug (and an embarrassing one).

We'll try to put up a region on Aditi later today for more testing... watch this thread.

Umm well I just took the Server of Linden Radio for that example..but did not find a Server of a Radio which is sending Shoutcast V1 which works..giving out a wokraround for my Beeswing Clubsystems to my customers at the moment which doesnt show any songtitles but work at least

[Flenser Juergens]

On behalf of the DJ and Staff of Castle Dark Haven, we want to thank you Oz for all the work you've done so far.

Here's hoping code can be patched temporarily till a permanent solution is found.

I had wanted to change over to a different group but the sim owner lives in Tokyo and one doesn't phone there often unless it's an absolute emergency.

Keep working on it and I'm sure our wait will have been worthwhile.

 

[Oz Linden]

We have made a change that we hope will reduce the request sizes enough to allow some more servers to work. We plan to put that up next week on the same RC channels on Wednesday, but you can test it now on the Beta grid Aditi.

Go to any of Leafeon, Sylveon, Umbreon and Glaceon on Aditi, or you can look for server version 17.06.28.327400 on http://aditi.coreqa.net/gridtool.php (we'll be adding some more regions shortly).

If you've never used the Beta grid, see http://wiki.secondlife.com/wiki/Aditi#How_do_I_log_in_to_Aditi.3F

[Qwalyphi Korpov]

Testing at Leafeon went well for me.  RC 200 and a good response.

[Emmerich Beeswing]

12 hours ago, Oz Linden said:

We have made a change that we hope will reduce the request sizes enough to allow some more servers to work. We plan to put that up next week on the same RC channels on Wednesday, but you can test it now on the Beta grid Aditi.

Go to any of Leafeon, Sylveon, Umbreon and Glaceon on Aditi, or you can look for server version 17.06.28.327400 on http://aditi.coreqa.net/gridtool.php (we'll be adding some more regions shortly).

If you've never used the Beta grid, see http://wiki.secondlife.com/wiki/Aditi#How_do_I_log_in_to_Aditi.3F

Oz I read that I need to contact support to get acces to Beta grid? Can you help me pls? Tried with my username Emmerich Beeswing but didnt work

Edited June 30, 2017 by Emmerich Beeswing

[Oz Linden]

2 hours ago, Emmerich Beeswing said:

Oz I read that I need to contact support to get acces to Beta grid? Can you help me pls? Tried with my username Emmerich Beeswing but didnt work

I'm not support; see https://support.secondlife.com/

[Emmerich Beeswing]

1 hour ago, Oz Linden said:

I'm not support; see https://support.secondlife.com/

Ok sorry :-)