Jump to content

Fred Allandale

Resident
 View Profile  See their activity

  • Posts

    19

  • Joined

  • Last visited

 Content Type 

Posts posted by Fred Allandale

    LSL HTTP Changes Coming

    in LSL Scripting

    Posted

    I moved my web service from a shared server to a VPS (virtual private server). I am very grateful to the tech support folks at my web host (www.stablehost.com) that helped me with this transition and it went very smoothly. While more expensive for me, it gives me much more control. Most importantly it stopped the IP blocking that was going on without needing to white list specific IP ranges. Things appear to be working normally now. Thanks for everyone's help.

    To Oz re: using experiences to store URL information. Unfortunately this is not really feasible for my application. My objects (more the 1000) that need to communicate with each other are located in many different regions and have many different owners. Enabling experiences in all of them is not feasible. I agree there are probably some applications where this might be a good solution. Just not mine. Thanks anyway.

    • Like 1

    LSL HTTP Changes Coming

    in LSL Scripting

    Posted

    My web host provider acknowledges it is probably their firewall that is blocking my HTTP requests. My current web hosting plan is on a shared server and the provider has limited capability to change the firewall rules. About all they can do is white list a specific range of source IPs. That is what they did about 2 years ago when I was having some initial blocking problems, and it worked flawlessly after that. Unfortunately IP white listing is not practical in this new world of clouds and proxy server pools. I'm therefore exploring the option of moving to a dedicated server (VPS) with the same provider. This should allow for more flexible firewall rules. While this is a more expensive solution, it is probably what is necessary to provide a reliable solution for my Second Life customers.

    I agree it may be helpful to other HTTP users, especially those with high traffic applications (e.g. networked vendor systems), to be aware of this possibility and to design their systems accordingly. I don't blame my web hosting service (Stablehost). Their service reliability and technical support has been excellent. However, my current plan was very inexpensive and was designed mainly to support vanilla websites. It therefore has an anti-robot firewall strategy. My application is not a website. In fact, because my requests come from many scripted objects, not people behind web browsers, it essentially looks and acts like a bunch of robots. Exactly the thing their firewall was designed to protect against. I think its just a case of a mismatch between my application and the server hosting plan and capabilities. Hopefully upgrading to a dedicated server will solve this problem.

    I would also like to suggest to Second Life that they seriously consider implementing a reliable method for in-world scripted objects in different regions to communicate without resorting to use of an off-world URL lookup database. It seems like an unjustifiably complex solution to a simple requirement and only serves to add additional load to outbound and inbound HTTP traffic as well as making it susceptible to third party policy changes. The only other in-world option is llEmail, which I used for several years for the same products. But it was horribly unreliable (as noted in the wiki) and I finally moved to the current HTTP solution, albeit kicking and screaming all the way in anticipation of problems such as the current one.

    Thanks again for your help and suggestions.

    LSL HTTP Changes Coming

    in LSL Scripting

    Posted

    Thanks for your quick response Monty. The domain subscriberkiosk.com is my own domain hosted on StableHost (www.stablehost.com). So I don't really have any control over their security policies. I passed the link with the AWS ip ranges to them, but they say they can't white list the full set of AWS ip's for the security reasons you stated. So IP white listing is not a viable solution.

    I'm still waiting for them to tell me exactly why their firewall is blocking some AWS IPs but not others, although they did say it was triggering a CAPTCHA response. My theory is that their firewall's rate-limiter is triggering the CAPTCHA response (403), then blocking further requests (502) for some period of time if some number of requests from the same ip are received within some period of time. The move to the cloud is exacerbating this because there are probably many fewer ip's in the AWS proxy pool than were previously used by the Second Life proxy servers dedicated to each simulator. In particular, during the weekly rolling restarts, hundreds (maybe thousands) of my in-world server objects are trying to register their new URL's to my PHP/MySQL database at subscriberkiosk.com. This could cause a flood of requests via a limited number of AWS proxy server ip's and trigger the host firewall's rate-limiter to block them. This may explain why only some (maybe 20%) of the requests succeed. Note that my in-world server objects repeat unsuccessful HTTP requests every 10 minutes, then stop after 18 tries. At that point, my remote terminals can no longer contact their servers and go offline. Not good. Also, if the remote terminals lose contact with their server due to the changed URL, they also try to get its new URL from the PHP/MySQL database, further increasing the request rate.

    It is unfortunate that this complicated process of using an off-world URL lookup database is necessary to facilitate purely in-world, reliable, cross-sim, object-to-object communications. Prior to moving to HTTP, I used llEmail() but it was horribly unreliable and the built-in 20 second sleep made it difficult to support higher traffic applications.

    So I'm still not sure how to fix this issue. I also filed a jira bug report on this (BUG-229714) in the hopes someone will come up with a creative solution. I suppose I could move my database to a different web host provider, but there is no guarantee their firewall wouldn't do the same thing. If it is a rate-limiter issue, perhaps they could increase (or eliminate) the rate limit on inbound requests to subscriberkiosk.com (I own the domain). If its not a rate-limiter issue, then perhaps there is something in the header or other aspect of the request their firewall doesn't like that could be fixed on my end or by Second Life. I would think this might be a problem for other high traffic in-world HTTP applications (e.g. web based vendors) since most web host firewalls have some sort of rate-limiter to prevent DDOS attacks.

    Still hoping there is a workable solution. Any help or suggestions would be appreciated as this issue is rendering my SL products unusable for many customers.

    LSL HTTP Changes Coming

    in LSL Scripting

    Posted

    For the past 2 days my in-world servers (hundreds of them) have been getting a very high number of status 403 and 502 responses to HTTP Requests made to my off world PHP/MySQL application. This has been working perfectly for the past several years without any changes. The error responses are coming only from object URLs that use the new AWS cloud-based proxy server pool and only started when this service was moved to the cloud. My web host service provider indicated that their front end firewall is likely rejecting and blocking some (but not all) of those IP's. They want me to give them a list of IP's used by SL/AWS for these proxy servers so they can white list them in their firewall. My understanding from your explanation above is that the set of IPs now used can change frequently, randomly and unpredictably, making white listing them impractical. If, however, you can provide me with a range of IP's SL uses, it might be possible.

    I have set up an HTTP Request Test logger which sends an HTTP request every 10 minutes  to my test script on the same web host server (http://subscriberkiosk.com/test.php). Last night it got the following results:

    [08:53] HTTP Request Logger: Good responses: 16 (23%) [status 200]
    [08:53] HTTP Request Logger: Bad responses: 54 (77%)  [52 of 54 were status 502, 2 of 54 were status 403]
    [08:53] HTTP Request Logger: No responses: 0 (0%)
    [08:53] HTTP Request Logger: Total responses: 70
    [08:53] HTTP Request Logger: Total requests: 70

    I've also asked my web host provider to tell me exactly why their firewall is blocking certain IPs and not others originating from SL, in the hopes there is something in the HTTP request format that is causing this and can be corrected if we know what it was. So far they have been unable to tell me. They use third-party firewall software and may not know its inner workings.

    Note that this problem first appeared on Nov 12th and lasted less then 24 hours before resuming normal operation without any intervention. This time, it started on Nov 19th and is still continuing 48 hours later with no sign of remission. This is coincidentally (or not) the same time as certain region's  HTTP proxy servers were being moved to the AWS cloud.

    NOTE: I opened an SL Support Ticket for this issue 2 days ago but have not received any attention yet. The purpose of my PHP/MySQL application is to act as a URL lookup table to facilitate SL cross-sim object-to-object communications. It is used by literally hundreds of my servers and remote terminals owned by my customers. This disruption is causing serious problems for my customers. It has been working flawlessly for several years up until this recent move to the cloud.

    Automatic Notification System (ANS) Launches for Direct Delivery

    in Merchants

    Posted

    Not sure if this thread is still active, but I'm having an ANS problem.  I've set up an ANS processor on a web host using PHP and SQL, following the method described at http://community.secondlife.com/t5/Merchants/Setting-up-Marketplace-to-use-ANS-and-log-to-an-external/m-p/2177571#M38386. I'm getting transaction reports from the MP OK, except when there are multiple line items in a cart purchased under one Tranaction ID, the MySQL database only shows the first transaction.  The corresponding report I get via Email correctly show all of the line items.

    Is this a bug or do I just have something set up wrong?  If the latter, what might it be?

    Problem with "Related Items" in listings

    in Merchants

    Posted

    Thanks for the comments Pamela.  My current problem is sort of the opposite but probably related.  The valid related items are randomly disappearing from listings.  I think it may be related to when I upload an updated a product version and  associate the new version with the existing listing.  I wondering if the related products in that listing may still point to the previously associated product version which is no longer associated with any listing and as a result, it does not show up as a related item in other listings, even though it does show in the edit pages for those listings.  There is really no way to tell without inspect all of your listings.  I was able to fix this by deleting the related items, then re-entering them, but this is a real hassle.

    It is still a mystery why some listings still display the related items while other listings have lost the same related items.  If it were due to the above theory about product upgrades breaking the associations, you would think all listing would lose the same upgraded related items.  But this is not the case.  I opened a support ticket, but LL just told me to clear browser cache and try again, and to file a jira if that didn't work.  Of course it didn't work, so I am going to file a jira on this issue.

    I would be interested in finding out if anyone else has seen this same problem.  Unfortunately with the new jira system, there is no way to check to see if anyone else has filed a similar jira before filing a new one.  For the life of me, I don't understand the point of that change.  But that's a topic for another disucssion.

    Problem with "Related Items" in listings

    in Merchants

    Posted

    While updating a MP listing, I noticed that the items listed as "Related Items" on the listings edit page are not showing up when the same listing is previewed.  In fact, there is no Related Items section showing at all when the actual listing is viewed, despite the fact they are there on the listing's edit page.  This appears to be the case for a number of my listings.  However it is not consistent, i.e. the related items in some listings show up but for other listings they do not.  One example of a listing with related items on the edit page but not on the actual listing page is https://marketplace.secondlife.com/p/Subscriber-Online-Sender/3460987.

    I have tried removing and re-adding a related item on the above listing, but while it appears on the edit page, it still does not appear in the actual listing. Has anyone else seen this problem?  If so, were you able to find a cure?

    ITEMS DELIVERED & NO PAYMENT

    in Merchants

    Posted

    I was also 100% successful in getting direct payment from my customers who got both the products and the refund. A few even contacted me first to volunteer.  What great folks!  I will, of course, be sending the 5% commission to the MarketPlace (yeah, right). 

    ITEMS DELIVERED & NO PAYMENT

    in Merchants

    Posted

    Pamela, I actually had to do more than just update my Magic Box version to stop the problem. At first I just updated the scripts in the old box and re-synced.  That didn't fix the problem.  Then I replaced my old MB with a brand new one with v3.0.11, and deleted the old one.  I then visited the MP merchant home page and selected Magic Box Status.  This takes you to an old XStreet SL page that displays your inventory. If you get a pop-up about this being an inactive site, cancel the popup.  I went to the section titled Registered Servers.  It should list all of your current MB's, even the one just deleted in world.  Where it says "Force", I clicked Remove for all but the new MB.  I also clicked Update on that one. Then I went back to the Manage Listing page and clicked on Sync Magic Boxes to Markeplace button again.

    After doing all of that, and waiting about an hour, my transactions started working again and have been working ever since.  I can't tell you how much of the above was really necessary to fix the problem.  Maybe all, maybe just waiting an hour.  But for what its worth, that's what worked for me. It seems like the Magic Boxes just spontaneously get constipated at random times and stop reporting successful deliveries (or the MP randomly fails to receive or process the reports).  It seems like if you jerk them around enough, they start to work again.  Doesn't say much for the quality of the underlying software. 

    Out of curiosity, I looked back over the last several months of my MP Orders log and noticed quite a few instanceds of "Delivery Failed" and no payment transactions.  Some had following transactions where the buyer tried again and was shown as delivered, so those were likely legitimate delivery failures where the buyer got a refund but not the product.  But there were a number of others where there was no attempt by the buyer to try again, leading me to wonder whether they got the product plus a refund, or went in world to buy it, or just decided it was too much trouble and didn't bother.  There is really no way to tell without conatacting the buyers, and they probably either don't remember or won't confess their good fortune.. 

    Its unfortunate that in moving to DD to improve delivery reliability (hopefully), we also give up the current MB emails that can be used to verify that an item really was delivered, even if the the MP log shows otherwise.  Sigh.....

    ITEMS DELIVERED & NO PAYMENT

    in Merchants

    Posted

    OK, I think I've solved the problem, at least for now and for me.  I downloaded a new MB v3.0.11, rezzed it, loaded it, synced it, and deleted my older MB v3.0.7.  I then tested deliveries with an alt and confirmed they are now working normally.  I still have a couple of transaction stuck in "undelivered" or "being delivered" status.  I am also stuck with trying to contact everyone who bought an item in the last 2 days, asking them to check their transaction logs and begging them to pay me directly if they received the item and were also refunded their payment.  Hopefully most of them will do the right thing, but its a pain for both me and my customers to have to put up with this.

    I guess I'll start migrating to direct delivery.  Hope there are not a whole host of new problems waiting there.

     

    ITEMS DELIVERED & NO PAYMENT

    in Merchants

    Posted

    Sorry its been a while since I have opened a JIRA report.  I didn't realize the rules had changed for who can see a bug report.  So I guess visibility into bug investigation and fix status has gone from "not much" to zero.  Well I won't go there off topic here.  Here is some additional info on the current topic:


    Environment where my MB is located:

    Firestorm 4.2.2 (29837) Aug 27 2012 19:20:05 (Firestorm-Release)
    You are at 287,971.0, 278,690.0, 105.8 in Distorted located at sim7345.agni.lindenlab.com (216.82.34.93:13002)
    Second Life RC BlueSteel 12.09.26.265152

    The first failed delivery occurred on Oct 6, 2012 at 7:58 AM PDT.  Since then, 100% of my MP deliveries have failed in the same manner, i.e. I have not been paid and the buyer's money has been refunded.  I have personally contacted several of my customers and all have confirmed they received the item and also the refund.  I have also confimed this behavior with an alt.

    The start of the problem is roughly coincident with my adding a new product to my Magic box on 10/5.  It initially did not appear in my listing management page on the MP, however it was there the next morning so I was able to complete activation of the new listing.  Note however, that deliveries are failing on ALL of my products, not just the one added recently. Once I discovered this problem, I tried resetting my MB, but the problem continued thereafter and is still ongoing. 

    I think this covers all the questions raised on this forum topic.  If I missed something, let me know and I will provide it. I have yet to hear anything back from LL on my support ticket.

    ITEMS DELIVERED & NO PAYMENT

    in Merchants

    Posted

    For the past 2 days I'm having exactly the same problem as Pamela described. Getting delivery notices from the MB, but marketplace orders log show "Delivery Failed".  No funds transferred to my account.  Checking with those customers (about 10 so far) confirmed the item was indeed delivered and a little while later their money was refunded by the Marketplace saying the delivery failed.  No money has been credited to my account (seller).  I also filed a support ticket. 

    I even tried buying one of my own products with an alt and confirmed delivery followed by a refund.  I've tried resetting my MB and confirmed I am still having the problem.  It is still going on and I am still losing sales.  If this persist I am going to have to unlist all my items to prevent further freebie giveaways. Fortunately, some of the customers who got the items free have offered to pay me directly, but I have not been able to contact them all.  This is REALLY BAD.

    Note that prior to 10/6, MP deliveries were working fine.  Since then, 100% are failing.  I wouldn't even have noticed this if one of my customers hadn't contacted me to say they got my products free.  Since Pamela, and maybe others, have seen the same problem, I assume it may be widespread so everone should check their Marketplace orders log for a lot of failed deliveries and file support tickets to get some attetion to this problem.

    Excessive Inventory Sent

    in Inventory

    Posted

    Why am I getting the following message?:  "Second Life: Objects you own in <region name> have sent out excessive inventory and their offers have been temporarily deactivated. The last inventory offer sent was from <object name>.

    The <object name> is a mailing list object that sends out newsletters to a list of subscribed customers.  This object has been in use for years and this week was the first time it had this problem.  is there some new limit imposed on the maximum send rate for inventory items?  I am aware that there is a limit of 5000 IMs per hour, but this only applies to IMs, not inventory items.  I can find no reference to any send rate limits for the llGiveInventory() function in the LSL Portal.  If there is a new limit that has been recently imposed, what is it?

×
×
  • Create New...