Deploys for the week of 2012-12-10

[Maestro Linden]

Second Life Server (main channel)

Due to an issue discovered last week in the RC channels around llInstantMessage() to object owners (BUG-1002), there is no new code eligible for the main channel this week, so there will be no roll.

 

Second Life RC BlueSteel and Second Life RC LeTigre:

BlueSteel and LeTigre are getting the same maint-server project that Magnum had last week.  The only changes are fixes for bugs that were discovered in RC:

https://wiki.secondlife.com/wiki/Release_Notes/Second_Life_RC_BlueSteel/12#12.12.10.267911

https://wiki.secondlife.com/wiki/Release_Notes/Second_Life_RC_LeTigre/12#12.12.10.267911

Scheduled Wednesday 2012-12-12 07:00-11:00 PST

 

Second Life RC Magnum:

Magnum is getting a maint-server project which is a superset of the changes scheduled for BlueSteel and LeTigre.  Magnum mostly includes extra bug fixes, including stability improvements and a memory leak fix.  The only new feature new to Magnum is an increase in the allowed animation asset size.

https://wiki.secondlife.com/wiki/Release_Notes/Second_Life_RC_Magnum#12.12.10.267913

Scheduled Wednesday 2012-12-12 07:00-11:00 PST

 

We will be monitoring this thread as the code gets released, so feel free to note any observations you have about the server updates.  If you have a specific bug you'd like to report, please file a Jira. 

 

[Whirly Fizzle]

---

Maestro Linden wrote:

The only new feature new to Magnum is an increase in the allowed animation asset size.

 (The 60kB size limit on

animation

assets has been raised to 120kB.)

---

Does this allow upload of animations longer then 30 secs?

[Maestro Linden]

---

Whirly Fizzle wrote:

Does this allow upload of animations longer then 30 secs?

---

Hi Whirly, this change doesn't allow animations longer than 30 seconds by itself (I believe this is a viewer-side limit), but it paves the way for longer (or more complicated) animations.  But there is an upcoming viewer change to increase the maximum animation duration to 60 seconds.

[Whirly Fizzle]

Oh that will be brilliant. Thanks Maestro!

[Perrie Juran]

---

Maestro Linden wrote:

---

Whirly Fizzle wrote:

Does this allow upload of animations longer then 30 secs?

---

Hi Whirly, this change doesn't allow animations longer than 30 seconds by itself (I believe this is a viewer-side limit), but it paves the way for longer (or more complicated) animations.  But there is an upcoming viewer change to increase the maximum animation duration to 60 seconds.

---

While you (or they) are at it, can we get an increase in the length of a sound file? 

10 seconds is so last century. 

[Elin Egoyan]

Since  3 days I have commplains of my tenants who live on one of the grandfathered homesteads in our Estate.

Here a quote:

 I log in and none of my scripted attached itmes are loaded and the msg says the sim is not accepting them, it happens when i logon.  When i log into a different location I am fine.

 

I wonder if this is because of the issue mentioned in this tread or  is it something else.?

I have restarted the sim but it just solved it for a little while. The ping is oke.  There are just 137 scripts running on the sim and  the lagmeters is green and bright. Perhaps  you can  help me with suggestions?

[Whirly Fizzle]

What is the name of the affected region?

If you bring up the stats floater with CTRL+SHIFT+1 and look at the "Memory allocated" reading under "Physics details" what is the reading when the region is having this problem?

If it is in the high 200's the region may have disabled rezzing until it is restarted.

What is the exact message the tennants get when .. "the msg says the sim is not accepting them, it happens when i logon" ?

 

[Trinity Yazimoto]

so can you tell more precisely when the bug about the scripted object not sending emails will be fixed, please ?

i dont recieve any of them since more than one week now and this starts to be really annoying for my business. 

So ill be really glad if the fix happens really soon. Thanks

[Elin Egoyan]

Whirly, ty for  trying to help me out. 

This concerns the  Serendipity  Vista Estate (Estate ID 14836) , the sim is called Serendipity Acres III. Gridposition 763/1264

The message the tenant have send me is as follows:

 I have logged in several times today 12/10 when i do all of the scripted attachments as well as mesh items I have worn or attached to my avi were dropped off, and in local chat we got the following msg for each item:   Unable to create item '' that has caused problems on this region
Too many errors... dropping further messages until the flood stops..

Another tenant  have send me this message:

I keep getting the following error message when trying to rez anything more than a prim at my parcel on Serendipity Acres III: [19:05] Second Life: Unable to create item that has caused problems on this region.

 

When I look at the Memory Allocated/Physics Details it tells me 26.5 MB. The problems occure when I am in bed:)) ( European timezone) The tenants are in the American time zone. So I could not check the Physics detail when the issue was there.

I hope you can do something with this knowledge:)

[Trinity Yazimoto]

oh awesome ! it works again ! i recieve again emails from my scripted objects in magnum region... thanks thanks thanks !

[Maestro Linden]

---

Trinity Yazimoto wrote:

so can you tell more precisely when the bug about the scripted object not sending emails will be fixed, please ?

i dont recieve any of them since more than one week now and this starts to be

really annoying

for my business. 

So ill be really glad if the fix happens really soon. Thanks

---

Hi Trinity, per my post to this thread and the release notes, the fix for this bug was/is being deployed between 7-11AM PST today.

[Maestro Linden]

---

Elin Egoyan wrote:

Whirly, ty for  trying to help me out. 

This concerns the  Serendipity  Vista Estate (Estate ID 14836) , the sim is called Serendipity Acres III. Gridposition 763/1264

The message the tenant have send me is as follows:

 

I have logged in several times today 12/10 when i do all of the scripted attachments as well as mesh items I have worn or attached to my avi were dropped off, and in local chat we got the following msg for each item:   Unable to create item '' that has caused problems on this region

Too many errors... dropping further messages until the flood stops

..

Another tenant  have send me this message:

I keep getting the following error message when trying to rez anything more than a prim at my parcel on Serendipity Acres III: [19:05] Second Life: Unable to create item that has caused problems on this region.

 

When I look at the Memory Allocated/Physics Details it tells me 26.5 MB. The problems occure when I am in bed:)) ( European timezone) The tenants are in the American time zone. So I could not check the Physics detail when the issue was there.

I hope you can do something with this knowledge:)

---

Hi Elin, I would recommend asking your tenants to check the physics memory stat when they see the rez failures.  Since 'Serendipity Acres III' is a homestead region, you would be hitting a known bug if the stat is greater than 230MB.

 

[Elin Egoyan]

TY Meastro:).  Can you tell me what can be the cause that the Physic details raise? Sorry it is perhaps a stupid question, but I never encountered such issue before

 

[Maestro Linden]

Hi Elin, it seems to happen sometimes following rapid navmesh rebaking.  One of the bugfixes in Magnum was to elminate a memory leak around this process, so we're hopeful that the issue will become more rare.

[Alexi Raynier]

Maestro, we are encountering another bug in the Blue Steel RC regions that has been seen on the main servers in the past but not in the Blue Steel areas, and whose appearance is directly associated with the last restart.  What is happening is that, after a login in or teleport entry, or anything that seems to cause serious rezzing like camara movement, fps ofter fall rapidly to the point where they reach a crippling 1 or less fps.  This is a totally new problem in this area and until the restart the Blue Steel regions have been of late models of efficiency, now though they are turning into lag hell and this problem needs to be carefully addresses, before people give up entirely and sell of abandon their land, which is fast becoming useless and which, since this is RC area implies that if this bug is advanced back to the main servers they too will suffer the same affliction.  Apparently this has resulted from a timing adjustment in either the local servers or the main server and needs to be looked at and fixed as quickly as possible before like the navmesh issue it becomes overwhelming.

 

[WolfBaginski Bearsfoot]

I experienced a similar problem a few months ago, which I reported on the new JIRA as BUG-28. Physics frame rates were running at about 8fps, which is worrying but understandable in complex situations. Trouble was, I saw bursts of 0.3fps, and I wasn't happy to be told it was "expected behavior".

If this really is something new, something you haven't seen before, I hope it doesn't get the treatment my report got. If 0.3fps really is "well within the expected range", we have a deeper problem than just a change in recent code. But I suppose I shouldn't expect a complete re-design and re-write of the servers, even though there could be a lot of crud buried deep in the system, left over from the early days.

 

[Ayesha Askham]

Just a small addition to this issue discussed in the last two posts.  I was on a Bluesteel server last night and tried to refresh a texture.  I was instantly reduced from over 20fps to less than one fps and there was no improvement over more than 30 seconds at which point I teleported out of the sim.

Upon return to the sim I found the performance back to normal, so I must assume that this issue is concerned with sim-to-viewer communication.

I did not check the stats during this issue, I was so surprised by it, but looking at a thread started by Toysoldier Thor back in September suggests to me that this is either a resurfaced bug or one that has arisen lately, but BUG it certainly is.

That thread is now buried on page 3 of this forum, but is still VERY active:

http://community.secondlife.com/t5/Second-Life-Server/Increase-in-Instant-SIM-LAG-amp-Crashes-During-Larger-Events/td-p/1683765

Whether these issues are related or not I do not know, but problems with server-viewer communication certainly seems to be a common factor, here and in several other issues.

[Alexi Raynier]

This is a bug that has been around awhile on the main servers, and appears to be more a issue of server settings than actual code problems, since one sim will have it and another not.  What is alarming now is that it is so wide spread that it is appearing across the Blue Steel servers which until this last restart were a model of stability.  Morever, since this restart featured more work on the servers than actual code changes it seems reasonable that some of that work is badly in need of review before this issue is spread throughout the main servers and promoted. 

As of it being ignored LL can ignore it at its own peril for if it becomes widespread I have no doubt that I and many other land and business owners will be leaving as our land and businesses will have been rendered unusable.  Least anyone think this an idle threat I advise they compare online numbers over the last year and land ownership, both of which have dropped dramatically as SL performance has become increasingly less stable.  If this bug or "expected result" is introduced on a widespread basis to the main servers then it may well be the final straw and deciding factor in an number of people and businesses leaving for good and LL can only blame it on themselves and their own arrogance.

[WolfBaginski Bearsfoot]

Alexi, I don't know if people will quit SL over it, but for this to be dismissed as "expected behaviour" prompts me to wonder whether it is the sims or the Lindens that are broken.

The grid is knee-deep in Christmas parties, and any time a region gets more than about thirty AVs in it, it looks as though people start losing control of their motion. They bounce off in odd directions, start rubber-banding, and sometimes vanish for no apparent reason.

It wasn't this bad last year and most of the problems seemed to start when Pathfinding turned up. Coincidence? Expected behaviour? 

 

[MB Robonaught]

Hi Maestro,

I wanted to pass along some data I've been watching the last couple months on my Magnum sim. Overall performance for things like lag, rezzing and other normal operations has been really good. However,there are a couple items I've been seeing often that I should pass along.

First is our old friend the prim shift is back with a vengeance affecting both linked prims moving in a link set and complete linksets moving on their own.

Second is I'm noticing a high rate of prim corruptions where scripts within will no longer function or recognize script editing without replacing the prim. This happens more often in corrupting the root prim with scripts then linkset prims containing scripts. I've also have been having trouble with scripts reverting to a previous version after a restart.

I'm still trying to find a way to duplicate these in order to fill a JIRA however no luck so far. I have noticed thought, that these both seem to happen after a sim restart. While these two items may seem to be low priority they do leave me looking like a sloppy builder that can't script to my customers. Please see what can be done.

Thanks!

MB

 

[Alexi Raynier]

In other words Wolf, LL has a major mess and judging by what I have seen today with the apparent lead up to this weeks resets it is about to get better or else to be chairitable they are pushing the system to the limits, to determine how are they can go and will return to a more sustainable mid point in the settings.  But for now SL, both in the main servers and the RC servers has been peppered with lag and lag induced crashes, very low frame rates, high server ping and huge packet loss not to mention, the return of creeping memory levels and the rebake problem we had earlier.

This does not bode well, nor does the fact that Maestro has made no comment upon this issue although until it was raise he was very active and helpful in the forums.  All of this is causing to think that these problems may be anticipated as you suggest and that LL in its arrogance thinks that players will continue to put up with the constant instability and dimishment of service and lack of information.  To that I suggest that they do so at their own peril because player patience is already stretched to the breaking point and one more crisis my prove to be just too much.

[Cincia Singh]

I think it would help if people turned down the rhetoric a notch or two because SL is running far better than it did at any time in it's past (I've been in SL daily since 2007) and I expect it will continue to improve as time passes. Sure, we'll have glitches as new features and options are introduced. Changing the SL server code (or viewer code for that matter) isn't like editing a word document ... it's incredibly complex and intertwined with a lot of unexpected results from seemingly minor code changes.

I live on the mainland. We have a club on the mainland occupying 1/2 of a main-channel server region that routinely has more than 30 guests during events and we frequently bump up against the 40 avatar limit. I also have land in a Magnum region of the mainland. We've had no difficulties other than the expected lag when LL doesn't restart regions weekly (and then we have Live Chat restart the region for us). We're not special, we just haven't seen the issues that are recently described here.

[Alexi Raynier]

To start with I have been in SL for well over six years and run successful businesses for most of that period, and far from getting better SL performance has steadily fallen for over three year now.  This by the way is clearly reflected in online number which over the last year have fallen from over 60k to under 50k.  Moreover, the number of new players has fallen off as well along with the retention of veteran players and business owners.  This is apparent both from looking at the sl performance number and from looking at my inventory where I find I own literally thousands of high quality items by shops that have left sl and not be replaced.  This though is not the crux of the problem what needs to be addressed is performance and after last weeks restart overall performance had drastically decreased with a resulting drop in online numbers that reflect the increasingly unstable service.

Now lets look at where the problem lies.   It does not appear to be in the viewers, and even the sl viewers appear to be doing a decent job, nor does it lie in our service providers because this problem is far too widespread to be attributed to service providers unless you wish to take the position that internet service in the US in Europe has suddenly declined to Third World standards. Likewse it does not appear to be in the code LL introduced last week for by there own admission very little code was added last week and it only to fix know bugs and was I assume well tested before installation.

No the problem appear to lie in the regional servers and in their ability to communicate with the core servers themselves and largely appear to be problems of timing where requested procedures time out before they are executed and get lost in our system cache to fill up our memory and choke our computers.  Time after time we see request for excecution sent to the regional server, be they the opening of inventory, asking for a group IM, a simple IM, the saving of a record or photo or any of a hundred other possible thing, and either the request not be executed or there be a huge delay in execution, or in cases such as a teleport a crash.  By LLs own admission maintenance was done last week in place of the restarts, and it apparently resulted in a great many problems as well as the reintroduction of a number of apparently fixed bug such as repeated and unasked for regional rebakes, gradual creep in allocated memory, massive server ping and bandwidth usage, and despite a relatively low fps massive and crippling periodic lag that can only be fixed by porting out of a sim or restaring your avi entirely.  These things are occurring to people all over SL have been reported and LL has yet to acknowledge, though we have been assured by at least one self appointed cheerleader that things are getting better and better with SL. 

The truth is they are not getting better, true we have short lulls like over the last few weeks when we have relatively good performance, but in the end things always revert back to the same pattern that SL has followed for years now and we see the same bugs, like" the region is logging you out crash" that we have been seeing for months and year, and as a result more and more businesses close and more and more people leave SL, and the new players that come in to replace them are disgusted by the instability as well and leave with them.  This makes no business sense and if these problems are anticipated speaks of a depth of technical inability that is truly staggering.  In short it is time for LL to face reality and either make a decision to fix this mess even if it means removing some of their favorite additions or admit they cannot or will not fix it and let it die.