What's up with sim crossings?

[Aishagain]

59 minutes ago, animats said:

Barlow's classic business book "A Complaint is a Gift" makes a crucial point: For every customer who complains, there were 10 to 100 who silently stopped buying from you. LL needs to be reminded of that occasionally. The people here on the forums are the ones who didn't get fed up and leave.

The saddest aspect of this is that LL actually don't think this applies to them or SL or they assume there is, in their purview, an unending supply of new users.  They seem unaware that serious issues with the "world" in SL haemorrhage valuable paying users from their base and that refusal to accept that fact has wounded them on several occasions.  I hesitate to say that it would cause the death of SL but death by 1000 cuts is a distinct possibility!

[Theresa Tennyson]

28 minutes ago, Aishagain said:

The saddest aspect of this is that LL actually don't think this applies to them or SL or they assume there is, in their purview, an unending supply of new users.  They seem unaware that serious issues with the "world" in SL haemorrhage valuable paying users from their base and that refusal to accept that fact has wounded them on several occasions.  I hesitate to say that it would cause the death of SL but death by 1000 cuts is a distinct possibility!

Homeowner (to Contractor): My basement keeps leaking - you come out to fix it and it starts leaking again a few months later!

Contractor: Your foundation walls are shot - you should really consider getting a new house. They put these houses up really quickly back then and they all have issues.

Homeowner: But I love my house!

Contractor: Well, we might be able to jack up the entire house and build new foundation walls. It'd be really expensive and you'd have to move out for months.

Homeowner: But I have to live in this house, I don't have anywhere else to go!

Contractor: Well.... we might.... be able to replace the walls piece by piece, but it would still be expensive and there's no guarantee it wouldn't start leaking through the joints we'd need to put between the sections of wall.

Homeowner: Well, how much would that cost?

Contractor: (Writes up a quote.)

Homeowner: That long? That much? I can't afford that!

Contractor: Well, in that case all I can do is try to fix the leaks again.

Homeowner (a few months later): My basement keeps leaking - you come out to fix it and it starts leaking again a few months later!

__________

The very idea of region simulators/region crossings as used in Second Life is fundamentally flawed, but it's what Second Life is built on. Ultimately, anything you do with Second Life will either be "fixing the leaks" or making such radical changes that it will become unrecognizable and risk destroying 16 years of history.

[Aishagain]

Theresa

Your point is?

In this case the contractor was the original builder.  

As to the concept of simulators and crossings being fundamentally flawed...how?  I cannot reconcile the process of making the data-transfer between two simulators work properly without losses or distortion and a fundamental flaw.

Edited April 5, 2019 by Aishagain

[Theresa Tennyson]

32 minutes ago, Aishagain said:

Theresa

Your point is?

In this case the contractor was the original builder. 

In computing terms, the current "contractors" would be the grandchildren of the original builder. As of next January, three complete generations of Microsoft Windows will have gone from initial release to official "end of support" in the time that the earliest items in Second Life have continuously existed.

32 minutes ago, Aishagain said:

 

As to the concept of simulators and crossings being fundamentally flawed...how?  I cannot reconcile the process of making the data-transfer between two simulators work properly without losses or distortion and a fundamental flaw.

 

Why are there two simulators at all?

 

Edited April 5, 2019 by Theresa Tennyson

[Aishagain]

2 hours ago, Theresa Tennyson said:

In computing terms, the current "contractors" would be the grandchildren of the original builder. As of next January, three complete generations of Microsoft Windows will have gone from initial release to official "end of support" in the time that the earliest items in Second Life have continuously existed.

 

I thought SL's server software was Linux-based and wasn't subject to the Windoze generation issue, but I am not certain of that. In any case, your logic regarding who made SL is flawed on that point, Windows did not make SL, LL compilers did.  And LL is still the contractor and builder.

 

2 hours ago, Theresa Tennyson said:

Why are there two simulators at all?

 

Er, you mean that SL should be just one single simulator?  I doubt that that would even be possible, and it certainly would not be desirable.

[Theresa Tennyson]

56 minutes ago, Aishagain said:

I thought SL's server software was Linux-based and wasn't subject to the Windoze generation issue, but I am not certain of that. In any case, your logic regarding who made SL is flawed on that point, Windows did not make SL, LL compilers did.  And LL is still the contractor and builder.

I was referring to the concept of generations of software in general. Actually, Second Life just updated to the sixth generation of Debian (the Linux distribution that the servers use) released since Second Life was in beta. When Second Life was created some of the beta testers were connecting by dial-up modem. I used to work at an architecture firm that turned 100 years old while I was there, and worked on buildings that the same firm designed decades ago but everyone who actually made the decisions on those buildings originally was dead. A 16-year-old software application is similar to a 100-year-old building.

56 minutes ago, Aishagain said:

Er, you mean that SL should be just one single simulator?  I doubt that that would even be possible, and it certainly would not be desirable.

Is it desirable that one avatar on an extended drive uses one-hundred and nine separate, permanent simulations? Why not just simulate the avatar instead of the land, and have each avatar's simulation start and end on log-in and logoff?

I know of no other online 3d environment that isn't a deliberate clone of Second Life using the "grid" approach - there may be a good reason for that.

Edited April 6, 2019 by Theresa Tennyson

[Fluf Fredriksson]

And now back to the current problem. Which is still weirdly random. Some night's I can slide though literally 100's of sim crossings with no problems. A couple of nights later I'm back to "PING!". And this is with no windlight enabled at all. I'm sort of praying this is because we have a "maybe" bug fix on some servers and a "not fixed at all" bug fix on a bunch of other servers, but even so it doesn't make much sense.

For all the armchair critics out there, sim crossings (and teleports to a degree) have always been a special issue with SL. It is nothing like as easy as you might assume it is. You're trying to move a 100% customisable avatar from one (nearly) 100% customisable sim to another, all the time you are trying to keep IM communications / group chats running smoothly along with a bunch of scripts. It's not at all comparable to "my MMO can do it! why can't SL?". Yes in theory it could all be re-written from the ground up, but you're looking at dev time in the 100's of hours to do that ("big cost"), and when it's working mostly ok, it works mostly ok as it is. So why pay the big cost? It looks bad at the moment because EEP (and I still think possibly other changes), have screwed up the baseline of "it mostly works ok". When it (I pray it) .. gets back to mostly working ok, nobody would dream of investing 100's of hours of dev time in to fixing it. Ideally maybe, but this isn't an ideal world. Sure. Politely nag the Linden's. I think they have bigger problems though (and they won't get addressed either).

EDIT: Thinking about it, 100's of hours is way too low, it's in the 1000's of hours.

Edited April 6, 2019 by Fluf Fredriksson

[KT Kingsley]

On 4/4/2019 at 1:26 PM, MBeatrix said:

Region name and approximate time, please? That is valuable info for LL, as they can check the log...

TP out of Coconut Island to Endless at approx 21:45 PDT Friday 5th May, failed. Relogged to Last Location. Message: Login failed. Agent not in region. Please try again in a minute. (Or words to that effect.). Relog returned to where I'd last logged into after a normal SL exit (which must be a few days ago by now).

Edited April 6, 2019 by KT Kingsley
date & destination

[animats]

On 4/2/2019 at 12:46 AM, MBeatrix said:

 Let the Lindens do their job, which means testing stuff properly — that's what we pay them for.

Anybody want that job? LL is hiring for it.

"The QA Engineer will perform testing across Second Life services (mostly Server, some Viewer & Web) and related client applications in an expansive system architecture. The ideal candidate will have experience in testing of software applications and client/server integration. This person will be highly innovative in testing strategies, as there is a continuous need for original and forward thinking with respect to the evolving needs of the products and internal systems as we move Second Life core services into the cloud."

[MBeatrix]

5 hours ago, Fluf Fredriksson said:

And now back to the current problem. Which is still weirdly random. Some night's I can slide though literally 100's of sim crossings with no problems. A couple of nights later I'm back to "PING!". And this is with no windlight enabled at all. I'm sort of praying this is because we have a "maybe" bug fix on some servers and a "not fixed at all" bug fix on a bunch of other servers, but even so it doesn't make much sense.

For all the armchair critics out there, sim crossings (and teleports to a degree) have always been a special issue with SL. It is nothing like as easy as you might assume it is. You're trying to move a 100% customisable avatar from one (nearly) 100% customisable sim to another, all the time you are trying to keep IM communications / group chats running smoothly along with a bunch of scripts. It's not at all comparable to "my MMO can do it! why can't SL?". Yes in theory it could all be re-written from the ground up, but you're looking at dev time in the 100's of hours to do that ("big cost"), and when it's working mostly ok, it works mostly ok as it is. So why pay the big cost? It looks bad at the moment because EEP (and I still think possibly other changes), have screwed up the baseline of "it mostly works ok". When it (I pray it) .. gets back to mostly working ok, nobody would dream of investing 100's of hours of dev time in to fixing it. Ideally maybe, but this isn't an ideal world. Sure. Politely nag the Linden's. I think they have bigger problems though (and they won't get addressed either).

EDIT: Thinking about it, 100's of hours is way too low, it's in the 1000's of hours.

So, instead, is it better spending those hours adding new features that eventually will work only more or less OK and may collide with the existing, messing it for a lot more hours? Of course that SL is a very complex system, and it's why serious consideration should be taken before adding new features.

Saying that it's a too old, too complex system to fix the existing can't be entirely true, it's just an excuse for not doing what should be done, which is having it work better than only more or less OK half of the time. If the existing is too old, why insisting in adding even more complexity to it? Will new features sell a product better if it only works more or less OK half of the time? Possibly, but not at all in the long term because at some point most people will feel they had been cheated. Cosmetics may help hiding the outer appearance temporarily, but they do not cure existing diseases.

[Qie Niangao]

6 minutes ago, MBeatrix said:

Saying that it's a too old, too complex system to fix the existing can't be entirely true, it's just an excuse for not doing what should be done, which is having it work better than only more or less OK half of the time.

It's possible that the cure could be much worse than the disease. Theoretically, fixing this for real might mean making a change that precludes any open-source viewer, meaning Firestorm and all other TPVs would simply be impossible. Would that be an acceptable change? (Even if I'd be fine with that, can you imagine the disruption and drama? It's hard to envision SL surviving.)

That's hypothetical, but not entirely. Suppose Theresa's new architecture were necessary: to have a shared physics experience among such interacting situated simulations, there'd need to be a whole lot more per-session processing, and to keep lag even remotely tolerable a bunch of that would have to migrate to the viewer. Maybe that sort of distributed, multi-stage physics engine could be open-sourced or licensed for unlimited distribution -- or maybe not.

(FWIW, I'm not convinced that SL's grid of situated physics simulations is such a dead end. When I try to think about instead distributing the physics simulation among processes corresponding to each interacting agent, I'm hard-pressed to see it as an easier problem. Maybe that's all somehow been solved for other platforms, though; I have no idea.)

[Fluf Fredriksson]

1 hour ago, MBeatrix said:

So, instead, is it better spending those hours adding new features that eventually will work only more or less OK and may collide with the existing, messing it for a lot more hours? Of course that SL is a very complex system, and it's why serious consideration should be taken before adding new features.

Saying that it's a too old, too complex system to fix the existing can't be entirely true, it's just an excuse for not doing what should be done, which is having it work better than only more or less OK half of the time. If the existing is too old, why insisting in adding even more complexity to it? Will new features sell a product better if it only works more or less OK half of the time? Possibly, but not at all in the long term because at some point most people will feel they had been cheated. Cosmetics may help hiding the outer appearance temporarily, but they do not cure existing diseases.

I said it could be done, but it won't be done. It's expense not age or complexity. Yes the EEP roll out is a complete and utter failure, but it's done to generate more revenue for LL (by selling lighting environments). When your project is driven by accountancy then trying to generate more money will always come before spending money. You think SL is messed up because of it? Look at the real world some time. Same problem.

[MBeatrix]

1 hour ago, Qie Niangao said:

It's possible that the cure could be much worse than the disease. Theoretically, fixing this for real might mean making a change that precludes any open-source viewer, meaning Firestorm and all other TPVs would simply be impossible. Would that be an acceptable change? (Even if I'd be fine with that, can you imagine the disruption and drama? It's hard to envision SL surviving.)

You have a good point there. Maybe.

I'm sure you noticed before that I don't have the technical knowledge — not even close — to discuss any of what you wrote, but I understand the general idea. Like many thousands of people who struggle in/with SL, I am a normal user who pays for a product and is tired of the degraded experience offered in return, and sees its manufacturer covering it up with powders and lotions hoping to attract new customers and/or keeping the existing distracted for a little longer.

I have been in SL for a bit more than 7 years now. Does it look better? Yes, it does look much better than it did when I joined, there is no doubt about it. Does it WORK better? No.

I believe that SL is still alive because its user base average age is quite high, possibly around 50 years old, and some are even disabled. Young people have no patience for SL, and there is so many offer around... Youngsters, will try it for a while and move on to the next game as soon as they experience the first serious issue. Yes, for them SL is just one more game... And I believe that is why SL has been slowly declining.

Has LL ever given any deep thoughts about it? About a working commercial strategy? I'm sure they did, but they should have noticed at some point that their initial one stopped working and they should do what must be done. And what must be done is not Sansar, not for the next 10 years. Simply because in 10 years those who are now 50 years old will be 60, and surely most won't be interested in starting it all over again — I guess they (me) will resume to watch tv... As for young people, Sansar is just a weird product among thousands of cool games. Yes, those who now are young will be less young in 10 years so maybe Sansar will "capture" some of them. But the thing is, can it survive for 10 years? And who knows what will happen during that period of time?

Has LL put themselves in my "shoes" and take the approach of a dumb user who thinks it's fun to put some prims together and build stuff or just need a little fantasy in their boring life but what they get is just an hard time trying to do simple things like buying clothes that make the avatar look cool? Or try things they would never do in the real world, like sailing or flying aircrafts? I'm sure they did in the beginning, and that's why SL was created, but I doubt they have for years now. So SL mostly became a bunch — a large bunch — of frustrated stubborn people who insist in hoping that some day SL will work better than more or less OK half of the time. What is LL going to do about that hope? Keep frustrating it until there is no one left?

[EDIT] Progressively moving SL to the cloud can be the opportunity to finally replace obsolete stuff by what may work better than more or less OK half of the time. See? I'm hopeful. Hopeful but not really believing it.

Edited April 6, 2019 by MBeatrix
additions

[MBeatrix]

Just now, Fluf Fredriksson said:

I said it could be done, but it won't be done. It's expense not age or complexity. Yes the EEP roll out is a complete and utter failure, but it's done to generate more revenue for LL (by selling lighting environments). When your project is driven by accountancy then trying to generate more money will always come before spending money. You think SL is messed up because of it? Look at the real world some time. Same problem.

We can't live without the real world, so we have to deal with it. We can live without SL, though... 😎

[Qie Niangao]

7 hours ago, Fluf Fredriksson said:

Yes the EEP roll out is a complete and utter failure, but it's done to generate more revenue for LL (by selling lighting environments).

I guess. But still, EEP is really what we were promised we'd get back when the Lab bought Windward Mark in 2007. I'll bet there's still a Torley video somewhere raving about how cool it will be Real Soon Now (years ago) when we'll be able to exchange windlight settings as first-class inventory assets. If one waits long enough, everything will happen.

[Fluf Fredriksson]

19 minutes ago, Qie Niangao said:

If one waits long enough, everything will happen.

 

Only if it looks like it has the shortest pay back time at the meeting though :)

[MBeatrix]

Just one final note to close my chapter, in case someone thinks LL has a good commercial strategy...

A good commercial strategy isn't only about good marketing, it's that combined with a solid product. The marketing is there but SL is anything but a solid product. Happy customers sell more than any marketing campaign costing loads of money. Good marketing only attracts people to try a given product but if what they get isn't exactly what they expected when they stumbled on the advertisements, they will not only drop it as they also will tell their friends it's not worth even trying.

[Whirly Fizzle]

 

[Aishagain]

Darn Whirly beat me too it.  Again.  Quick off the mark that Whirly!

[JoyofRLC Acker]

On 4/4/2019 at 10:51 PM, animats said:

"Don't half-heartedly wound problems - kill them dead." - Kelly Johnson, head of the Lockheed Skunk Works, designer of the U-2 and SR-71 spyplanes. What he meant by that is, if it's not working, don't go for a fix that makes it fail less often. Redesign it so it doesn't fail.

The Linden Lab approach to region crossing problems has been to make some minor change to timing or message order that makes the problem behave slightly differently, then ignore it until users complain again. This approach has repeatedly failed. For years. This is a track record of sustained failure.

Linden Lab needs to build enough tools and develop enough understanding that all region crossing problems are caught, logged, analyzed, and a permanent fix that always works developed. Nothing short of a sim crash or total loss of communications with the viewer should cause a sim crossing failure.

The ongoing failure of Linden Lab to fix this problem properly devalues the whole big-world concept of Second Life. We cannot have routine group activities that span regions. Thousands of customers have boats, aircraft, and road vehicles which can seat multiple avatars. Yet we cannot routinely get a group of people together and take a trip without the risk of something failing badly and breaking immersion.  So most of those big yachts stay docked. Group tours are rare. Merely driving around is difficult. Users retreat to private islands or their own little spaces to avoid the risks of venturing out in the big world.

We, as users, need to keep the pressure on Linden Lab and its management to fix the problem. This needs to be brought up by many people whenever Linden Lab CEO, Ebbe Altberg, speaks in public. Fixing this problem properly may require substantial resources. That's why it needs to be addressed at the CEO level.

Here is Mr. Altberg's Twitter feed.

Remember that what they built was a basically a shopping mall.  256 sq m sims and badly kludged logic for moving vehicles across was clearly never meant to be a full world simulation.  Its an architecture issue as far as I can see.

[JoyofRLC Acker]

On 4/6/2019 at 5:16 AM, MBeatrix said:

You have a good point there. Maybe.   ........

....

[EDIT] Progressively moving SL to the cloud can be the opportunity to finally replace obsolete stuff by what may work better than more or less OK half of the time. See? I'm hopeful. Hopeful but not really believing it. 

I would not bet against the move to cloud servers being part of the problem.  Its hard to see how it could improve latency in server handoffs.   (sorry I cant find the irony font).   iirc at one point they got some improvement in Blake Sea by co-locating the Blake servers.  I guess thats out the window now.

[Qie Niangao]

24 minutes ago, JoyofRLC Acker said:

I would not bet against the move to cloud servers being part of the problem.

What problem? The sims are nowhere near ready to be rehosted on cloud servers, so sim-crossing handoffs are still the same as always (modulo whatever sim-to-viewer messaging screw-up is causing the current disconnects). Well, unless we make up some weird dependencies... like if for some reason handoffs can't complete without Presence giving the go-ahead, or something similarly unlikely. Hard to put much value on such speculative straw-grasping, though.

[JoyofRLC Acker]

Thanks for the clarification, I had read somewhere that the move to cloud servers was under way.

I find the log files almost impossible to read but the blizzard of timeout  & not found warnings cant be good.  Its at the point where just writing all the error messages must take a fair bit of resource!!

[JoyofRLC Acker]

On 4/5/2019 at 3:27 PM, Theresa Tennyson said:

Homeowner (to Contractor): My basement keeps leaking - you come out to fix it and it starts leaking again a few months later!

Contractor: Your foundation walls are shot - you should really consider getting a new house. They put these houses up really quickly back then and they all have issues.

Homeowner: But I love my house!

Contractor: Well, we might be able to jack up the entire house and build new foundation walls. It'd be really expensive and you'd have to move out for months.

Homeowner: But I have to live in this house, I don't have anywhere else to go!

Contractor: Well.... we might.... be able to replace the walls piece by piece, but it would still be expensive and there's no guarantee it wouldn't start leaking through the joints we'd need to put between the sections of wall.

Homeowner: Well, how much would that cost?

Contractor: (Writes up a quote.)

Homeowner: That long? That much? I can't afford that!

Contractor: Well, in that case all I can do is try to fix the leaks again.

Homeowner (a few months later): My basement keeps leaking - you come out to fix it and it starts leaking again a few months later!

__________

The very idea of region simulators/region crossings as used in Second Life is fundamentally flawed, but it's what Second Life is built on. Ultimately, anything you do with Second Life will either be "fixing the leaks" or making such radical changes that it will become unrecognizable and risk destroying 16 years of history.

Its not fundamentally flawed if you are building a shopping mall / nightclub, which is what I suspect the original idea was.  This architecture is bonkers for anything like flying or sailing etc.  256m "regions"  - come on!

[Sharie Criss]

the 256m limit is more of an LL configuration setting, just like the prim limits. That said, everything has been built with that size in mind, so allowing larger sims would have implications for scripts, the world map, and pretty much everything else. The simulator software itself would probably have no issues with larger regions, it's just everything ELSE that would.