• entries
    67
  • comments
    3,233
  • views
    14,263

"Re-experience the fun of customizing your Place Page!" A Tale of Oops from Ops

Linden Lab

1,887 views

Heya! April Linden here.

Yesterday afternoon (San Francisco time) all of the Place Pages got reset back to their default state. All customizations have been lost. We know this is really frustrating, and I want to explain what happened.

A script that our developers use to reset their development databases back to a clean state was accidently run against the production database. It was completely human error. Worse, none of the backups we had of the database actually worked, leaving us unable to restore it. After a few frantic hours yesterday trying to figure out if we had any way to get the data back, we decided the best thing to do was just to leave it alone, but in a totally clean state.

An unfortunate side effect of this accident is that all of the web addresses to existing Place Pages will change. There is a unique identifier in the address that points to the parcel that the Place Page is for, and without the database, we’re unable to link the address to the parcel. (A new one will automatically be generated the first time a Place Page is visited.) If you have bookmarks to any Place Pages in your browser, or on social media sites, they'll have to be updated.

Because of this accident, we’re taking a look at the procedures we already have to make sure this sort of mistake doesn’t happen again. We’re also doing an audit of all of our database backups to make sure they’re working like we expect them to.

I’d like to stress that we’re really sorry this accident occurred. I personally had a bunch of Place Pages I’d created, so I’m right in there with everyone else in being sad. (But I’m determined to rebuild them!)

Since we’re on the topic of human error, I’d like to share with you a neat piece of the culture we have here at the Lab.

We encourage people to take risks and push the limits of what we think is possible with technology and virtual worlds. It helps keep us flexible and innovative. However… sometimes things don’t work out the way they were planned, and things break. What we do for penance is what makes us unique.

Around the offices (and inworld!) we have sets of overly sized green ears. If a Linden breaks the grid, they may optionally, if they choose to, wear the Shrek Ears as a way of owning their mistake.

If we see a fellow Linden wearing the Shrek Ears, we all know they’ve fessed up, and they’re owning their mistake. Rather than tease them, we try to be supportive. They’re having a bad day as it is, and it’s a sign that someone could use a little bit of niceness in their life.

At the end of the day, the Linden takes off the Shrek Ears, and we move on. It’s now in the past, and it’s time to learn from our mistakes and focus on the future.

There are people wearing Shrek Ears around the office and inworld today. If you see a Linden wearing them, please know that’s their way of saying sorry, and they’re really having a bad day.

april-baloo-bear.jpg

Baloo Linden and April Linden, in the ops’ team inworld headquarters, the Port of Ops . 

  • Like 11