Jump to content

OMG! What happened to my listings!? (Information for merchants with missing listings)

Chaser Zaks

By Chaser Zaks,
in Merchants

 Share
You are about to reply to a thread that has been inactive for 84 days.

Please take a moment to consider if this thread is worth bumping.

Recommended Posts

Chaser Zaks

Chaser Zaks

Posted
Posted (edited)

Whoops! Looks like the marketplace is having issues. I originally posted this in the Discord, but since not everyone is in it, I figured I would post it here so people can get a link to it.
You can track the status of this issue here: https://status.secondlifegrid.net/incidents/dtkx144qf150

Q & A:

Question: Are my listings gone?
Answer: No, your listings still exist! They are just being a little shy right now.

Question: Do I need to recreate my listings?
Answer: It is very unlikely you will need to do this. Your listings should re-appear once Linden Lab fixes the issue. If they do not re-appear once the issue tracked on the status page is said to be resolved, please open a support ticket at https://support.secondlife.com.

Question: Should I recreate my listings for now, or make new listings and delist the missing ones when they re-appear?
Answer: I'd recommend against it. You will lose reviews, number of purchases(Statistic), and also create issues for customers who have bought the currently unlisted listing(Such as marketplace redelivery, issues with duplicate entries on redelivery systems / ANS, etc). The negatives outweigh the benefits in regards to recreating a new listing. Best solution is to just wait for Linden Lab to resolve the issue.

Question: Is it safe to create new listings on the marketplace right now?
Answer: It should. But due to my limited ability to see what is going on, I'd personally recommend on holding off on making new listings until the issue is resolved. This would especially be so in the unlikely scenario a database rollback is required, in which, any new listings would be removed and have to be re-created. Though this is unlikely.

Question: Can customers still buy my products that are not visible at the moment?
Answer: Yes! See section below!

Question: Is Linden Lab aware of this issue?
Answer: Yes! And they are already working on a fix!

Question: Can I be notified when the issue is resolved?
Answer: Yes! Go to https://status.secondlifegrid.net/incidents/dtkx144qf150 and click "Subscribe to updates"! You will be emailed when the issue is updated or resolved.

 

How can customers still buy my items right now?

You may have customers who wish to buy your items which are a bit shy at the moment, you can use these steps to give access to said listings to customers:

  1. Log into the viewer.
  2. Open your marketplace listings floater (Avatar > Marketplace Listings...).
  3. Right click a listing in question and select "Edit Listing".
  4. From the web browser that opens, look at the URL. You will see something like https://marketplace.secondlife.com/merchants/326458/store/products/6397940/edit.
    Copy the last number in the URL. In my case, it would be "6397940". This is your listing's unique ID.
  5. Add that number to the end of this URL: https://marketplace.secondlife.com/p/product/
    In my case, it would look like https://marketplace.secondlife.com/p/product/6397940
  6. Visit your crafted URL to ensure that it directs to the correct listing. If it does, you have succeeded in crafting the direct link to your product's page! You can share this link with customers who wish to buy a product that is currently not showing up due to technical issues.

 

Why are listings being shy?

While I do not know the exact reason, I have reasons to believe it is due to a programming error which affects search results only. There is a chance that it has something to do with the end of the year, but that may just be coincidence.

When I say search results, I mean any page that lists multiple products. This is because behind the scenes, the front page, manage listings page, store front, and search results are technically all search results, just made more human friendly(Yes, even if there is nothing in the search bar!). Think of visiting your store front more like running a search query for "Show me listings made by <name here>" on store fronts, or "Show me listings I created" on the manage listings page.

 

When will Linden Lab fix it?

They are already on it! It'll take some time to isolate the issue, craft a solution, test, and then roll out. Rest assured, it will be fixed.

 

I am a nerd, I want to know whats going on behind the scenes.

⚠SPECULATION WARNING⚠ 
Speculation means that IT IS NOT CONFIRMED. This is just educated guesses on what MAY have caused the issue. Please do not take this or claim that this is the actual issue. 

 

Possible issue #1:
My assumption is that the search query is probably trying to hide items that are old/have gone unpurchased since a period of time (I.E. the recent-ish "Marketplace cleanup" thing where old things got delisted), but something has caused it to go wonky because of the end of the year.
Since all queries are based off models(Marketplace is a Ruby on Rails app and is using MVC design pattern), it is likely that something on the search query constructor added in that condition in a way that doesn't quite fit scenarios where it is December 31st. It is entirely possible that date math(As a programmer, date math is terrible. It took me 3 HOURS to get daylight savings time code working properly) got screwed up, and it is inserting a invalid date, and MySQL isn't quite happy about that and it isn't returning all results. Though it could be any variety of date math things, such as leap years, daylight savings time, or time zones.

Possible issue #2:
Search engine has become de-synced. When you have millions of entries with thousands of requests per minute, you don't want to query the database with a bunch of "AND" statements. You can, but it will become really slow, and slow down other operations on the database. So developers will often employ a "Search Engine" as a middle man. The search engine will populate it's self with entries from the database, and instead of asking the database, which is good for storing data and looking up something by a primary key, a search engine is better at searching large number of fields and finding matches.
Often, it will be Elasticsearch or Opensearch. Both are industry leaders in search technology, and they are self hosted(meaning, they run on LL's servers, they aren't something that is sent to someone else's servers). If they become desynced, they may be uninformed about specific listings, and when the Rails app asks the search engine "Ok, find me stuff matching x query", it will return "OK, I found these entries <list of primary keys to the database>". The Rails app will look up the results based on the primary keys. However if the engine isn't in sync, it wont return all the primary keys that would match if you queried the database with the same question.
In this scenario, the solution is simply to bring the search engine back up to sync with the database, and fix the issue thats causing it to come out of sync. The problem is, if you don't know what is causing it to come out of sync, it may fix it one day, but then eventually break again, and you have to try another fix. You won't know you fixed it until it doesn't break again.

 

Related Links:

Edited by Chaser Zaks
  • Like 3
  • Thanks 4
  • Sad 1
Link to comment
Share on other sites
Elvina Ewing

Elvina Ewing

Posted
Posted

thanks so much for the heads up! i vote for "end of year"+"not bought recently" combination as a possible explanation. FYI, half of my store is gone missing! Thanks so much, LL!! *facepalm* I have about 1500 items, now only 731 of them are showing up. Among them are brand new items that were listed only couple of days ago. Like, 2 colours of the same dress, the one that was bought since then, is showing, the one that wasn't bought yet, is missing. The same thing is happening when i go to my Manage Listings page: all of the same listings are not there. So, i cannot even edit them, unless i do the trick that you suggested. Another trick: all those listings are still showing as Related Items. So, from there i can access pages of the colours that are missing, if i want to edit them.

Link to comment
Share on other sites
Froukje Hoorenbeek

Froukje Hoorenbeek

Posted
Posted (edited)

I appreciate your explanation. I still have further questions though. Like why is this the 6th time in 6 months? Why is it also brand new and/or best selling items? Why sometimes a handful, sometimes hundreds? Why are they not waving the 10% marketplace fees they put up right before the search changed to rubbish for relevance search, listing enhancements went missing, then listings, then whole stores? Why is LL not doing their best to keep us informed so we can maintain a certain level of trust? Why are the not fixing it right away when we report it? That’s just the first few that pop up. Anyways, I appreciate your explanation, but 6th time in 6 months. It is very unnerving. 

Edited by Froukje Hoorenbeek
  • Thanks 1
Link to comment
Share on other sites
Chaser Zaks

Chaser Zaks

Posted
  • Author
Posted
5 minutes ago, Froukje Hoorenbeek said:

I appreciate your explanation. I still have further questions though. Like why is this the 6th time in 6 months? Why is it also brand new and/or best selling items? Why sometimes a handful, sometimes hundreds? Why are they not waving the 10% marketplace fees they put up right before the search changed to rubbish for relevance search, listing enhancements went missing, then listings, then whole stores? Why is LL not doing their best to keep us informed so we can maintain a certain level of trust? Why are the not fixing it right away when we report it? That’s just the first few that pop up. Anyways, I appreciate your explanation, but 6th time in 6 months. It is very unnerving. 

Why it is like the 6th time in 6 months:
Bugs are often difficult to fully fix. You can patch one, and another one appears somewhere else. Linden Lab doesn't like it when their services don't work, because that upsets customers. So I can tell you it most certainly isn't intentional and that they do want it fixed.

Why it is also for brand new items:
Honestly, I'm not quite sure. My assumption on it being related to dates might be wrong, and it might instead be an issue with the search engine back end. If the search engine isn't fully synced with the database, that could also be causing an issue. Having a separate search engine is beneficial as it will speed up how fast pages are served, but sometimes connecting the two isn't very easy. Honestly knowing this gives me further insight on what may be causing the issue, and I'm going to update the speculation in regards to it.

Why they aren't waving the 10% fee:
I'm not LL, so I can't answer this.

Why sometimes a handful, sometimes hundreds?
Assuming it is the search engine becoming out of sync, it'd really depend on which entries are cached, and how often the engine is informed which entries need to be repopulated.

Why LL isn't keeping people informed:
So this comes down to a chain of issues. You have developers working on fixing stuff, but if you have them stop to write a explanation on why it isn't working, that's time that could be used trying to fix the issue. So often the case is where PR will ask what is going on so they can write a response, and the developers will say "Search engine isn't staying in sync with the database", and PR will try to write it in a way that most people will be able to understand, so it will just be written as something like "There is currently issues with the search". Having this go on constantly would end up being like PR asking devs "Are we fixed yet? Are we fixed yet? Are we fixed yet?", and this would just slow down efforts to repair the issue, so often once it is fixed, developers will tell PR "Hey we fixed the issue with the search engine.", and then PR will say "It's fixed now!".
While having people technically informed on what is going on, it often isn't feasible.

Link to comment
Share on other sites
Elvina Ewing

Elvina Ewing

Posted
Posted

Why they aren't waving the 10% fee:

Because that's the only incentive for LL to even keep trying to fix it, because if it isn't working, it will be their own pocket, too, that will feel it. Maybe 10% isn't enough, so i will not be surprised if at the end of March we will get another fee increase...

I'm guessing PR people are out of office celebrating New Years Eve...

Link to comment
Share on other sites
Dema Fairport

Dema Fairport

Posted
Posted (edited)
1 hour ago, Froukje Hoorenbeek said:

I appreciate your explanation. I still have further questions though. Like why is this the 6th time in 6 months? Why is it also brand new and/or best selling items? Why sometimes a handful, sometimes hundreds? Why are they not waving the 10% marketplace fees they put up right before the search changed to rubbish for relevance search, listing enhancements went missing, then listings, then whole stores? Why is LL not doing their best to keep us informed so we can maintain a certain level of trust? Why are the not fixing it right away when we report it? That’s just the first few that pop up. Anyways, I appreciate your explanation, but 6th time in 6 months. It is very unnerving. 

Froukjie, I am showing up to the Web meeting next week... I am so done with this as well. My sales have dropped at least 50% since April and no one can find my products anymore. Also, they need to start having web meetings more than once a month.. this is all ridiculous. I am sorry you are having to go through all of this, I feel your pain ♥

Edited by Dema Fairport
spelling
  • Like 1
  • Thanks 1
Link to comment
Share on other sites
Dema Fairport

Dema Fairport

Posted
Posted (edited)
2 hours ago, Chaser Zaks said:

Why it is like the 6th time in 6 months:
Bugs are often difficult to fully fix. You can patch one, and another one appears somewhere else. Linden Lab doesn't like it when their services don't work, because that upsets customers. So I can tell you it most certainly isn't intentional and that they do want it fixed.

Dean, respectfully, I am a Technical program manager who manages a team of 8 devs who work on one of the largest humanitarian nonprofit websites in the world. While I appreciate your explanations, this has been happening since April when they made the changes to the MP and introduced elastic search. These bugs should be the number one priority for Linden Labs, and whoever their product owner is should have prioritized these "bugs" months ago when the tickets were submitted.

Instead, we have had tickets open since October and prior, Creator's entire stores are gone, no one can find products anymore, and customers are complaining they can't find things and are instead seeing results of items that were made back when mesh wasn't even a thing. None of my products can even be found in search anymore due to people keyword stuffing the features tab, and exploiting the search - just type in the word bed and you will see what I mean. I have reported this and their answer is to flag the items, hard to do when it is the WHOLE store of hundreds of items and they don't give you a way to explain what is happening. 

This was never the case and was introduced when they started using elastic search... my MP sales have dropped at least 50% since April and others have dropped up to 80%.

As for items missing... this has been happening also since they purged the MP. Someone messed up big time and Linden Labs needs to own it! Their Product Manager needs to own it and do their job.

I for one will not excuse Linden Labs anymore, this is on them, this is a failure of prioritizing and not having people on staff skilled enough to fix what they broke. People live off their earnings from their sales, and this affects people in real financial ways. Linden Labs needs to stop treating this with a "gaming" attitude and instead start seriously prioritizing the very things that keep the lights on. I am starting to wonder if the execs even know what is happening at this point... this is a sinking ship if they don't fix this.

Also - this is BS from the status update "We are currently investigating a rare issue where unlisted items on the marketplace are getting un-indexed, resulting in the viewer showing active items that are not visible in the Marketplace Store/Manage Listings page."

This is NOT a RARE issue, this issue has been happening since April. 

Edited by Dema Fairport
spelling
  • Like 2
  • Thanks 1
Link to comment
Share on other sites
Dema Fairport

Dema Fairport

Posted
Posted (edited)

I am sending this to Execs at Linden Labs, just so you all know

I hope this message finds you well. My name is xxxx, known as Dema Fairport in Second Life (SL). In my primary profession, I serve as a Senior Program Manager for a global humanitarian non-profit, holding certifications as a Scrum Master and Product Owner with 25 years of experience in managing projects and products for leading corporations.

I am reaching out today, alongside numerous creators and shoppers, to express our deep frustration with the state of the Marketplace following the changes implemented in April 2023. While I recognize the necessity for Marketplace cleanup, the consequences of these modifications, compounded by the introduction of Elastic search, have given rise to a myriad of issues affecting both creators and shoppers.

Since April, our community has witnessed entire stores disappearing for users still on the grid, items going missing regularly, and the introduction of an exploit enabling spamming of the features tab on products. Additionally, customers struggle to locate items due to search algorithms disproportionately prioritizing product titles. There are reports of products falsely labeled with icons in the Product Title field, creating misleading impressions, and creators are facing financial challenges as MP sales decline, forcing some to close their regions or in-world stores.

Numerous tickets have been submitted, with many still awaiting review, including the following open ticket: https://jira.secondlife.com/projects/BUG/issues/BUG-234913

These issues have persisted for eight months, and despite widespread complaints on forums and Discord from creators and customers alike, there seems to be no resolution in sight. Personally, I have experienced a 50% drop in sales since April, with my products no longer appearing in search results, and new exploits undermining the integrity of the Marketplace.

I understand the importance of a well-functioning Marketplace for the benefit of all stakeholders. However, the current state of the Marketplace is far from optimal, impacting the user experience and potentially affecting the overall financial performance for Linden Labs.

I urge you to review and address these concerns promptly to restore the Marketplace to a state that aligns with the high standards associated with Linden Labs. Your attention to these matters will not only benefit creators and shoppers but will also contribute to the long-term success of the Second Life platform.

Thank you for your time and consideration.

 

NOTE: I also linked to all the Jira tickets and the forum complaints - this has to get fixed and needs highers ups to take this seriously. 

Edited by Dema Fairport
  • Like 5
  • Thanks 1
Link to comment
Share on other sites
Froukje Hoorenbeek

Froukje Hoorenbeek

Posted
Posted

I got this news from LL support: 

While the symptoms of the outage on December 31, 2023 were similar to those experienced periodically by many Merchants in 2023, the cause was not the same. Our Engineers are still working on permanent fixes to ongoing intermittent issues. 

So they may have changed the status of this problem to resolved, but it hasn't been. Keep checking your listings. 

  • Thanks 1
Link to comment
Share on other sites
  • 1 month later...
Schimmelpelz

Schimmelpelz

Posted
Posted (edited)

I've checked the status report and LL says this problem is resolved now since 12 days ago BUT that's not true! Some of my objects in my store are still unlisted like my Undertaker-photo or one of my "Downfall Of The Vatican" cartoon! Where should I report this issue? This site here https://jira.secondlife.com/plugins/servlet/oauth/callback?code=LDqZabZTSamBPd95Mxz5aA&state=FKG7NPBYK2 is currently not working so I don't know where else I should mention that? Froukje Hoorenbeek is therefore absolutely right!

Edited by Schimmelpelz
Link to comment
Share on other sites
Rowan Amore

Rowan Amore

Posted
Posted
25 minutes ago, Schimmelpelz said:

I've checked the status report and LL says this problem is resolved now since 12 days ago BUT that's not true! Some of my objects in my store are still unlisted like my Undertaker-photo or one of my "Downfall Of The Vatican" cartoon! Where should I report this issue? This site here https://jira.secondlife.com/plugins/servlet/oauth/callback?code=LDqZabZTSamBPd95Mxz5aA&state=FKG7NPBYK2 is currently not working so I don't know where else I should mention that? Froukje Hoorenbeek is therefore absolutely right!

LL moved off the jira site.  Post your current issues with MP here...

https://feedback.secondlife.com/web-bugs

  • Like 1
  • Thanks 1
Link to comment
Share on other sites
You are about to reply to a thread that has been inactive for 84 days.

Please take a moment to consider if this thread is worth bumping.

Please sign in to comment

You will be able to leave a comment after signing in



Sign In Now
 Share
Go to topic listing
×
×
  • Create New...