Marketplace Beta Search - New Information

[Whirly Fizzle]

---

Grumpity Linden wrote:

Whirly and animorf, you'll notice you now have same number of items in beta and production

---

Yep, confirmed those two stores are fixed.

Thanks!

[Polymath Snuggler]

When a search of any type is performed it asks the database for a result and the database answers with a list of what it thinks are relevant items.  ( not to be confused with sorting by [Relvance] ).  Returned search results are determined by:

[ This list is not exhaustive, just the ones that I am aware of at the moment, and it's worth pointing out that LL is adjusting this constantly, that's the whole entire point of this entire beta Marketplace project. ]

Associated Keywords : How the item is marked.

Listing Title : If the listing is titled your search term~  It's likely to show up in the search.

Store name: If the store name had a possible connection to the search the term the results would show up.

Approximate keyword / Fuzzy results:  This is a new critera that the new search seems to be doing, If you enter Dutchie, it'll pull up all of Froukje's store first ( YAY BIG IMPROVEMENT ~ way to go LL! ) and then some other things like Dutchy, Dutch Oven etc etc. followed finally by listings for Hutches.

First 150 characters of the listing description.:  Used to have some bearing ~ I think~ maybe?  Now we're getting into voodoo-guesswork ~

[Relevance] : Is the default ordering for the marketplace search results. "Relevance to search term."  The returned results are (intended to be) a balanced result tuned to take into account the following factors::

Sales popularity:  How well this particular item has sold.

Novelty:  how 'new' the posted item is ( IE was it recently posted )

Reviews: How much positive / negative feedback a specific listing has recieved.

Older algorithms that I don't know if they've been reimplemented used to include:

Overall store popularity:  How well this stores other items sell.  If a popular store has a listing relevant to the performed search, even if the item itself is not a best-seller, it'll filter up a few notches in the search results.

 

[best Selling] : Is the ranking tab that's meant to organize a search result by EXCLUSIVELY what sells the most frequently. Excluding most other factors, specifically Novelty, possibly Reviews as well.

[ChinRey]

---

Dakota Linden wrote:

Keyword Spam violates the Marketplace Listing Guidelines.

Anyone aware of keyword spam on a product listing is strongly encouraged to report the listing so that the issue can be addressed with the seller as quickly as possible.

---

That's good to hear. But the questions are:

1. Do you have the capacity to deal with the problem?

It's not uncommon when you do a search on MP to get a list of hundreds of irrelevant keyword spam listings. I think I better use an example not related to my own store. Let's say you want a texture kit for yout lovely Pacha sailboat.

A search for Pacha on MP brings up 652 hits. Most of them are from a store named Pacha - illustrating the issue with store names being included in search. Oh well.

Let's narrow it down a bit and search for Pacha texture kit. That helps of course. But on the first page you still have one texture kit for Pacha's main competitor, Bandit B60 and you even have two planes there! Look at the source code for those listing and yes, they have Pacha in the keyword field even though they have no relevance whatsoever to anything called Pacha. I chose that example because I happen to know that these listings have been flagged several times by several people. They're still there.

 

2. When a listing is flagged for keyword spam, do you know what to look for?

That's a genuine question btw. In my first post in the old thread I used Medieval Stable as the test search. I didn't flag any of the keyword spam listings then for obvious reasons but if I had, would you have known that the words medieval and stable were what you needed to look for or would you have had to check each and every keyword for each and every of the several hundred flagged listings?

 

3. Do you have good enough security measures to protect merchants from being delisted because of accidental keyword spamming?

I suppose you're well aware of the autofill bug by now - where keywords from the source listing may be retained in the new lsiting without the merchant knowing about it. What do you do when you come across cases like that?

[Polymath Snuggler]

Boolean searches for NOT [four letter word] are now properly excluding wildcard-type results such as **[four letter word]** and *[four letter word]*

Thank you for the fix !   ♥

[Sassy Romano]

---

Oz Linden wrote:

Ok.  We get that you would like demo items excluded from search. We can't do that immediately, but we get it. You can all stop repeating that now, and really we don't need 30 point bold type for anything.

 

---

Lookout to bridge: "CAPTAIN!  There's a big 'king iceberg ahead!"

Captain to Lookout: "YES I know but don't bother us now, we're busy trying to go faster...then we'll think about turning ok?"

On a serious note Oz, excluding demos from search IS a fundamental element of improving search to the end user and it's mildly worrying that only in that post has it been acknowledged that it's a very long standing request after it was first misinterpreted as us wanting a different category when that had never been the request, so clearly it wasn't "got" until then?

You see, when you throw words like "relevance" and "improving" into the work, then i'd return by saying NOT (as in boolean) and "relevant" are DEMO items.  Thus it follows that improving relevance can be achieved by reducing what is not releant therefore your adamant reluctance doesn't compute.

Anyway, now that you have stated that you accept it's something we want, it's not going to be attended to immediately, would it be too much to ask when this long standing improvement may expect to be vaguely considered for possibly getting some finger lifting attention? 

 

[ChinRey]

---

Oz Linden wrote:

Ok.  We get that you would like demo items excluded from search. We can't do that immediately, but we get it. You can all stop repeating that now, and really we don't need 30 point bold type for anything.

 

---

Sorry to hear you feel that way, Oz. But you have to understand - as Sassy so subtly hinted at - that this is hardly something new. Merchants have requested, suggested and even begged for this change for years now and when Linden Lab posts a reply where they don't even seem to understand what it's about, there's no wonder people need to vent their frustration.

Now, this project went off to a bad start, and it certainly wasn't the right one to start with anyway, But now that it seems to be on track, the best thing to do is to finish it and let all the other issues wait.'

Don't let it stop there though. There are so many problems with MP that should have been fixed years ago and one isolated upgrade isn't going to make that much difference to the big picture. You'll have to keep on delivering. And it's going to be really tough. Many merchants have been banging their heads against the wall for so long now they simply can't believe there's finally a hope for a breakthrough.

[Dakota Linden]

Ladies and Gentlemen,

Linden Lab is aware of many requests from both Merchants and Sellers regarding features that you would like to see on the Marketplace.  

Please keep the comments, suggestions, and issues regarding the Beta Search on topic to the changes outlined in the thread starting post.

If you wish to have conversations about other topics or issues related to search, we ask that you please create a new thread where those topics can be discussed.

 

[Polymath Snuggler]

I said it in the last thread and I'll say it again ~ since people are so intently focused on DEMO items. 

( I can't really blame them~ I'm focused on being able to exclude them as well ~ HOWEVER!!! )

 

There is a substantial fraction of marketplace listing / visibility that is maintained exclusively through the popularity of the products demo listing.

Let me give you an example.  Let's say I'm a vendor selling VeryAwesomeBoxes.  I have a 7 listings for my VeryAwesomeBoxes.  They are as follows.

 

Yellow VeryAwesomeBox : Sold 2 units.

The ASSOCIATED DEMO for my Yellow VeryAwesomeBox : Sold 79 units.

Blue VeryAwesomeBox : Sold 9 units.

Red VeryAwesomeBox : Sold 11 units.

Green VeryAwesomeBox : Sold 2 units.

Black VeryAwesomeBox : Sold 23 units.

White VeryAwesomeBox : Sold 15 units.

Now~ since overall Relevance ranking is tied to prior item sales and since each individual color is it's own search listing.  As far as the system is concerned there's no way of identifying that this bunch of listings as a *product group* has sold 62 Units.

So when someone searches for "boxes", the varying colors of VeryAwesomeBoxes will be tremendously low in search ranking due to their low individual sales.  But the DEMO is near the top of page 1.  The demo is what keeps this product being visible and finding buyers.  A demo which I may remind you is only tied to ONE LISTING, which has sold 2 copies because I was a stupid merchant and made my Banana Yellow VeryAwesomeBox my primary listing instead of my White one.  But in this scenario, which mind you is very very real the DEMO ~ ironically is the lynchpin that keeps the listing alive.

So ~ as you can see demos and colors are very closely intertwined in terms of how they need to be handled.  Which is why both are out of the scope of this project.  Both involve going back through the item database and recatagorizing and  reattributing sales populartiy from one item in the database to another as well as grouping massive quantities of products based on a new ranking scale.  THIS IS NOT A TRIVIAL TASK.

 

Searching for "NOT DEMO"  presently removes items titled "DEMO" from the search results.  If you need to use it it's there.  But I can understand LL's hesitation to implement it as a standard.

 

Edit: Sorry Dakota ~ was writing this post while you were posting yours!!

[Sassy Romano]

Thanks Dakota for bringing this back on track. So, to quote from the latest blog post:-

 

"We’ve made infrastructure changes to improve the relevance of search results - making it easier for you to find what you’re looking for! "

 

Why is it SO hard to get through that you improve relevance immediately by removing irrelevance?

 

I did a search, got results, don't know what the test criteria are, test passed, move on!

[Perrie Juran]

New search results for "Martian."

Old Search:    467 Items

Beta Search:  1379 items.

 

Wow, Almost 900 Martian items I'd been missing.  NOT.

Applying Boolean logic, "martian NOT martial" new result:  449 (very close to Old Search but not sure cause I am not taking the time to see if all the same items were shown).

Point is using similarly spelled words to generate results again results in huge clutter.  If I had accidentally mispelled "Martian" (had typed "Martial") I think I am smart enough that I would have caught my mistake.

 

At the risk of getting off track, why are we so dependent on shoppers using Boolean terms?  It would be an interesting survey to find out how many people even know what they are any more and/or that they work in the MP.  I know I have certainly had to explain to a lot of people about theiir use in the MP.

[Pamela Galli]

I thought you were supposed to put an asterisk if you wanted the search engine to treat your search term as a root.

But in any case, martian is NOT a root of martial!

 

Back to the drawing board.

[Starr Midal]

And yet still nothing...

[Grumpity Linden]

A quick update on open bugs:

BUG-10688 ( Marketplace Beta Results do not take into account frequency of Sale when ranked by "Relevance") - has been fixed. 

BUG-10761 ([sLM Beta Search] Boolean search only returning a few results for certain search terms when there should be thousands ) has been fixed. 

BUG-10762 ( [sLM Beta Search] Beta search often fails to return a lot of relevant search results)  is looking better.  We currently believe that discrepancies in count are due to issues with keeping all the listing data in synch between standard search and beta.  Additional investigation ongoing. 

BUG-10769 ( [sLM Beta Search] Some store pages have missing listings when using beta search ) is due to an indexing issue with beta.  We're getting rolling reindex of the entire marketplace going, and in the meantime, if editing the "missing" listing via betasearch-marketplace does not fix it, please file a new jira.

BUG-10799  ([sLM Beta Search] Using Boolean search terms causes [Relevance] results to lose ranking from sales popularity.) - We're looking into it. 

Thank you! 

[Polymath Snuggler]

Yay!:  I created one other BUG for you~  ( I posted it in the prior pages of this thread but I figured I'd put in a BUG Jira since it seems to have been overlooked )

 

https://jira.secondlife.com/browse/BUG-10830

[Sassy Romano]

Bugs that fix relevance... Fixed

Ok so since this question seems to be a slippery eel for LL to pin down, when you say you've fixed relevance, what EXACTLY does relevance mean to you and more importantly, what do you believe it means to me?

If you can't answer this precisely then how do you expect to know if you've fixed it?

Let's work with an example, say I use just "RLV" as a search, without you actually testing this first, what is "relevant" in terms of an expected search result? How should the search result set be ordered when "relevance" if chosen? How are you measuring that against my expectation of what I want as relevant?

You see, none of this has been properly started, this this is why when I do a search and get a result set, the test passes, there is no test criteria and therefore the results are woefully subjective.

[Arwen Serpente]

A few observations/comments:

1.  BUG-10769 regarding missing items in stores on the Beta Website - Please indicate when this "reindexing" is taking place.  My store is still missing 3 items. However, the JIRA was closed so I can't comment there.  The idea of trying to match my current store with 602 items to the beta store with 599 in order to "edit" a missing listing is rather odd - how exactly am I supposed to find a listing that is missing?

2. In the Beta, sorting the stores by "Best Selling" returns results that are fine for the first 12 slots.  After that it is a jumble that makes no sense and has no relationship to what has been or currently is best selling (and this has been reported before).  In fact, it defaults to the least frequently sold items.  By contrast, the current "relevance" sort (on the regular website) is far more indicative of the best selling items - it shows most recently sold in the first 12 slots followed by a mix of recent best sellers and historic best sellers.  I would far prefer to have that way of sorting renamed "best selling" for the Beta.

3. And as I asked before, what is your definition of "relevance" (also asked by Sassy) and "best selling"? 

[Drake1 Nightfire]

Just created a bug.. https://jira.secondlife.com/browse/BUG-10833

I have never been able to have more than the first page stay how i sorted it. If i want new items i can only see the first page. then it switches to "relevant"

[Sassy Romano]

Ok a little more feedback.

What does "best selling" mean to you and what do you think it means to me?  Similar question to relevance.

When I look at "Top Selling" report in marketplace reports, why do you feel that the customer view of my best selling products should be ABSOLUTELY COMPLETELY DIFFERENT?    There is ZERO correllation whatsoever, how can this possibly be the case that a search result bears no relationship to the report when both are returning exactly the same intended query result?

Please explain.

Again, comparing "relevance" in existing and beta on just my own items produces garbage in terms of sense, there's no relevance whatsoever, one return isn't even a proper product but a test item called "dress" that has no picture and has never sold.  It's not in the current marketplace search result for relevance.

"Relevance" is irrelevant unless you can explain exactly what it means.

Oh and I got demos in search result which are irrelevant to me, thus a search fail

[Drake1 Nightfire]

Can i ask why merchants names show up in a regular search? Shouldn't they only how up if we search by merchant? If i want a sword, why does every merchant that has sword in their name show up as well? Seems like a simple fix, no? If you have a merchant search tab that doesn't show items why doesn't the items search tab remove merchants names?

[Pamela Galli]

---

Drake1 Nightfire wrote:

Can i ask why merchants names show up in a regular search? Shouldn't they only how up if we search by merchant? If i want a sword, why does every merchant that has sword in their name show up as well? Seems like a simple fix, no? If you have a merchant search tab that doesn't show items why doesn't the items search tab remove merchants names?

---

In the OP they said they had fixed that. They broke it again?

 

 

[Drake1 Nightfire]

---

Pamela Galli wrote:

---

Drake1 Nightfire wrote:

Can i ask why merchants names show up in a regular search? Shouldn't they only how up if we search by merchant? If i want a sword, why does every merchant that has sword in their name show up as well? Seems like a simple fix, no? If you have a merchant search tab that doesn't show items why doesn't the items search tab remove merchants names?

---

In the OP they said they had fixed that. They broke it again?

 

 

---

I sort of forgot they were talking about the beta MP...

[Darrius Gothly]

---

Oz Linden wrote:

Ok.  We get that you would like demo items excluded from search. We can't do that immediately, but we get it. You can all stop repeating that now, and really we don't need 30 point bold type for anything. 

---

(Sorry Oz .. been unaviodably outta the loop for a week.)

I get the raw nerves thing Dude .. I really do. But I do hope you appreciate that we have nerves rubbed just as raw by having made this specific request many many times without anyone having the last name Linden even so much as acknowledging it.

So while to you it may seem like the sixty-bazillionth time ... to us this is the first reply to our sixty-bazillion attempts.

[Darrius Gothly]

Now for a general (and sorry but off-topic kinda) bit of feedback.

I have seen WAY more involvement from Lindens in this thread than in quite some time. THAT IS FREAKING MARVELOUS!

Keep that part going .. k?

[Sassy Romano]

Sixty bazillionth requests that now constitute harassment and that's now against the TOS.

 

Change the TOS to become beyond reproach, nice move!

 

As such, I and others place ourselves at risk by further involvement in this thread and as such must cease any further beta testing and feedback. Instead I'll just use an alt and declare each new offering as AWESOME - JUST WHAT WE NEED, PERFECT, MEETS EVERY CRITERIA.

[Grumpity Linden]

Much has been written on the art and science of search relevancy in both technical whitepapers and industry publications and online sources.  It's a mysterious combination of factors that should, with a little fairy dust, give every user the results they are looking for.  The actual algorithms involved are complex and varied, depending on, among other things, the specifics of search areas.  The measure of success is also up for debate in the literature, but we can likely settle on Precision and Recall as an adequately acceptable answer.

It is most certainly outside the scope of this forum to get into much detail on this very complex topic.  Therefore the best feedback for us is what we have gotten here so far (particularly in jira with details) where search results do not meet expectations and later on we will be measuring marketplace volume and conversions from search results to assess whether our changes have been successful.  

We can certainly appreciate your frustration with the difficulty of pinning down search relevance, but unfortunately cannot resolve it.

Better news on Best Selling sort - there's a bug in there and we're investigating. Updates on that soon.