SL Marketplace - Ratings Are No Longer Accurate?

[Pickle Soup]

The old xstreet rating system had half stars implemented into the ratings... i see the new one doesn't. For example, if you have a product that has 50% 4 star ratings, and 50% 5 star ratings, the end result will be a 4 star rating, not a 4.5 star rating.

Yes, i realize this may seem like a frivolous detail to most, but hey, i'm bored and wondering if anyone else cares.

[Athena Mornington]

There are no more half stars or partially filled stars showing a true average of ratings in the SL Marketplace. Even if a seller has an item with a zillion 5's and just one 4, the rating shows up as a 4. It was reported by the sellers but it doesn't seem to be something that will be changing.

[Pickle Soup]

Hmm, well that kinda sucks. lol

[Suella Ember]

Athena Mornington wrote:

There are no more half stars or partially filled stars showing a true average of ratings in the SL Marketplace. Even if a seller has an item with a zillion 5's and just one 4, the rating shows up as a 4. It was reported by the sellers but it doesn't seem to be something that will be changing.

Are you sure that is correct? I haven't looked into it deeply myself, but my understanding is that, in your example, the item would have 5 stars. I think it still takes an average, but rounds that average to the closest number of full stars which, in this case, would be 5 stars.

It would only have a rating of 4 stars if more people giave it 4 than 5. I stand to be corrected though if someone can show me this is not the case?

[Pickle Soup]

Suella Ember wrote:

Athena Mornington wrote:

There are no more half stars or partially filled stars showing a true average of ratings in the SL Marketplace. Even if a seller has an item with a zillion 5's and just one 4, the rating shows up as a 4. It was reported by the sellers but it doesn't seem to be something that will be changing.

Are you sure that is correct? I haven't looked into it deeply myself, but my understanding is that, in your example, the item would have 5 stars. I think it still takes an average, but rounds that average to the closest number of full stars which, in this case, would be 5 stars.

It would only have a rating of 4 stars if more people giave it 4 than 5. I stand to be corrected though if someone can show me this is not the case?

It would be nice if you were right, but i don't think you are.

I'll give you another example. One of my my items has 6 ratings; 4 - 5 star ratings, and 2 - 4 star ratings... yet it's final rating is 4 stars. So it would seem that Athena may be correct. So it seems to be that you could have 5000 - 5 star ratings, but because no half stars exist, if 1 person gives it a 4 star rating, the final rating will, in fact, say 4 stars.

[Yoki Enoch]

Athena Mornington wrote:

 

There are no more half stars or partially filled stars showing a true average of ratings in the SL Marketplace. Even if a seller has an item with a zillion 5's and just one 4, the rating shows up as a 4. It was reported by the sellers but it doesn't seem to be something that will be changing.

So, by that logic, if one had a gazillion 5 star ratings, and just one, 1 star rating, the overall rating would be 3 ? Let us hope not for the sake of the merchants.

[Suella Ember]

Yoki Enoch wrote:

Athena Mornington wrote:

 

There are no more half stars or partially filled stars showing a true average of ratings in the SL Marketplace. Even if a seller has an item with a zillion 5's and just one 4, the rating shows up as a 4. It was reported by the sellers but it doesn't seem to be something that will be changing.

So, by that logic, if one had a gazillion 5 star ratings, and just one, 1 star rating, the overall rating would be 3 ? Let us hope not for the sake of the merchants.

Indeed.

I'd certainly take exception to that, despite being a 'cheerleader' (which I'm not, I'm just constructive, but you've called me a cheerleader before ) Although even if this is the case I'll still try to discuss it constructively and get some logical answer from LL rather than having a pointless rant about it!

I'll have to do a bit of digging round in the commerce forums and stuff I guess to see if there is a definite answer to this.

I don't particularly have a problem with it rounding to the nearest whole number, but it needs to be rounding correctly based on the mean.

In other words, if an item has ten ratings, nine of them 5 stars and one of then 1 star as follows, then the mean would be 4.6 which should give it an overall rating rounded up to 5 stars:

5+5+5+5+5+5+5+5+5+1 = 46

46 / 10 = 4.6 (rounded up to the nearest whole number = 5 stars)

[Suella Ember]

Pickle Soup wrote:

I'll give you another example. One of my my items has 6 ratings; 4 - 5 star ratings, and 2 - 4 star ratings... yet it's final rating is 4 stars. So it would seem that Athena may be correct. So it seems to be that you could have 5000 - 5 star ratings, but because no half stars exist, if 1 person gives it a 4 star rating, the final rating will, in fact, say 4 stars.

Interesting!

Maybe it is rounding down to the nearest whole number. That should be a 5 star rating if they are using the mean.

5+5+5+5+4+4=28

28 / 6 = 4.66 (recurring) - so that should round to the nearest whole number, which is 5.

Gonna see if I can find an official answer to this somewhere!

[Yoki Enoch]

Suella Ember wrote:

 

Yoki Enoch wrote:

Athena Mornington wrote:

 

There are no more half stars or partially filled stars showing a true average of ratings in the SL Marketplace. Even if a seller has an item with a zillion 5's and just one 4, the rating shows up as a 4. It was reported by the sellers but it doesn't seem to be something that will be changing.

So, by that logic, if one had a gazillion 5 star ratings, and just one, 1 star rating, the overall rating would be 3 ? Let us hope not for the sake of the merchants.

Indeed.

I'd certainly take exception to that, despite being a 'cheerleader' (which I'm not, I'm just constructive, but you've called me a cheerleader before ) Although even if this is the case I'll still try to discuss it constructively and get some logical answer from LL rather than having a pointless rant about it!

I'll have to do a bit of digging round in the commerce forums and stuff I guess to see if there is a definite answer to this.

I don't particularly have a problem with it rounding to the nearest whole number, but it needs to be rounding correctly based on the mean.

In other words, if an item has ten ratings, nine of them 5 stars and one of then 1 star as follows, then the mean would be 4.6 which should give it an overall rating rounded up to 5 stars:

5+5+5+5+5+5+5+5+5+1 = 46

46 / 10 = 4.6 (rounded up to the nearest whole number = 5 stars)

Hi Suella:

There is no need to take exception to my observation. That is just what it was: an observation. Your math is perfect in your analysis. It all depends on whether LL staff can understand that rounding down is not of benefit to SL merchants.

[Suella Ember]

Yoki Enoch wrote:

Hi Suella:

There is no need to take exception to my observation. That is just what it was: an observation. Your math is perfect in your analysis. It all depends on whether LL staff can understand that rounding down is not of benefit to SL merchants.

I meant I'd take exception to LL's method of calculating an 'average' if they were rounding it incorrectly

Judging by the evidence so far I'd say that what would happen in the example you gave (a gazillion 5 stars and one 1 star) it would get an overall rating of 4 stars, because it seems to be rounding down to the nearest whole number. (I'm not gonna attempt the maths, but a gazillion 5 stars and one 1 star would probably have a mean value of something like 4.9999999999999999999 ).

If that's what it is doing, then I think they need to fix it to round exactly to the nearest whole number. If the mean is 4.51 or higher then it should get 5 stars. If the mean is 4.49 or lower then it should get 4 stars (don't ask me what it does for a 4.5 exact - I guess they can just make a judgement on whether it rounds up or down!)

I may be missing something though. Hopefully there is an official answer on this somewhere. /me goes off to try and search the JIRA!

[Suella Ember]

Ok, so I just managed to ask brodesky Linden about this and he kindly replied (yeah, yeah "FIC!" etc - just be grateful I got a decent answer to give here! )

Apparently it is on their list to show partial stars and actual ratings. They have done an analysis of lots of other sites to try to come up with the best way of doing things. I get the impression they are considering something along the lines of what Amazon does:

I'm not saying this is definitely what we will get so please don't take it as gospel, but that seems to be where brodesky was leaning. It's also on their backlog of things to do so I guess we can safely assume it won't be tomorrow we have this. It's good to hear that they are thinking along these lines though. Looks like, for now, we have to put up with what we have, but that we should get a much better solution at some point in the future.

[Pickle Soup]

Thanks a lot Suella. I just don't understand why they have to change something that was perfectly fine in the first place. I understand the new Marketplace - they messed up the old one when tehy took it over, but why scrap a perfectly good rating system? The way it is now is fairly inaccurate. I will continue to watch what happens, and i sincerely hope they do start rounding up soon, because as of now they are rounding down... meaning if you do have a rating of 4.99999, the rating will be a 4, which sucks.

Again, that's for your detective work.

[Suella Ember]

For info. Torley has added this comment to the marketplace wiki:

Why don't ratings show partial stars like the old Xstreet?

Showing partial stars is on our to-do list (as of 2010-10-20), and showing the actual, raw, numerical rating value. We've extensively studied other ecommerce sites to understand best practices for stars and ratings.

[Laroelle MacMoragh]

So one person with a hard on for trashing what a majority of other users rate as a five could effectively bring down a nice item to a one star rating.  I can see griefers enjoying this little gaffe.

 

I don't normally spank LL, but c'mon guys!  This just isn't right.  I sense another sh*tstorm on the horizon.

[Pickle Soup]

Laroelle MacMoragh wrote:

 

So one person with a hard on for trashing what a majority of other users rate as a five could effectively bring down a nice item to a one star rating.

I don't think it's as bad as that. If a rating has a ton of 5 stars, and one person rates it a 1 star, i think it would still read as 4 stars - because the way it is now it rounds down... even if it's 4.9999999999999999, it will still round down to an even 4 stars.

[Pickle Soup]

Suella Ember wrote:

 

For info. Torley has added this comment to the marketplace wiki:

Why don't ratings show partial stars like the old Xstreet?

Showing partial stars is on our to-do list (as of 2010-10-20), and showing the actual, raw, numerical rating value. We've extensively studied other ecommerce sites to understand best practices for stars and ratings.

TY!

Now, let's see how long it takes them to implement this. lol

[Torley Linden]

I'm glad Suella asked Brodesky and me. Shortly thereafter, I corrected the date to "2010-09-20", since accurate ratings are on our current to-do list (not a future one, heh). Anyway, yeah, keep referring to the Marketplace Sellers' FAQ for updates.

[Suella Ember]

Pickle Soup wrote:

Laroelle MacMoragh wrote:

 

So one person with a hard on for trashing what a majority of other users rate as a five could effectively bring down a nice item to a one star rating.

I don't think it's as bad as that. If a rating has a ton of 5 stars, and one person rates it a 1 star, i think it would still read as 4 stars - because the way it is now it rounds down... even if it's 4.9999999999999999, it will still round down to an even 4 stars.

Yep, Pickle is correct.

Effectively someone could, at present, drag a genuine 5 star review down to a 4 star with one 'griefer' vote. They couldn't drag it down to a 1 star overall though (not without a huge army of alts and a lot of money spent, and they'd almost certainly get caught doing that!)

Anyway - the good news is that LL have it on their radar and when we get half stars and (hopefully) some nice distribution information too, it will be easy to see what people genuinely think of a product. If we can easily see the distribution and see, for example, that an item with 4.5 stars overall has fifty 5 star reviews, ten 4 star reviews and one 1 star review, it's pretty easy to deduce that the 1 star review was probably just to grief and can be ignored.

Oh and Torley - if you happen to still be reading this, I noticed another typo on the Marketplace/Sellers FAQ: One of the first questions is 'How do i add a new item to the Marketplace' and the link underneath says 'Quick the question to learn how' rather than 'click the question to learn how'

[George Orellana]

Sorry for this necropost, I would have started a new thread but I thought it was more relevant here and maybe Suella can shed some light to my confusion. Yesterday I was bored and browsing the SLM and decided to rate my purchased items. On the old Xstreet I usually was able to rate but would not write an actual review (I procrastinate and let the unreviewed purchases pile up), I do however write a review if I find something that needs reviewed, i.e. exceptional customer service. On the SLM however I was "forced" to write a review in order to rate. So what did I do? I rated and wrote a review the first item, then gave up and left the page. Maybe I don't see where the option is to just rate? If there is such option, someone please point me to it. If no such option is available, this is dumb, I can already see that far less people will be rating items.

[Suella Ember]

Hey George!

Unfortunately no, there is no option to just rate an item. You have to leave a review (at least i'm pretty certain that is the case anyway).

It's a tricky one and I see where you are coming from but, overall, I'm happier that people are forced to write a review. It means there is some accountability and goes some way to stopping competitors / griefers from just leaving a bad rating with no comment as to why they gave a bad rating. Ok sure, it doesn't completely solve that problem, but its a step in the right direction. If somone leaves a bad rating at least they now have to back it up with words. Of course, someone could still just put "trash" as a title and "i hate this item" as a review, but shoppers are savvy enough to see past such things and will take heed of constructive reviews.

The other thing i like is that merchants can comment on all reviews. So if someone does leave a bad review a merchant has a chance to address that bad review.

The problem, as I see it, is that there is no easy way to quickly see all un-rated \ un-reviewed items like there was on Xstreet. If we could easily see all un-reviewed items in a quick list / thumbnail view it would be easier to write a quick review (and ideally be returned to that list when submitting the review). The other major problem is that, as the easiest way to currently access un-reviewed items is from our order history, once we do review an item it still remains in our order history with a 'review this item' link (you cant actually review it again though). This makes it very unclear as to what you have and haven't reviewed though and I can see people not bothering to review more for this reason.

Having to leave a review i'm happy with, but there needs to be a way to see just un-reviewed items more quickly.

[Paladin Pinion]

Suella Ember wrote:

It's a tricky one and I see where you are coming from but, overall, I'm happier that people are forced to write a review.

 

I'm with you on this. Someone trashed one of my items over on XStreet, gave it 1 star, never contacted me, never said what the problem was, nothing, just rated and ran. I'm convinced it was a griefer but there was nothing I could do about it. Now that people are forced to leave their names and actually write something, we'll have a way to follow up.

My item didn't have many ratings, so that one star really pulled it down. I think the old XStreet method was unfair.

[Pickle Soup]

Was just browsing the new Marketplace and i saw some half stars! Yay.

Thanks Torley. =^.^=

[Suella Ember]

Pickle Soup wrote:

Was just browsing the new Marketplace and i saw some half stars! Yay.

Thanks Torley. =^.^=

Oh cool! Well spotted

I'm sure kudos are you to the Commerce Team too for addressing this quickly.

I'm not sure how long this has been there, but I also noticed that, if you go to the reviews tab on an actual listing, there is a distribution display for how the ratings have been assigned. You have to click the little down arrow next to 'Distribution' though. Would be great if this was immediately on display without the need to expand it.

[Irene Muni]

Athena Mornington wrote:

 

There are no more half stars or partially filled stars showing a true average of ratings in the SL Marketplace. Even if a seller has an item with a zillion 5's and just one 4, the rating shows up as a 4. It was reported by the sellers but it doesn't seem to be something that will be changing.

???

At this moment I have an item rating with half star: https://marketplace.secondlife.com/p/PARACAIDAS-Y-SITIOS-PARACHUTES-LANDMARKS-gratis-freebies/274137