Previous title: Less than .02366865 probability that Amazon does not censor 1-star reviews?
This is stored here in lieu of deleting it totally because I don't like the idea of removing posts people have commented on, even posts I don't think have interest or value.
[Further update: As I originally mentioned in the piece, the statistical calc was "for our amusement." I didn't take it very seriously and had no idea it would get this much attention. The (very appropriate) criticism emerged in discussions that at least some of the 10 Amazon reviews I was counting as the basis for the stats were undoubtedly the result of the original reddit attention to the Brian Powell issue, invalidating the calcs. I had explicitly noted that possibility in the original piece, but people didn't apparently didn't notice that very much and still took it seriously. As the discussion went on someone suggested that as few as 2 of the posts might have been legit. So I redid the calcs with 2, resulting in the .02366865 result in now the title: still statistically significant, and I did take it more seriously at that point. But now I'm trying to confirm that there actually were 2 "legit" reviews of the book in question on the day in question on Amazon. So far, one of the reviews I thought was likely to be independent has turned out to be in response to the reddit post. It's legit in the sense that the reviewer really did look at the book and write intelligently about it, but he did so in response to the reddit attention, which invalidates it from a statistical point of view. I have a couple more queries out there and if it does turn out that there are 2 independent reviews, I'll note it here. (If YOU wrote one on March 30 for The Final Theory, let me know.) For now, however, my feeling is that this was a not-very-serious and not-very-careful thing that got too much attention. Move on, there's nothing happening here.]
[Update: Since this piece was posted there has been a fair amount of discussion about the calculations. I was really doing the most pessimistic possible calculation originally, since the actual number was more for my amusement than anything else, and I wasn't going to the trouble of actively screening out Amazon reviews that were very likely (or certainly) posted due to the attention this issue received on reddit. The probability was .0000000074279, and that number was in the title of this post. But since then there has been a lot of community input and I think the calculation should be taken more seriously. The most optimistic (from Amazon's point of view) version of the calculation still arrives at a probability of .02366864 -- which is still statistically significant. Accordingly, I've changed the title of the piece, but other than that and the updates in bold below, the piece is the same. Partly because I just don't want to spend more time on it, and partly due to a possibly misguided sense of "journalistic integrity". I may at some point right ANOTHER piece covering the same material and link to it from here. Also, if you are seriously interested in this, I strongly recommend that you read the discussion in the comments to get an idea of how seriously to take this result.]
This morning I noticed this story by Brian Powell on reddit in which he contends that Amazon rejects one-star reviews. Brian wrote a one-star review for a popular science book. The book, called The Final Theory, apparently claims claims that most physics prior its publication was wrong, and that it advances the first overall theory of physics to be correct. As a grad student in physics, Brian unsurprisingly took exception to many of the book's assertions.
His review was rejected repeatedly for different reasons. Finally he gave up in frustration, although at one point it was up on Amazon for two hours before being removed.
After reading the reviewer's story I decided to investigate for myself.
As of this morning (March 31, 2006) there are a total of 62 reviews. A total of 18 of them gave one star; most of the rest gave five. The question is: is Amazon censoring the negative reviews, as Brian contends? On the surface it looks like there is a mix of different reviews (albeit oddly bipolar).
First of all, it seems odd that the overall review Amazon gives the book is four stars given that 18 of the reviews give one-star. I added up the reviews and found there were 18 one's, 5 two's, 0 3's, 4 fours, and 35 fives. The average review according to the simple calculation any elementary school student would do turns out to be 3.3 stars. I understand that Amazon has to round the number because they show the review as a graphic, but they do round to half-stars, so why are they rounding 3.3 up to 4? Something seems fishy here.
But let's look a little deeper. Every one of the 10 reviews posted on March 30 (yesterday) gave one star. Hmm. That seems a trifle odd given that there are only 8 previous one star reviews since July 22, 2003 when the first (five-star) review was posted, among the other 52 previous reviews.
I don't believe it takes a background in statistics to get a hint that those numbers don't add up. But for our amusement, let's look at it from a statistical point of view.
Brian says one of his reviews was visible for two hours, and then removed. So apparently there can be a brief period when a one-star review is visible, before it is removed. Of course, that's a policy that would lead to their being a tendency for one star-reviews to be unusually prevalent among the very newest reviews (in this case, it's 100% of the reviews since yesterday morning).
Another possibility is that Amazon is responding to the high visibility of Brian's post on reddit by suddenly letting the bad reviews through.
But let's assume that Amazon would claim nothing is wrong: there is no underlying pattern that might lead to a higher propensity of one-star reviews since yesterday. That would imply that the any special preponderance of one-star reviews since yesterday would be a mere coincidence. So let's assume it is, and see where that takes us.
More specifically, let's test the assumption that whatever factors might result in one-star reviews being published for a book like this one are not different in the period since yesterday morning than they were in the previous period. Any propensity for recent one-star reviews would therefore be mere innocent coincidence.
In the previous period, we had a total of 8 one-star reviews, out of 52 reviews: a proportion of 0.15384615.
What is the probability that our assumption is true?
Using the assumption that underlying probability of .15384615, we can easily calculate the probability that all 10 out of 10 reviews since yesterday might be one-star. It's the same calculation as calculating the probability that tossing a coin 10 times in a row would be heads. As most people know, the way to do that is to multiply .5*.5*.5*.5*.5*.5*.5*.5*.5*.5. This is also denoted with the expression .5^10 (".5 to the 10th power").
In our case, we need to calculate .15384615^10. The answer is .0000000074279. In other words, there is a .00000074279% chance that the unusual preponderance of one-star reviews is due to coincidence.
Note: this is not a rigorous calculation. It's an estimate. For instance, one might point out from the earlier 52 reviews, we can only get an estimate of the proportion of one-star reviews that Amazon would usually let through for a book like this. The upshot is that our computed probability is also an estimate. That's OK. It is standard in statistics to reject an assumption when the probability of its being true is .05. .0000000074279 is a very long way from .05.
In addition, there may be other explanations to the propensity of recent one-star reviews. For instance, they could all be due to the reddit posting. However, reading the reviews, this does not appear to be the case. Most of them give ample evidence of direct knowledge of the book and thoughtful criticism.
[UPDATE: This is in response to the comments I'm getting that I ignored the possibility mentioned in the paragraph above. I didn't ignore it -- I explicitly discussed it. Right there. Above. See it? About two inches above this sentence? I also said above that the calc is "for our amusement". It is NOT rigorous. The thing that I'll be interested to see is how many of those reviews remain over time. The hypothesis is that they are like Brian's review that was up for two hours before it was taken down. Will these reviews STAY up? I'll post something here when we know.]
[FURTHER UPDATE: Someone over at reddit notes that there are now 11 neg reviews, and guesses that between 3 and 9 of the one-star reviews are due to the reddit posting. As the most pessimistic case assume that there are only 2 legit ones. That gives a result of .15384615^2 or 0.02366864; still below the usual statistical limit of .05 that leads the rejection an assumption (null hypothesis). So, as a statistical test, the conclusion stands even under most pessimistic of the guesstimates that appear in the the harshest numerical criticism I've received.
However, I do now wish I had counted the suspicious ones instead of forming a rough impression before doing my initial calculation. It's a nuisance that people are focusing on the extreme number I calculated, when it really isn't necessary to be that extreme for deriving the statistically valid conclusion, and it IS probable that at least a couple of those reviews were due to reddit. I'd change the calc now, but there are already a lot of links out there so I think I should just leave it. This isn't a wiki such that it's supposed to be improved under community input; it's a post and is therefore more subject to journalistic standards.]
One other thing should be mentioned. Brian's story quotes Amazon's responses to his reviews. On the surface, they sound like they could reflect a legitimate policy. But as Brian points out, the standards Amazon expresses there don't seem to be enforced for positive reviews. But even if they are, any policy which has the effect of screening out negative reviews more than positive ones, even if it is consistently applied, must be considered very, very suspect.
On a personal note, I have made a lot of use of Amazon's reviews. I have thought that Amazon's reviews are one of the great Internet resources. But recently I've occasionally felt suspicion about them. It's seemed that there were too many five-star reviews for items that haven't seemed that good, and the summary star rating has sometimes seemed higher than my informal guesstimate of the average of the published ratings.
This is the first time I've done a real analysis. Unfortunately it seems to confirm my suspicions: unless I'm missing something, Amazon now has policies in place that slant their reviews toward the positive. It's not hard to figure out why they might want to make the items they sell look better than they are!
But I am very disappointed because I have always thought that Amazon reviews had integrity; I thought that they had realized that by having integrity, they earn the trust of buyers, and can use that trust to build their business. It always seemed to me that that was the smart approach, as well as the Right Thing, and it seemed to be what they were doing.
But the fact is that Amazon is now in such a dominant position that their dominance probably wouldn't be threatened just because they lost some integrity. But, losing some integrity could increase revenues, as long as that dominant position remained. So it is arguable that their underlying financial pressures are now different than they were in earlier times, when they were still rapidly growing. And in fact, it appears that they may be responding to their new situation by having a new policy toward reviews.
I very much hope that that is not true, or that if it is, that it's a temporary thing that Jeff Bezos somehow isn't aware of and that will be fixed.
Right now, however, I'm feeling pessimistic. I notice that Brian says that Barnes & Noble did accept his review. I'll bet you can guess where I'm going to go next time I am considering purchasing a book.
Sorry pleaes :(
Wromg category...
wiol eb carefuhl
Posted by: fluisaFaimi | June 29, 2007 at 03:21 PM
Posted by: payday loans | August 06, 2007 at 09:51 PM
Right after I ran out of my funds my Car AC died! It is too hot! That sucks!
Posted by: OweniaKendRed | August 15, 2007 at 03:08 PM
Hello. I found a very interesting site.
http://jolie.150m.com
Enjoy.
Posted by: estictage | September 09, 2007 at 04:11 PM
AdultFriendFinder is a commercial website that claims to be the world's largest sex and swinger personals community online,meet real people looking for sex, association through the web-camera. 1-on-1 sex, group sex, descreet relationship. Erotic chat, email.
Come in!!!
Posted by: g877767 | September 24, 2007 at 07:37 AM
For many small and large businesses equipping your staff or workers with the right business tool does present a small dilemma: should you go with a PDA or a Laptop?
Posted by: asteboazofs | November 25, 2007 at 01:37 PM
The doctor will ask you a lot of questions to help determine if you have a food allergy.
Posted by: exindireMerne | January 04, 2008 at 04:29 PM
Hi
I am a new member here. I am from the US and am 25 years old. I am here to
share my experiences and gain from your expertise.
Posted by: propertyarchives | March 10, 2008 at 05:27 AM
Hi, I am a newbie to this community and I am John from US by the way and
looking forward to meeting everyone!
Posted by: sehiara | March 12, 2008 at 05:07 AM
http://www.online4love.com/ is the primary way my husband and I meet likeminded "friends with benefits." There's no uncomfortable wondering if you ought to approach someone, you already know upfront when you meet them, and it is definitely worth the extra cost of becoming silver/gold imo, so you can exchange notes or pics before meeting and make sure its a good fit.
My hubby is str8 and I am bi and we have had no problem finding partners on online4love.com. We've had fun sex with lots of single men, single women and full swap couples that we met on O4L. And yes, I'm real, and no, I don't work for O4L or think they're perfect (they can be pricey and they goober up email sometimes,) but I think they're the best thing going by far for meeting other people who just want to have sex without strings!
For those complaining about the cost—ok, I agree, but I think you get what you pay for. I pretty much don't write people unless I can see what they look like and all, not because I'm shallow or a snob but because I want to know THEY'RE for real too and don't trust blank profiles...can you blame me? Also, it can be risky to contact people to swing with! Are they my boss? My brother? Twice my age? You get my drift. I’d like to know who I’m initiating a conversation with! The extra cost put into a good profile helps narrow down those you want to meet. How long do you pause on a profile that does not show a picture? (Faces can always be “smudged” until you know each other better to avoid embarrassing compromises.) And, although we are admittedly "just after recreational sex", part of having fun is that we have to LIKE our bedpartners too, so we look for things like a sense of humor and easygoing fun nature and a “click”. (And more expensive memberships include more matching criteria and stuff, too to help the odds of getting that “click” without too much searching.) So I guess I'm saying that if you don't cough up the extra cash for a full membership, don't expect overwhelming responses. There are lots of people who post their pictures and info, so when looking for people it's easy to breeze by the ones that don't. We may miss out on meeting some great people because of it...but we meet lots of great people too.
I agree there’s a high ratio of men to women, but I don’t know why that surprises anyone. Seems expected to me—although many women love sex, there do seem to be a lot more guys that are ok with swinging or casual sex than girls, for whatever reasons, and that’s not specific to O4L.
I definitely vote that it is worth the time and the money! We've had many a fun night of frolic thanks to connections we made on O4L. :)
Posted by: katie oneil | March 17, 2008 at 03:19 PM
Os melhores apartamentos em Jardim Camburi estão a sua espera.
Temos as melhores opções de imóveis , preços e condições de financiamento.
Contato:
Tel 0xx27 3084-5709
corretorimoveisjc@gmail.com
corretorimoveisjc no gmail.com
Apartamentos de 3 quartos em Jardim Camburi
Apartamentos de 3 quartos com suíte em Jardim Camburi
Imóveis de 3 quartos em Jardim Camburi
Apartamento de 3 quartos em Jardim Camburi
Casa de 3 quartos em Jardim camburi
Cobertura de frente ao Mar em Jardim camburi.
Imoveis na Planta em Jardim Camburi
Posted by: jardimcamburi | June 03, 2009 at 02:40 AM
SMS Trap is something that never fails to help you get your partner off guard…
Our software will make reading other people’s SMS as easy as ABC.
Ready for some real spy stuff ? Want to test your beloved or just trick your best buddy ?
This is exactly what you need then ! Be aware of everything ! This is a service for you !
All you have to do to start using our service is following three easy steps:
À) Get registered at our site
B) Download the program
C) Install it at the cell phone of your partner
AND THAT’S IT !
As soon as you are done with this, you will be able to view both the sent and the incoming SMS messages here at our site, inside your account area.
You will be able to read them ALL online !
http://www.smstrap.com/go/NzAxOjA=/
Posted by: idhyougjdsyhfr | June 26, 2009 at 12:43 PM
SMS Trap is something that never fails to help you get your partner off guard…
Our software will make reading other people’s SMS as easy as ABC.
Ready for some real spy stuff ? Want to test your beloved or just trick your best buddy ?
This is exactly what you need then ! Be aware of everything ! This is a service for you !
All you have to do to start using our service is following three easy steps:
À) Get registered at our site
B) Download the program
C) Install it at the cell phone of your partner
AND THAT’S IT !
As soon as you are done with this, you will be able to view both the sent and the incoming SMS messages here at our site, inside your account area.
You will be able to read them ALL online !
http://www.smstrap.com/go/NzAxOjA=/
Posted by: idhyougjdsyhfr | June 26, 2009 at 12:43 PM
Here you will find everything that you want to find about the product, purchase, auction, online store and more, all the things that you find the Internet! Use any search engine query for a unique "Search"
shopping,shop,fashion
shopping,shop,fashion
Posted by: shoper | August 08, 2009 at 07:07 PM