We continue with out series on Beauty Skepticism. Today’s topic, anecdotal evidence. What is it? What good is it? And what should you do with it?
What is anecdotal evidence?
We frequently use the phrase “anecdotal evidence” to describe various press releases and advertisements we see and also 
comments posted here. Essentially, any statement presented as proof of something that features no scientific support is considered anecdotal evidence. It’s called anecdotal because it often involves some kind of story or anecdote.
In the beauty / cosmetic world, you find examples everywhere.
- When your hairdresser says, “I can always tell when someone uses Pantene because there is a layer of wax on their hair,” this is anecdotal evidence.
- When a friend says, “my best friend had severe allergies and medicine didn’t work. When she tried acupuncture she got better,” this is anecdotal evidence.
- When you read a beauty product review that states whether a product worked or not, this is anecdotal evidence.
- Testimonials from celebrities, your mother-in-law, or even your best friend are all forms of anecdotal evidence.
The one thing that is true about all these examples is that they are inherently unreliable. You have no way of knowing from the statements whether they are true or not. This is why anecdotal evidence is not useful for figuring out truth and why it is of limited value in science.
No matter who you get it from, you should regard all anecdotal evidence as useless for determining whether something is true or not.
What are the problems with anecdotal evidence?
There are various problems with anecdotal evidence that make it unreliable and should raise your beautiful skeptical senses.
A. It could be a mistaken conclusion. While parts of anecdotal evidence can often be true, the conclusions that it leads to is often wrong. For example, we get comments from people all the time that say “Pantene made my hair fall out.”
While it might be true that they used Pantene and they noticed that some of their hair fell out, it does not logically follow that Pantene caused their hair to fall out. There are numerous other better explanations which have nothing to do with Pantene. Only through a carefully controlled scientific experiment can the cause of their hair loss be determined.
B. It could be mistaken perception. There is a thing in the beauty business called the “halo effect.” This is characteristic of beauty products that causes people to report that a product works better even though there is no reason it should. When beauty companies change things like color, fragrance, packaging, brand name, etc. people will often insist that the product is improved. In fact, a simple way to improve the moisturizing effect of a shampoo is to just take a regular shampoo and call it a “moisturizing” shampoo.
That’s why when someone tells you a beauty product worked in a certain way, you can’t always believe that it will work for you. Maybe you aren’t as susceptible to the “halo effect.”
C. It could be the result of a faulty memory. People’s memories are fallible. We mistakenly remember things much more often than we are willing to admit. When someone reports to you that they used a product a few months back and it made their acne go away, they might just be remembering it wrong. It’s possible that the product didn’t work at all. They just remember that it did.
D. It could be a lie. Whenever someone is trying to sell you something, they have an incentive to be convincing. Sometimes this is an innocent bias that makes them exaggerate the benefits of their products. Other times, they make blatant lies to convince you. You can easily find lies all over the Internet about various beauty products. In fact, any review you read could simply be a person hired by the company to write something positive about the product.
E. It could be case specific. People are different. When a beauty product works for one person, that doesn’t mean it will have the same effect on someone else. If your friend says that the Clairsonic skin system worked wonderfully for her skin, that doesn’t mean it will work wonderfully for yours. You are genetically different. The evidence from your friend may not be applicable to you.
What is the value of anecdotal evidence?
While anecdotal evidence isn’t valuable for figuring out the truth, it is not completely worthless. Beauty product reviews can be helpful, especially the parts where the reviewer tells you details about using the product. This won’t tell you whether the product will work for you or not but it may help you figure out if you will enjoy using it.
Anecdotal evidence can also be used to help generate new hypotheses for future testing. While there isn’t yet any proof that cosmetic acupuncture has any positive effect, the positive experience of people who have gotten the treatment suggests there is something about the procedure that’s worth testing. And if people keep telling Pantene that the product is making their hair fall out, you can bet they will scientifically test this assertion at some point in the future. (Knowing P&G they probably already have).
What to do about anecdotal evidence
When you read a headline or hear some beauty advice, your best bet is to assume it is not true. Usually people aren’t trying to lie to you, but their conclusions are almost always wrong. Real knowledge takes numerous experiments and many years to develop. If these experiments aren’t done you can never be sure what is real and what is a beauty delusion.
Real knowledge takes numerous experiments and many years to develop
Beauty Brains bottom line
Anecdotal evidence is not difficult to identify and is rarely exactly true. If you want to be your beautiful best without being duped into buying beauty bogus, stay skeptical about all the unsupported anecdotal evidence you read or hear.
Loading ...
{ 1 trackback }
{ 7 comments… read them below or add one }
I have been waiting for this portion of you beauty science review for a while now – thank you. Anecdotal evidence is nice for stories, but it has very little to do with science.
I’d like to add that anecdotal evidence is inherently biased because people tend to share such testimonials only when they are particularly pleased or displeased with a product.
Plenty of people try something, aren’t impressed either way, and never speak of their experiences because they just don’t feel compelled to. Or if the product was ineffective or even really bad, they might not say anything because they don’t want people to think that they were dumb enough to waste money on that product.
I think the problem with anecdotal evidence is that it’s a starting point, not a conclusion.
If people say, “Pantene seems to make my hair fall out,” that’s where the scientific method STARTS. That’s when you say, “Hm, let’s test this — let’s design an experiment, let’s try to couble-blind it, let’s make sure we have a lot of data points and that we publish so other people can check what we say.” An anecdote is like the key to the car — it turns the crank and gets the whole thing started.
The problem is that people use anecdotes as the endpoint, and not the beginning. Instead of using an anecdote as the beginning of a process to see what’s really happening, they stop there. “Pantene makes my hair fall out, end of story.”
An anecdote is the FIRST brick on the road to knowledge, not the LAST. If you stop there, you never actually get on the road.
I’m only really fond of anecdotal evidence/studies when it’s studying something rare enough that you can’t necessarily find enough people with the same experience to create a generalizable sample. They’re interesting for rare diseases/phenomena but even then they’re just a starting point.
What Janis said, totally! I like this series and this part in particular. I still think a lot of this is due, on the US’s part, to the fact that critical reasoning is so absent in American education (and culture, really) and science is so /scary/. I like the way you’re explaining these: simple and to the point. I heard that’s the best way to explain things. ^_~
Thanks for this skepticism series! As a non-scientist who unfortunately has made it through high school and college with almost no science education, I have a question for you scientists:
Is there any value in collective anecdotal evidence such as product ratings on sites like drugstore.com, makeupalley, and folica? If so, how many individual product ratings need to be averaged for an average rating to be meaningful?
If over 100 people give a product 4 out of 5 stars, I might try it. If over 100 people give it 1 star, I probably wouldn’t buy it. If only a few people rate a product, I ignore the rating. What do you think of this use of anecdotal evidence?
That’s a good question @professorauntie, I hope someone from the Brains answer it.