• Visit
    • Calendar
    • After Dark Thursdays
    • Buy Tickets
    • Exhibits
    • Museum Galleries
    • Artworks on View
    • Hours
    • Getting Here
    • Visitor FAQ
    • Event Rentals
  • Education
    • Professional Development Programs
    • Free Educator Workshops
    • Tools for Teaching and Learning
    • Learning About Learning
    • Community Programs
    • Educator Newsletter
  • Explore
    • Browse by Subject
    • Activities
    • Video
    • Exhibits
    • Apps
    • Blogs
    • Websites
  • About Us
    • Our Story
    • Partnerships
    • Global Collaborations
    • Explore Our Reach
    • Arts at the Exploratorium
    • Contact Us
  • Join + Support
    • Donate Today!
    • Membership
    • Join Our Donor Community
    • Engage Your Business
    • Attend a Fundraiser
    • Explore Our Reach
    • Thank You to Our Supporters
    • Donor & Corporate Member FAQ
    • Host Your Event
    • Volunteer
  • Store
  • Visit
    • Frequently Asked Questions
    • Calendar
      • Today
      • This Week
      • Online
      • After Dark Thursday Nights
      • Arts
      • Conferences
      • Cinema Arts
      • Free + Community Events
      • Fundraising Events
      • Kids + Families
      • Members
      • Highlighted Artworks + Exhibits
      • Special Hours
      • Private Event Closures
    • Prices
    • Hours
    • Getting Here
    • Museum Map
    • Reduced Rates & Community Day
    • Accessibility
    • Tips for Visiting with Kids
    • How to Exploratorium
    • Exhibits
    • Tactile Dome
    • Artworks on View
    • Cinema Arts
    • Kanbar Forum
    • Black Box
    • Museum Galleries
      • Bernard and Barbro Osher Gallery 1: Human Phenomena
        • Tactile Dome
          • 1971 Press Release
        • Black Box
        • Curator Statement
      • Gallery 2: Tinkering
        • Curator Statement
      • Bechtel Gallery 3: Seeing & Reflections
        • Curator Statement
      • Gordon and Betty Moore Gallery 4: Living Systems
        • Curator Statement
      • Gallery 5: Outdoor Exhibits
        • Curator Statement
      • Fisher Bay Observatory Gallery 6: Observing Landscapes
        • Wired Pier Environmental Field Station
        • Curator Statement
    • Restaurant & Café
    • School Field Trips
      • Getting Here
        • Bus Routes for Field Trips and Other Groups
      • Admission and Tickets
      • Planning Guide
      • Reservations
        • Field Trip Request Form
      • Resources
    • Event Rentals
      • Full Facility & Gallery Bundles
      • Fisher Bay Observatory Gallery & Terrace
      • Moore East Gallery
      • Bechtel Central Gallery & Outdoor Gallery
      • Osher West Gallery
      • Kanbar Forum

      • Weddings
      • Proms and School Events
      • Daytime Meetings, Events, & Filmings

      • Rentals FAQ
      • Event Planning Resources
      • Rental Request Form
      • Download Brochure (pdf)
    • Exploratorium Store
    • Contact Us
  • Education
    • Black Teachers and Students Matter
    • Professional Development Programs
      • Free Educator Workshops
      • Professional Learning Partnerships
      • Teacher Institute
        • About the Teacher Institute
        • Summer Institute for Teachers
        • Teacher Induction Program
        • Leadership Program
        • Teacher Institute Research
        • CA NGSS STEM Conferences
          • NGSS STEM Conference 2020
        • Science Snacks
          • Browse by Subject
          • Special Collections
          • Science Snacks A-Z
          • NGSS Planning Tools
          • Frequently Asked Questions
        • Digital Teaching Boxes
        • Meet the Teacher Institute Staff
        • Resources for Supporting Science Teachers
      • Institute for Inquiry
        • What Is Inquiry?
        • Watch and Do Science
        • Inquiry-based Science and English Language Development
          • Educators Guide
            • Conceptual Overview
              • Science Talk
              • Science Writing
            • Classroom Video Gallery
              • Magnet Investigation
              • Snail Investigation
            • Teacher Professional Development
            • Project Studies
            • Acknowledgments
          • Conference: Exploring Science and English Language Development
            • Interviews with Participants
            • Plenary Sessions
            • Synthesis, Documentation, and Resources
        • Workshops
          • Participant Portal
          • Fundamentals of Inquiry
            • Summary Schedule
          • BaySci Science Champions Academy
          • Facilitators Guides
          • Commissioned Workshops
        • Resource Library
        • Meet the IFI Staff
      • Resources for California Educators
      • K-12 Science Leader Network
      • Resources for Supporting Science Teachers
      • Field Trip Explainer Program
      • Cambio
    • Tools for Teaching and Learning
      • Learning Toolbox
      • Science Snacks
      • Digital Teaching Boxes
      • Science Activities
      • Tinkering Projects
      • Recursos gratuitos para aprender ciencias
      • Videos
      • Exhibits
      • Publications
      • Apps
      • Educator Newsletter
      • Exploratorium Websites
    • Educator Newsletter
    • Advancing Ideas about Learning
      • Visitor Research and Evaluation
        • What we do
        • Reports & Publications
        • Projects
        • Who we are
      • Center for Informal Learning in Schools
    • Community Programs
      • High School Explainer Program
      • Xtech
      • Community Educational Engagement
      • California Tinkering Afterschool Network
        • About
        • Partners
        • Resources
        • News & Updates
        • Further Reading
  • Explore
    • Browse by Subject
      • Arts
      • Astronomy & Space Sciences
        • Planetary Science
        • Space Exploration
      • Biology
        • Anatomy & Physiology
        • Ecology
        • Evolution
        • Genetics
        • Molecular & Cellular Biology
        • Neuroscience
      • Chemistry
        • Combining Matter
        • Food & Cooking
        • Materials & Matter
        • States of Matter
      • Data
        • Data Collection & Analysis
        • Modeling & Simulations
        • Visualization
      • Earth Science
        • Atmosphere
        • Geology
        • Oceans & Water
      • Engineering & Technology
        • Design & Tinkering
        • Real-World Problems & Solutions
      • Environmental Science
        • Global Systems & Cycles
        • Human Impacts
      • History
      • Mathematics
      • Nature of Science
        • Measurement
        • Science as a Process
        • Size & Scale
        • Time
      • Perception
        • Light, Color & Seeing
        • Listening & Hearing
        • Optical Illusions
        • Scent, Smell & Taste
        • Tactile & Touch
      • Physics
        • Electricity & Magnetism
        • Energy
        • Heat & Temperature
        • Light
        • Mechanics
        • Quantum
        • Sound
        • Waves
      • Social Science
        • Culture
        • Language
        • Psychology
        • Sociology
    • Browse by Content Type
      • Activities
      • Blogs
        • Spectrum
          • Arts
          • Behind the Scenes
          • News
          • Education
          • Community & Collaborations
          • Science
        • Eclipse
        • Studio for Public Spaces
        • Tangents
        • Resonance See & Hear Blog
        • Fabricated Realities
        • Tinkering Studio: Sketchpad
        • Exploratorium on Tumblr
      • Exhibits
      • Video
      • Websites
      • Apps
        • Total Solar Eclipse
  • About Us
    • Our Story
    • Land Acknowledgment
    • Explore Our Reach
    • Impact Report
    • Fact Sheet
    • Awards
    • Our History
      • 50 Years 1969–2019

    • Senior Leadership
    • Board of Trustees
    • Board of Trustees Alumni
    • Staff Scientists
    • Staff Artists

    • Arts at the Exploratorium
      • Artworks on View
      • Artist-in-Residence Program
      • Cinema Arts
        • History and Collection
        • Cinema Artists-in-Residence
        • Resources and Collaborating Organizations
        • Kanbar Forum
      • Center for Art & Inquiry
        • Begin Here
          • Lessons
            • Bob Miller/Light Walk
            • Ruth Asawa/Milk Carton Sculpture
          • Workshops
      • Resonance
        • About the Series
        • See & Hear
        • Past Seasons
      • Over the Water
      • Black Box
      • Upcoming Events
      • Temporary Exhibitions
      • Arts Program Staff
    • Teacher Institute
    • Institute for Inquiry
    • Explainer Programs
    • Studio for Public Spaces
    • Exhibit Making
    • Partnerships
      • Building Global Connections
        • Global Collaborations
          • Projects
          • Approach
          • People
          • Impact
      • Partnering with Science Agencies
        • NASA
        • NOAA
      • Partnering with Educational Institutions
      • Osher Fellows

    • Job Opportunities
    • Become a Volunteer

    • Contact Info
    • Newsletter
    • Educator Newsletter
    • Blogs
    • Follow & Share
    • Press Office

    • FY20 Audit Report
    • 990 FY19 Tax Return
    • Use Policy
      • Privacy Policy
      • Intellectual Property Policy
  • Join + Support
    • Donate Today!
    • Membership
      • Membership FAQ
      • Member Benefits
      • After Dark Membership
      • Member Events
      • May Is for Members
    • Join Our Donor Community
    • Engage Your Business
      • Corporate Membership
      • Luminary Partnerships
    • Attend a Fundraiser
      • Wonder Funday
      • Science of Cocktails
      • Party at the Piers
        • Event Leadership and Host Committee
    • Explore Our Reach
    • Thank You to Our Supporters
    • Donor & Corporate Member FAQ
    • Volunteer
      • How to Apply
      • Application for Internships
      • Our Contract
      • Application for Individuals
  • Press Office
    • Press Releases
    • News Coverage
    • Events Calendar
    • Fact Sheet
    • Photographs
    • Press Video
    • Press Kits
    • Press Visits
    • Exploratorium Logos
    • Recent Awards
    • Praise for the Exploratorium
    • Join Our Press List
  • Store

Masks are required for all visitors 2+. Vaccines recommended. Plan your visit  

Visitor FAQ Buy Tickets Donate Today
Exploratorium
Exploratorium
  • Visit
    • Calendar
    • After Dark Thursdays
    • Buy Tickets
    • Exhibits
    • Museum Galleries
    • Artworks on View
    • Hours
    • Getting Here
    • Visitor FAQ
    • Event Rentals
  • Education
    • Professional Development Programs
    • Free Educator Workshops
    • Tools for Teaching and Learning
    • Learning About Learning
    • Community Programs
    • Educator Newsletter
  • Explore
    • Browse by Subject
    • Activities
    • Video
    • Exhibits
    • Apps
    • Blogs
    • Websites
  • About Us
    • Our Story
    • Partnerships
    • Global Collaborations
    • Explore Our Reach
    • Arts at the Exploratorium
    • Contact Us
  • Join + Support
    • Donate Today!
    • Membership
    • Join Our Donor Community
    • Engage Your Business
    • Attend a Fundraiser
    • Explore Our Reach
    • Thank You to Our Supporters
    • Donor & Corporate Member FAQ
    • Host Your Event
    • Volunteer
  • Store

Can You Believe It? Seven Questions to Ask About Any Scientific Claim

by Pearl Tesler • December 15, 2016

  • Facebook logo
  • Reddit logo
  • Twitter logo

1. What’s the claim?

2. Who says?

3. What’s the evidence?

4. How did they get the evidence?

5. Is there anything (or anyone) to back up this claim?

6. Could there be another explanation?

7. Who cares?

Screaming headlines abound in our media-saturated world.  “Killer Moths Invade Homes.” “New Drug Promises Alzheimer’s Cure.”  “Experts Confirm: Sky Is Falling.”

Some are obvious click bait, but others seem legit—especially when seemingly backed by science. Inevitably, we’re all regularly confronted with scientific facts and factoids. Claims about household products, technology, medicine, and even politics often come steeped in the presumed authority of scientific research.

The truth is slippery, and it’s not always graspable even by experts. So how can we non-experts decide what to believe?

The seven questions here can help you weigh the validity of scientific information, wherever it might appear. You may not be able to get answers to them all—and this in itself might be telling—but if you can, you’ll be well on your way to separating science fact from science fiction.

 

1. What's the claim?

Simple as it might sound, the first step toward weighing a scientific claim is to establish what it is, as well as what it isn’t.

Read or listen carefully. What exactly is the claim? Where does it sit on the spectrum from likely to outlandish? Do the findings confirm or challenge existing beliefs?

Try to read between the lines when you’re assessing the validity of a claim. Ask yourself: What aren’t they saying? A reputable source will acknowledge missing pieces of the puzzle, or areas where more research is needed.

Finally, don’t be tricked into confusing correlation with causation. Correlation is when two things change together: the relationship could be chance, or there could be a third variable causing both changes. Causation, on the other hand, is a direct cause-and-effect relationship between two things.

Suppose you read that high rates of violent crime are associated with increased sales of ice cream. Ice cream sales may be correlated with violent crimes—both might increase with warmer

 

2. Who says?

They say you’re only as good as your reputation . . . but who is “they?”

Any decent claim requires that someone stand behind it—preferably a well-respected source from an equally well-respected institution. You may not know the reputation of the scientist or institution involved, but chances are you can find out.

Once you figure out who did the research and where, you can go further by finding where the research was originally published. Most respected scientific journals are peer-reviewed, which means that other scientists read the articles vying for publication and screen out any shoddy science.

Beware science stories that go directly to mainstream media, also known as “science by press release.” This can be a ploy to circumvent the peer-review process. Notorious examples include a 1989 press conference announcing successful cold fusion and a 2002 press conference announcing successful human cloning. Both stories were later debunked.

Last but not least, it never hurts to find out who paid for the research. Research funded by sources with vested interests (drug companies and advocacy groups, for example) should be given extra scrutiny. Some manufacturers publicize the positive aspects of their product while burying any research that doesn’t support their desired outcome.

3. What's the evidence?

Evidence is the bread and butter of science. Reaching a scientific conclusion of any kind requires observation and measurement—ideally, the careful, repeated observation and measurement known as empirical evidence.

Evidence can take many forms, because research itself can take many forms. Sometimes, evidence may appear pictorially as a chart or graph. Pay careful attention to the labels and scales on graphs and charts, because just like words, visuals can mislead, as well as tell hidden stories.

Whatever form evidence takes, it's likely to be at least partly numeric. Alas, it's at precisely the moment when numbers appear that most people begin to tune out. That's unfortunate because numbers can't (usually) lie, which is why looking at the actual evidence can be most illuminating in evaluating a claim. For starters, how much data was collected? You don't need a degree in statistics to know that the more people there are involved in a study, the less likely it is that the results are just chance.

Sometimes, a claim may be made with no empirical evidence at all. File these claims under “S” for speculation. In other cases, a claim may rest on evidence that is limited or downright scanty. In paleontology, for example, where preserved specimens of ancient life are few, entire theories may rest precariously on the discovery of a single bone. In physics, string theory redefines the universe without any evidence at all. String theory holds that everything in our universe results from vibrations of miniscule strings, but no one has figured out how to test if the theory is true.

 

4. How did they get the evidence?

Where data collection is concerned, the devil is in the details. Exactly how measurements are made, with what equipment, and under what conditions, can have make-or-break significance.

Methodologies are important not just in polling situations, but in every science—even in the “hard” sciences, where measurements may be made using billion-dollar machines. No matter the field, data collected one way may support one conclusion; data collected another way may support a completely different conclusion.

Huh? This is science, isn’t it? Actually, the process of collecting data is fraught with error. First of all, there is no such thing as an exact measurement—all results contain a certain unavoidable fudge factor called error, the result of living in an imperfect, imprecise world. Then there’s the possibility of a systematic error, a flaw in a measuring device or method that skews the data one way or another. Uncontrolled variables can play evil tricks on data, too; these are factors that influence results but haven’t been taken into account, possibly because no one even knows about them.

So ask yourself: What methods were used to collect the evidence for this claim? Are the methods even explained? Be warned that even methods that seem reasonable may rest on false assumptions. One hundred years ago, scientists perfected methods of estimating human intelligence by measuring the volume of a person’s brain cavity. The method of measurement was fine, but the underlying assumption—that brain size predicts intelligence—was bogus.

 

5. Is there anything (or anyone) to back up this claim?

No one—not even an astrophysicist—works in a void. All research takes place in the context of what we currently believe to be true, and this context can either lend credibility to a claim or erode it. The newer and stranger the result, the greater the burden of proof.

How does this claim compare to other studies on the same subject? Is there consensus in the field? Who disagrees, and why?

Scientists form a community, and as in all communities, not everyone is in perfect agreement. Even so, if there’s one thing all scientists agree on it’s reproducibility. For one person’s research to be believable, other people using the same tools or techniques must be able to produce the same result.

Has anyone else in the field verified the result? If a researcher is using a new tool or technique, are there other tools or techniques that can verify the result? Searching online for other articles on the same topic is an easy way to find a second opinion—and often a third and fourth, to boot.

 

6. Could there be another explanation?

Sometimes, it’s not the research methods or the data that are flawed, but the interpretation of the data.

It’s human nature to see what we’re looking for—whether it’s really there or not—and not see what we’re not looking for. Scientific truth sometimes falls prey to this tendency of ours, when scientists inadvertently leap to conclusions their research doesn’t really support.

For a classic (and literal) example of just such a logical “leap,” consider the story of Italian anatomist Luigi Galvani who, in 1871, poked a brass hook into one of the frog legs he was preparing for dissection. When he saw the leg jump, he wrongly attributed the phenomenon to “animal electricity,” a then-popular concept that animal tissues contained a reservoir of electricity that gave them life.

Actually, it wasn’t the frog leg that had produced the electricity, but contact between the brass hook and the iron railing from which it hung—a misunderstanding not corrected until years later. Galvani didn’t realize it, but he hadn’t discovered proof of “animal electricity” at all. He’d discovered the battery.

Sometimes researchers will admit to other possible interpretations of their results, but mistakes are often lodged hopelessly where no one can see them (yet), within the dominant paradigm. All science is necessarily provisional; today’s facts become tomorrow’s fiction as new measuring tools, new discoveries, and new paradigms continually expand our knowledge and understanding.

 

7. Who cares?

There are always people interested in the outcome of scientific research—other researchers within and outside the field, funders, special-interest groups, manufacturers, and anyone else who needs the information to make personal or policy decisions.

All research happens in a social context, and that context can be at least as important as the claim itself. Bias and predispositions can affect whether research happens at all, whether and how the results are made public and, most importantly, how results are “spun” and interpreted both inside and outside the scientific community.

Who supports the claim, and who doesn’t? What are their biases? Who funded the research? Why? Be extra wary of research that was either funded or conducted by someone with something to gain.

Finally, keep in mind the hype factor. Big news sells, and science stories are easy prey for people looking to make mountains out of molehills. Never take headlines or sound bites at face value. There’s almost always more to the story.

....................

This content is from How Do We Know What We Know? Resources for the Public Understanding of Scientific Evidence, which was made possible by a grant from the National Science Foundation with the additional generosity of the Gordon and Betty Moore Foundation, The Jim Clark Endowment for Internet Education, and the McBean Family Foundation.

 

Download Booklet
This folded booklet contains all seven questions in a handy to-go format.

evidence
evidence based science
peer-review
scientific literacy
media literacy
fake news
informed citizenry

Blogs

  • Spectrum
    • Arts
    • Behind the Scenes
    • News
    • Education
    • Community & Collaborations
    • Science
  • Eclipse
  • Studio for Public Spaces
  • Tangents
  • Resonance See & Hear Blog
  • Fabricated Realities
  • Tinkering Studio: Sketchpad
  • Exploratorium on Tumblr

Recent Contributors

  • Exploratorium Staff
  • Mary Miller
  • Exploratorium Staff
  • Heike Winterheld
Exploratorium
Visit
Join
Give

Pier 15
(Embarcadero at Green Street)
San Francisco, CA 94111
415.528.4444

Contact Us

  • Plan Your Visit
  • Calendar
  • Buy Tickets
  • Getting Here
  • Store
  • Event Rentals
  • About Us
  • Become a Member
  • Donate
  • Jobs
  • Volunteer
  • Press Office
  • Land Acknowledgment

Get at-home activities and learning tools delivered straight to your inbox

The Exploratorium is a 501(c)(3) nonprofit organization. Our tax ID #: 94-1696494
© 2022 Exploratorium | Terms of Service | Privacy Policy | Your California Privacy Rights |