How to Read This Research

This site presents research summaries drawn from the peer-reviewed behavioral science literature. If you do not have a background in academic research, some of the terminology and conventions used in our summaries may be unfamiliar. This guide is designed to help you evaluate what you read here -- and, frankly, what you read on any website that claims to be "science-based."

We want you to be a critical reader. That includes being critical of us.

What Is a Meta-Analysis?

A single study, no matter how well designed, tells you what happened in one sample, in one context, at one time. It might be an anomaly. The results might not replicate. The sample might be unrepresentative. This is not a flaw in the scientific process -- it is the nature of empirical inquiry.

A meta-analysis addresses this limitation by statistically combining the results of multiple independent studies that examined the same question. If twelve different research teams, working with different populations in different settings, all find that a particular intervention improves wellbeing, that convergence is far more informative than any single study alone.

Meta-analyses occupy a high position in the evidence hierarchy for good reason: they reduce the influence of any one study's idiosyncrasies, provide more precise estimates of how large an effect actually is, and can reveal patterns that individual studies lack the statistical power to detect. When we cite meta-analytic findings on this site, we are pointing to the strongest form of evidence available in the behavioral sciences.

That said, meta-analyses are not infallible. Their conclusions depend on the quality of the studies they include, the decisions made about which studies to include or exclude, and the statistical methods used to combine results. A meta-analysis of poorly designed studies will produce a precise but potentially misleading answer.

What Are Effect Sizes?

One of the most important concepts for evaluating research is the distinction between statistical significance and practical significance.

Statistical significance tells you whether an observed effect is likely to be real (i.e., not due to chance). It does not tell you how large that effect is. With a large enough sample, even a trivially small effect will be statistically significant.

Effect size tells you how large the effect actually is. The most common measure is Cohen's d, which expresses the difference between groups in standardized units. A widely used interpretive scale is:

• Small effect (d = 0.2): A real but modest difference. You probably would not notice it in daily life without careful measurement. Many educational and public health interventions produce small effects, which can still be meaningful at a population level.
• Medium effect (d = 0.5): A noticeable difference. You might perceive it through careful observation. This is roughly the average effect size in social psychology research.
• Large effect (d = 0.8): A substantial difference. Likely visible without formal measurement.

In our research summaries, we report effect sizes where available and use language calibrated to their magnitude. When we describe an effect as "robust" or "substantial," we mean large effect sizes replicated across studies. When we describe an effect as "modest but reliable," we mean a small effect size that appears consistently.

How to Evaluate Citations

Not all sources are created equal. Here is a rough hierarchy of evidence quality, from strongest to weakest:

1. Systematic reviews and meta-analyses published in peer-reviewed journals
2. Randomized controlled trials (RCTs) -- experiments where participants are randomly assigned to conditions
3. Longitudinal studies -- research tracking the same individuals over time
4. Cross-sectional studies -- research measuring associations at a single point in time
5. Expert opinion and theoretical frameworks -- informed but not empirically tested
6. Popular press and self-help books -- may reference science but often selectively and inaccurately

On this site, we draw primarily from categories 1 through 4. We cite specific studies with author names and publication years so that you can locate the original work. When we rely on theoretical frameworks (category 5), we identify them as such.

A useful rule of thumb: if a website makes scientific claims without citing specific peer-reviewed studies, treat those claims with skepticism. "Research shows" without a citation is not evidence -- it is rhetoric.

How to Check Our Sources

We encourage you to verify our claims. Here is how:

Google Scholar (scholar.google.com) is the most accessible tool for locating peer-reviewed research. Enter the author name and year we cite (e.g., "Sheldon Elliot 1999"), and you will typically find the original paper. Many papers include freely available abstracts that summarize the key findings.

University library databases such as PsycINFO, Web of Science, and PubMed provide access to full-text articles, often through institutional affiliations. If you are affiliated with a university, you likely have free access to these databases.

Sci-Hub and open access repositories provide broader access to published research, though the legal status of some of these services varies by jurisdiction.

Preprint servers such as PsyArXiv host pre-publication versions of papers that may not yet have undergone peer review. We generally do not cite preprints unless they have subsequently been published in peer-reviewed venues.

If you find that our summary of a study does not accurately represent the original findings, we want to know. Contact us at research@activationscience.edu.

What "Statistically Significant" Means (and Does Not Mean)

"Statistically significant" is probably the most misunderstood phrase in science communication. It does not mean "important." It does not mean "large." It does not mean "proven."

What it means is this: the observed result is unlikely to have occurred by chance alone, given the assumption that there is no real effect. The conventional threshold is a probability of less than 5% (p < .05) -- meaning that if there were truly no effect, you would expect to see a result this extreme less than 5% of the time.

This is a useful but limited piece of information. A statistically significant result with a tiny effect size tells you that something real is probably happening, but that "something" might be too small to matter in practice. Conversely, a non-significant result does not prove that no effect exists -- it may simply mean the study lacked the statistical power (usually due to a small sample) to detect it.

When reading our research summaries, pay more attention to effect sizes and the consistency of findings across studies than to p-values. A medium effect size replicated across ten studies is far more informative than a single study with a very small p-value.

A Note on Our Approach

We present narrative reviews of peer-reviewed literature. While we strive for accuracy, completeness, and balanced representation of the evidence, we encourage readers to consult the original studies for complete findings. Our summaries are intended to make the research accessible, not to replace engagement with the primary literature.

We are also transparent about what our reviews are not. They are not systematic reviews conducted according to formal protocols such as PRISMA guidelines. They are not original empirical research. They represent our best effort to accurately synthesize the available evidence on topics relevant to the Activation Science framework, and we acknowledge that selection and interpretation biases, however carefully guarded against, can never be fully eliminated.

Your skepticism is welcome. Good science depends on it.