Category: Science Explained
Not all reviews are equal. A systematic review is the difference between an expert cherry-picking studies that suit their view and a transparent, repeatable method that hunts down all the evidence and appraises it honestly. Here is how it works and why it sits near the top of the evidence pile.
Category: Science Decoded | Reading time: ~9 min | Level: Intermediate
Two people can read the exact same research literature and reach opposite conclusions, and both can honestly call what they wrote a review. One combed through every study, wrote down the rules in advance, and reported the ones that undercut their hunch alongside the ones that supported it. The other read widely, formed a view, and quoted the studies that fit. From the outside, both look authoritative. Only one is systematic, and the gap between them is the difference between evidence and eloquence.
That distinction sits underneath almost every health claim you will ever weigh. When a supplement cites a review, or a headline says the research shows, the crucial question is what kind of review, done how. A systematic review is a specific, disciplined method, and understanding how it works, PRISMA, inclusion criteria and all, is what lets you tell a genuinely well-supported claim from a well-argued one. Here is the method laid out.
A systematic review gathers, appraises and summarises all the studies relevant to a clearly defined question, using a method that is decided in advance, written down, and transparent enough that someone else could repeat it [2]. That last property, repeatability, is the heart of it. The reviewers specify the exact question, how they will search for studies, which they will include, and how they will judge quality, before they start pulling conclusions [1][2].
The mechanism sentence to remember is that a systematic review is defined by its method, not its subject. It is comprehensive, it is pre-specified, and it is designed to minimise the reviewer's own bias, so that the conclusion reflects the whole body of evidence rather than a hand-picked slice of it [2][4]. Everything else, PRISMA, inclusion criteria, the place in the evidence hierarchy, is machinery in service of that one aim: find all the relevant evidence and appraise it honestly.
The single most important discipline in a systematic review is deciding, before looking at the results, which studies count. These are the inclusion criteria, and they typically specify the population, the intervention, the comparison, the outcomes and the study designs the reviewers will accept [2]. A review might, for instance, include only randomised controlled trials in adults, lasting at least eight weeks, measuring a defined outcome.
Setting these rules in advance is not bureaucratic box-ticking. It is the main defence against cherry-picking. If reviewers could choose which studies to include after seeing whether they supported the desired answer, they could build almost any conclusion they liked, consciously or not [2]. Pre-specified criteria close that door. They commit the reviewers to a fixed net, so the studies that fall into it are determined by the rules, not by the answer the reviewers hoped to reach. This is exactly the discipline a narrative review lacks.
Good methods are worth little if nobody can check them, which is where PRISMA comes in. PRISMA, the Preferred Reporting Items for Systematic Reviews and Meta-Analyses, is a widely used reporting standard that specifies what a systematic review should disclose: how the search was run, how many studies were found, how many were excluded and why, and how quality was assessed [1].
The distinction is worth being clear about. PRISMA does not conduct the review or judge the evidence. It makes the review transparent, so a reader can see the process from search to conclusion and judge whether it was thorough and fair [1]. A review that follows PRISMA usually presents a flow diagram tracking studies from the initial search down to the final included set, which is why you often see a chart with numbers dwindling at each stage. When methods are laid bare like this, you can trust the review not because you are told to, but because you can inspect how it was built.
Three terms get muddled, and separating them sharpens how you read evidence.
A narrative review is an expert's summary of a topic, drawn from their own reading and judgement, with no defined method for finding or selecting studies [4]. It can be genuinely insightful and is often a good introduction to a subject, but it is vulnerable to bias, because the author may feature the evidence that fits their view and pass over the rest. It is a considered opinion, not a comprehensive appraisal.
A systematic review, as described, uses a transparent, pre-specified method to find and appraise all relevant studies, which makes cherry-picking far harder [2]. It is designed to be comprehensive and repeatable in a way a narrative review is not.
A meta-analysis is different again: it is a statistical technique that combines the numerical results of several studies into a single pooled estimate [2]. A systematic review may include a meta-analysis when the studies are similar enough to combine sensibly, but it does not have to, and a meta-analysis ideally sits inside a systematic review rather than standing alone [2]. Put simply, the systematic review is the search and the appraisal; the meta-analysis is the maths that may or may not follow. Our dedicated piece on meta-analysis goes into the pooling in more depth.
A well-conducted systematic review sits near the top of the evidence hierarchy, and the reason is straightforward. Instead of resting on a single study, which might be small, poorly designed or simply a fluke, it gathers all the credible evidence on a question and appraises it as a body, using a method others could repeat [4]. That guards against building a conclusion on one unrepresentative result or on the reviewer's preferences.
But high in the hierarchy is not the same as infallible, and the failure modes matter. A systematic review inherits the quality of the studies inside it: a rigorous review of small, weak or industry-funded trials is still a review of weak evidence, a problem often summarised as garbage in, garbage out [2]. Reviews can also be distorted by publication bias, where studies with positive results are more likely to be published and therefore more likely to be found, skewing the pooled picture towards benefit [2]. Good reviews assess and report these risks rather than hiding them. So the honest reading is that a systematic review is strong evidence conditioned on the quality of what it contains, and its conclusions belong alongside its stated limitations.
The practical skill is to read past the label. When a claim cites a systematic review, that is a genuinely good sign, but it is the start of the check, not the end of it. Look at what the review actually concluded, because reviews frequently report modest, mixed or uncertain findings that marketing rounds up to proven. Look at the quality of the included studies, since a review of flawed trials carries their flaws. And look for the marks of a good review: a clear question, pre-specified inclusion criteria, a thorough search, PRISMA-style transparency, an appraisal of study quality and an honest discussion of bias.
Do that, and systematic review stops being a magic phrase and becomes what it should be, a signal to look one level deeper with confidence that the deeper look is possible. Our guide to how we grade evidence puts the systematic review in the full hierarchy, our piece on meta-analysis covers the statistical pooling, and our explainer on in vitro versus in vivo covers why the studies feeding a review need to be human to answer human questions.
There is a trust dimension here that goes beyond method. The whole point of a systematic review is to protect a conclusion from the wishes of the person writing it, and that is exactly the protection health claims most need. A field awash with enthusiastic narrative summaries and single striking studies is a field where confident, well-supported-sounding claims can rest on very little. The systematic review is the tool that calls that bluff.
For anyone making decisions about their health, this is why the type of evidence matters as much as the direction of it. A calm, honest systematic review reporting a modest or uncertain effect is worth more than a glowing claim built on one study, even though the claim sounds more convincing. Judging evidence by its method rather than its confidence is the single most protective habit a reader can build.
Pregnant, breastfeeding, or on medication? Check with a healthcare professional first.
The systematic review is close to the backbone of how we grade anything. When we describe what an ingredient does, we lean on the whole body of appraised evidence rather than the most flattering single study, and we weight a well-conducted systematic review above an isolated trial, while keeping in view that a review only reflects the quality of what it pooled. That is not caution for its own sake, it is how you keep a conclusion honest.
If you want to go further, our guide to how we grade evidence lays out the full hierarchy, our explainer on meta-analysis covers the statistical pooling that sometimes sits inside a review, and our piece on in vitro versus in vivo covers why the underlying studies need to be in people to mean much. You can also browse the Remedy Library to see how we describe evidence, strengths and limits alike.
1. Page MJ, McKenzie JE, Bossuyt PM, et al. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. PMID 33782057. Reporting standard for systematic reviews and meta-analyses. 2. Higgins JPT, Thomas J, Chandler J, et al. (2023). Cochrane Handbook for Systematic Reviews of Interventions. Methodological handbook covering inclusion criteria, appraisal, meta-analysis and bias. 3. National Center for Complementary and Integrative Health (2023). Know the science: study types and the evidence hierarchy. Government research guidance on evidence types. 4. Murad MH, Asi N, Alsawas M, Alahdab F (2016). New evidence pyramid. BMJ Evidence-Based Medicine. PMID 27339128. Review of the evidence hierarchy and where systematic reviews sit.
It is a structured way of answering a research question by finding and assessing all the relevant studies, rather than picking a few that suit a point of view. Reviewers set out in advance exactly what question they are asking, how they will search for studies, which ones they will include, and how they will judge quality. Then they carry out that plan transparently, so anyone could follow the same steps and reach the same set of studies. This method is what separates a systematic review from an ordinary opinion piece dressed up as a review.
PRISMA stands for Preferred Reporting Items for Systematic Reviews and Meta-Analyses. It is a widely used set of reporting standards that tell researchers what a systematic review should disclose, such as how they searched, how many studies they found, how many they excluded and why. PRISMA does not do the review, it makes the review transparent, so readers can see the process and judge whether it was thorough and fair. A systematic review that follows PRISMA is easier to trust because its methods are laid bare rather than hidden.
Inclusion criteria are the pre-set rules that decide which studies make it into a systematic review and which are left out. They typically specify the population, the intervention, the comparison, the outcomes and the study designs the reviewers will accept. For example, a review might include only randomised controlled trials in adults lasting at least eight weeks. Setting these rules in advance matters because it stops reviewers from quietly including studies that support a conclusion and excluding those that do not, which is how bias creeps in.
A narrative review is an expert's summary of a topic, written from their reading and judgement, with no defined method for finding or selecting studies. It can be insightful, but it is vulnerable to bias because the author may, consciously or not, feature the evidence that fits their view. A systematic review uses a transparent, pre-specified method to find and appraise all relevant studies, so it is far harder to cherry-pick. Both are called reviews, but only the systematic kind is designed to be comprehensive and repeatable.
They are related but not the same. A systematic review is the method of comprehensively finding and appraising the studies. A meta-analysis is a statistical technique that combines the numerical results of multiple studies into a single pooled estimate. A systematic review may include a meta-analysis if the studies are similar enough to combine sensibly, but it does not have to, and a meta-analysis should ideally sit inside a systematic review rather than stand alone. In short, one is the search and appraisal, the other is the pooling of numbers.
Because it is comprehensive and transparent. Instead of resting on a single study, which might be small, flawed or a fluke, a systematic review gathers all the credible evidence on a question and appraises it as a body, using a method others could repeat. That reduces the influence of any one biased or unlucky result and of the reviewer's own preferences. This is why systematic reviews sit near the top of the evidence hierarchy. The important caveat is that a review is only as reliable as the studies inside it, so a rigorous review of weak studies is still limited.
Yes. A systematic review reduces bias but does not eliminate it. If the underlying studies are all small, poorly designed or industry-funded, the review inherits those weaknesses, a problem summed up as garbage in, garbage out. Reviews can also be undermined by publication bias, where studies with positive results are more likely to be published and therefore found, skewing the picture. Good reviews assess and report these risks. So a systematic review is strong evidence, not infallible evidence, and its conclusions should be read alongside its stated limitations.
It is a good sign, but it does not automatically settle the matter, and it is worth looking one level deeper. Check what the review actually concluded, since reviews often report modest or mixed findings that marketing rounds up to proven. Check the quality of the included studies too, because a systematic review of small, weak or industry-funded trials carries the weaknesses of those trials. A systematic review is among the stronger things a claim can rest on, provided you read its real conclusion and the quality of what it pooled rather than just the words systematic review.
Because a single study, however striking, can be small, flawed, unlucky or contradicted by other work you never saw. A systematic review gathers the whole body of credible evidence and appraises it transparently, which guards against building a conclusion on one unrepresentative result. Weighting it higher is simply following the evidence hierarchy. The caveat we keep in view is that a review only reflects the studies it contains, so we look at the quality of those studies, not just the fact that a review exists.
A few things signal quality. It should state a clear question and pre-specified inclusion criteria, describe a thorough search across multiple databases, and follow a reporting standard such as PRISMA so the process is visible. It should appraise the quality of the included studies and discuss risks such as publication bias, rather than just counting positive results. And its conclusions should match the strength of the evidence it found. A review that hides its methods or overstates weak findings is worth less than its label suggests.