Key takeaways

  • Every benefit claim links to at least one primary source. Click the citation. If the link breaks, contact us.
  • We aggregate. We do not currently commission lab tests. The COAs we cite belong to manufacturers and third-party programs.
  • Three evidence levels: Strong (replicated RCTs or meta-analysis), Moderate (one or more well-designed RCTs, limited replication), Preliminary (cell, animal, or single small human trial).
  • We do not publish a benefit claim that has only preliminary evidence without labeling it as such.
  • We will not cite Wikipedia, supplement-company blogs, or press releases as the basis for a benefit claim. Ever.
  • Affiliate revenue does not move products up our rankings. The NootroScore methodology is the same for products we earn a commission on and products we do not.

What we actually do

We aggregate primary research from PubMed, systematic reviews and meta-analyses (Cochrane preferred when available), regulatory guidance (NIH ODS, EFSA, FDA), and publicly published certificates of analysis from manufacturers and third-party programs (ConsumerLab, IFOS, NSF, Eurofins, Alkemist Labs). Every benefit claim on this site links to at least one primary source you can click and read. When a claim is contested in the literature, we say so explicitly rather than picking the friendlier study. When the evidence base shifted between our last review and now, we update the page and note the change in the changelog at the bottom.

What we do not do (yet)

We do not currently commission our own lab tests. When a comparison page references lab-verified data, the data comes from someone else's lab work that is publicly available. We disclose this on every page rather than pretend otherwise. Commissioning our own lab work is a future plan, not a current capability. We also do not run our own clinical trials, do not have an in-house research team beyond the editor, and do not maintain a panel of testers. We are honest about this scope because it matters for how readers should weigh our recommendations against, say, ConsumerLab's paid lab work or the New York Times' Wirecutter testing budget.

What 'evidence level' means

On every benefit pill we publish one of three labels. Strong: multiple replicated RCTs or a meta-analysis converging on a clinically meaningful effect, ideally in the population the supplement is marketed to. Moderate: one or more well-designed RCTs but limited replication, smaller samples, mixed direction across studies, or effects observed in adjacent populations only. Preliminary: cell, animal, or single small human trial only, or evidence that is suggestive but not yet replicated. We do not publish a benefit without at least preliminary evidence. We also flag effect-size context: an effect that is statistically significant but clinically tiny is reported with that caveat in the body copy, because a 0.1 standard-deviation improvement on a memory test is not the same as a 0.5 SD improvement, even if both have a p-value under 0.05.

How we weight trial quality

Not every RCT is equal. Things we look for, in roughly this order: pre-registration of the trial protocol (a strong signal against p-hacking), declared funding source (manufacturer-funded trials get an asterisk, not a dismissal), sample size relative to the effect size being claimed (small trials with surprisingly large effects often do not replicate), blinding rigor (single-blind is weaker than double-blind), the specificity of the cognitive outcome (a generic 'cognitive function' composite is weaker than a pre-specified single outcome), and dropout rates (high dropout introduces selection bias). When two trials disagree, we look for what differs at the protocol level before we pick a side.

Source hierarchy in practice

Our citation order, in descending priority: meta-analyses and Cochrane reviews; regulatory monographs (NIH ODS, EFSA, EMA, FDA); double-blind randomized controlled trials in peer-reviewed journals; open-label or single-arm human studies; animal studies; cell or in-vitro studies. We will reach down the ladder for context (a mechanism is often only described at the cell level) but a benefit claim has to land at the RCT level or above. Concrete example: the Lion's Mane benefit pill on this site cites Mori 2009 (RCT), Saitsu 2019 (RCT), and Brandalise 2017 (mechanistic, supportive). The cell and animal mechanism papers add color but they are never the load-bearing citation.

How we read a certificate of analysis

We expect three sections to be readable on a COA we will cite: the active fraction (does the percent or milligram match the label claim within 10 percent?), the heavy metals panel (lead, cadmium, mercury, arsenic, all under USP-grade thresholds), and the microbial panel (total aerobic, no pathogens). If a COA is from the manufacturer's own lab rather than a third-party accredited lab, we note it as 'manufacturer-tested' and weigh it less. If a COA is older than 12 months, we ask the manufacturer for a current one before referencing it. We also confirm the batch number on the COA matches a batch currently for sale, because a COA on batch 22A says nothing about what is in the bottle of batch 24F.

How we verify pricing

We will not publish a price we have not verified within the last 30 days. Comparison tables list a 'last verified' date next to each price. When a retailer is the only source of pricing (Amazon, the brand site), we cite that. When prices have moved more than 15 percent since the last comparison page refresh, we re-verify and update the table the same week. Subscription discounts, first-order coupons, and Amazon Subscribe and Save savings are noted but never used as the headline price (because most readers do not subscribe, and headline-priced-on-the-coupon is a deceptive pattern even if it is technically real). Cost-per-active-mg is computed from the headline price, not the discounted price.

How we pick which supplements to cover

We cover a supplement if (a) there is at least preliminary evidence of cognitive benefit in healthy adults, (b) it is sold OTC in our primary market (US) with reasonable availability, and (c) at least three brands are competing on it (so a comparison page has substance). We deliberately do not cover supplements that are too small for a useful comparison (a single brand controls 90 percent of the market with no real alternative), and we do not cover supplements with no human evidence at all (we will not publish 'preliminary evidence' content for a compound with only animal data). Editorial calendar priorities are driven by reader search intent and our own judgment of where the evidence is shifting, not by which categories pay the highest affiliate commissions.

When and why we update existing pages

Comparison and buyer's-guide pages re-score every 90 days at minimum. Informational articles refresh on a 12-month review cycle, sooner if a major study lands or a regulatory body issues new guidance. Specific triggers we watch: a new meta-analysis on a covered ingredient, a price shift over 15 percent on a top-3 product, a new COA from a manufacturer that materially changes our potency or purity scoring, a label revision (recommended dose changed, ingredient list changed), an FDA warning letter or recall, a peer-reviewed paper retracted that we had cited. Each substantive update gets a one-line entry in the page changelog at the bottom of the page so you can audit our history.

What we will not say

We will not call a supplement a treatment for any condition. We will not claim a benefit on the basis of a press release or a manufacturer-sponsored trial alone. We will not cite Wikipedia or a supplement company blog for a benefit claim. We will not publish a price we have not verified within the last 30 days. We will not describe a product as 'doctor recommended' unless we can name the doctor and link the disclosed financial relationship. We will not give medical advice and we link to a clinician-referral note in our footer. We will not publish a supplement comparison without disclosing every affiliate relationship we have on the page.

How we handle conflicts of interest

Every comparison and buyer's guide on this site discloses the affiliate relationship at the top of the page. Affiliate revenue does not move products up our rankings. The NootroScore methodology is the same for products we earn a commission on and products we do not. When two products are within 0.3 NootroScore of each other, we surface both rather than picking the one with a higher commission. We also publish a public list of brands that have asked us to remove or revise content; the list is short, but the page exists so the policy can be checked. If a brand offers us a higher commission rate in exchange for editorial positioning, we decline and document the offer in a quarterly transparency note.

How we handle reader feedback

Email contact@nootroworld.com with the citation and the issue. If the underlying evidence supports a different conclusion than what we published, we revise. We have updated comparison pages on the basis of reader-supplied COAs and trial citations more than once. The corrections are noted in the page changelog. We do not respond to PR pitches asking us to add a brand to a comparison; brands are added on the editorial calendar based on evidence strength and category coverage, not solicitation. We do respond to substantive technical objections, even hostile ones, because being wrong publicly is the cost of doing this honestly.

FAQ

Why don't you run your own lab tests?

Cost. Commissioning HPLC and heavy-metals testing on a single product runs about USD 400 to 1200 per assay. To do this with statistical credibility for a single comparison, we would need to test multiple batches across multiple brands. That is a future plan, not a current line item. Until then, we cite the lab work others have published.

How do I know your citations are real?

Click them. Every PubMed citation links directly to the abstract on pubmed.ncbi.nlm.nih.gov. If a link is broken or a paper has been retracted, we want to know - contact@nootroworld.com.

Do you accept sponsored content?

No. We do not accept paid placements, sponsored articles, or ranking-for-fee arrangements. The only commercial relationship is affiliate links in product slots, fully disclosed at the top of every commercial page.

How often do you update existing pages?

Comparison and buyer's-guide pages re-score every 90 days at minimum. Informational articles refresh on a 12-month review cycle, sooner if a major study lands or a regulatory body issues new guidance. Every page shows its last-updated date in the byline.

What if I disagree with a score or a claim?

Email us with the citation. If the underlying evidence supports a different conclusion, we revise. We have updated comparison pages on the basis of reader-supplied COAs and trial citations more than once. The corrections are noted in the page changelog.

Do affiliate commissions change between brands?

Yes, they do. Real Mushrooms pays a different commission rate than Host Defense. We disclose the relationships individually but never let the rate influence ranking. When two products tie within 0.3 NootroScore, we surface both rather than letting the commission be the tiebreaker.

What about products you don't cover at all?

Either the evidence base is too thin for us to publish a useful comparison, the category is too small (one brand effectively controls it), or the safety profile is too uncertain for daily-use OTC recommendation. Email us if you think we are missing a category that meets our coverage criteria - several articles on this site started as reader requests.

Are your scores adjusted by reader feedback?

Indirectly. Reader-supplied evidence (a recent COA, a trial we missed, a price update) can change the underlying inputs to the score. The score itself is computed by the NootroScore methodology and is not a popularity vote. We do not adjust scores up or down based on customer ratings or reviews on retailer sites.

Key sources

Direct citations for the claims above. Click through to the original.