How We Audited Our Own Citations (and What We Got Wrong)
Last reviewed: October 27, 2026
The short answer
In September 2026 we checked every study citation on this site against the paper it points to. Of 37 distinct PubMed identifiers, 13 pointed at unrelated papers. Of 41 claims we compared with their cited abstracts, 12 misstated what the paper found. All were corrected and are in the corrections log.
A supplement site that grades other people’s documents has to be held to the same standard, and because the editors are not named, the only thing that can earn trust is evidence of the checking. Here is what the checking found, and what it could not.
Check 1: does the identifier match the paper?
Every PubMed link in the articles was resolved and compared with the author and year printed beside it. 13 of 37 distinct identifiers belonged to unrelated papers: 24 citations across 11 articles. One safety review was cited under the identifier of a head-and-neck oncology case report in 24 articles. The failure is silent: the citation text is plausible, the link works, and nothing looks broken. Only the destination is wrong.
Check 2: does the sentence match the paper?
A correct identifier does not make a correct claim. We pulled every sentence that cites a study (41 claim-and-citation pairs across 14 articles), read ten at random against their abstracts (six accurate, four wrong), then checked the remaining 31. The final count was 12 of 41 wrong, or 29%. Typical cases:
| What we had published | What the source says | Fix |
|---|---|---|
| A rat study was described as a human trial of chronic fatigue at 200 mg twice daily (six articles). | The study gave shilajit to rats in a forced-swim test at 25 to 100 mg/kg for 21 days. | Rewritten as animal evidence and moved out of the human-data sections. |
| A 2010 fertility study was called a randomised, double-blind, placebo-controlled trial of 96 men at 200 mg twice daily. | It was open-label with no control group; 60 men were assessed, 35 enrolled and 28 completed, at 100 mg twice daily. | Rewritten with the real design and numbers. |
| A collagen marker (serum hydroxyproline) was reported as raised in three articles. | In the study it was lower in the shilajit group (1.5 vs 2.4 µg/mL). | Direction corrected. |
| Seaweed iodine was said to exceed the tolerable upper limit more than 100 times in a single serving. | The paper reports an average of 3.86 mg/kg dry weight and says 4 g a day poses no risk; roughly 286 g a day would be needed to pass the limit. | Warning removed; the thyroid paragraph that depended on it was rescaled. |
| Three pages said a clinical study showed better haemoglobin in women with iron-deficiency anaemia after 12 weeks. | We could not find any such study in PubMed or Europe PMC, only animal studies. | Claim removed; the pages now say no human trial has tested this. |
We introduced errors while fixing errors
Three of the wrong claims were created the same day by fixing a bad identifier without opening the paper behind it. Swapping in the right identifier made the citation look correct while the sentence beside it still described something else. Our rule now is that when a citation changes, someone reads the abstract, not just the identifier.
What the checks cannot catch
- A script can confirm an identifier matches an author and a year. It cannot tell whether a sentence matches the finding. That takes reading, and we have done it for the claims above, not for every sentence on the site.
- Abstracts omit detail. Some figures we had published (one trial’s sample size and percentage changes) were not in the abstract and the full text was not available to us, so we removed them instead of keeping numbers we could not verify.
- Reading an abstract is not the same as appraising a study. The research hub records each paper’s design and caveats, but it is a summary, not a systematic review.
What we do now
- Articles cite through a registry of papers, each entry written from the paper’s own abstract. A citation to a paper that is not in the registry fails the build, and bare PubMed links are not used.
- A script compares every registry entry’s first author and year with PubMed and flags any hand-pasted link. It is run before an article is published.
- Claims an abstract does not support are marked unverified on the research hub, or removed.
- We say plainly when no human trial exists, and we keep evidence ratings per topic rather than per claim.
- Every error found goes in the log, with the open items, so a clean-looking page is never the only signal.
The same approach applies to product grades: each is built from a document you can open, scored by a fixed formula, and published with the grade’s inputs in the downloadable data. If you spot something we have not caught, email hello@shilajitdb.com. Our full approach is in the editorial policy.