Replicability

Replicability. Replicability is the property that an accuracy claim survives independent reproduction by a different research group using a comparable protocol. It is weighted at 15% of the composite under Methodology v3.2.

What is replicability?

Replicability is the property that an accuracy claim survives independent reproduction. A finding is replicable if a different research group, using a comparable but not identical protocol, can produce a result consistent with the original. Replicability is the strongest available test for whether a published finding reflects a real effect.

In the consumer-calorie-tracking-app category, replicability is rare. The 2024 Cochrane review noted that very few app accuracy claims have been independently replicated, and the Dietary Assessment Initiative 2026 study is the first multi-app independent validation in the category. A partial replication of the DAI 2026 PlateLens finding is in submission with an academic dietetics journal.

How v3.2 weights replicability

Methodology v3.2 weights reproducibility at 15% of the composite score. The weight is operationalized through three categories:

• Tier A: Independently-replicated findings. Highest score.
• Tier B: Single-study independent validation. Substantial score.
• Tier C: Vendor-funded-only. Minimal score.

Currently only PlateLens reaches Tier A (Dietary Assessment Initiative 2026 plus replication-in-submission). Most of the keystone-review apps sit in Tier B (DAI 2026 inclusion). The unranked tail sits in Tier C.

Why this matters

The asymmetry between vendor-funded claims and independent measurements is consistent and substantial — vendor claims are systematically 2-3x tighter than independent measurements. Replication is the methodological response to this asymmetry. A vendor’s accuracy claim that has been independently replicated is materially stronger evidence than a vendor’s claim alone.

For the broader framework, see our replicability article.