Challenge
The evidence test's first criterion does not say which measured quantity has to be large
Published · Publication and review standards
Effect on the theory
Clarifies an assumption
Effect score 42/100Substantive
Substantive. Sharpens a claim or proposes a test that could be run. Substantive at the low end. It names one criterion in section 7 and proposes an amendment that could be adopted as written, but it is held down because criterion 1 already contrasts magnitude with bin-shopped significance and criterion 5 already fixes the observable, so the gap is narrower than claimed.
Progress to an answer: 1 of 100 — a small step. A small step. Requiring the operative quantity to be declared in advance sharpens what a qualifying result would have to look like, but it removes an ambiguity in the test rather than narrowing the space of possible answers. See the running chart.
AI-assisted score · · How scores work
The target is real and correctly quoted. Criterion 1 of the evidence test in foundation section 7 asks for a factor-scale effect rather than a marginal excess in a favourable bin, and it does not say in which measured quantity the factor has to appear. The contribution illustrates the gap with a peer-reviewed ice-core study that, as reported here, measures two operationally different iron fractions in the same Greenland core over the same years and returns a seventeenfold change in one and a twenty-nine percent change in the other, with the smaller change carrying the overwhelming statistical significance and attaching to the mechanism actually under argument. The requested change is one sentence: require the operative quantity, and the mechanism that makes it operative, to be named before the data are examined. Nothing in the foundation changes until that amendment is made, and no architecture, bound or result is touched. The weakness is that the gap is narrower than the contribution allows. Criterion 1's own wording already contrasts effect magnitude against significance won in a favourable bin, and criterion 5 requires a match to a specified architecture's signature in sign, scale and threshold behaviour, which fixes the observable for any candidate the test would really be applied to. The ice-core case is an analogy from a domain where the evidence test was never in play, as the contribution concedes when it says the record does not distinguish a natural catastrophe from an intentional reset, so what it carries across is the familiar methodological point that effect size and significance are different questions and the observable must be declared in advance.
Next step
Make the amendment, then test it against cases the foundation already holds. Add to criterion 1 the requirement that the quantity in which the factor must appear be named in advance, together with the mechanism that makes it the operative one. Then re-read the anomaly ledger in section 8 and the two null bounds in section 3.6 and ask whether the amended wording would have changed the verdict on any of them. If it would not, record it as a clarification of criterion 1 rather than a change to the test, and state explicitly how the work divides between criterion 1 and criterion 5.
AI-assisted editorial assessment · · This assessment can be challenged; it is separate from the contributor’s claim.
The contribution
Explain simply
The programme's evidence test says a finding only counts if the effect is large rather than marginal. This peer-reviewed ice-core study shows that is not yet a usable rule. Measuring the same Greenland ice, over the same years, for two operationally different forms of iron gives a 17-fold change across the Younger Dryas boundary for one and a 29 percent change for the other. The small number is the one attached to the mechanism being argued about, and it is also the one with overwhelming statistical significance. So the criterion needs to name in advance which quantity the factor has to appear in.
Summary supplied by the contributor; it shares the review status of the full submission.
Test, uncertainty or challenge
Explain simply
This section describes what could support or challenge the idea, or what evidence is still missing. Check whether the proposed result would really separate different explanations.
Reading guide, not a summary of the contributor’s claim.
The challenge fails if the two iron fractions are shown not to be measuring different things in the relevant sense, for example if the labile fraction turns out to be an artefact of the continuous-flow method rather than a chemically meaningful pool. The authors themselves flag two post-depositional processes, microbial reduction and secondary mineral formation, that could bias the labile figure in opposite directions, and they argue but do not demonstrate that these are uniform across the record; a demonstration that either is depth-dependent would undercut the specific numbers without touching the general point. The general point fails if a rule already exists in the foundation naming which quantity criterion one applies to, in which case this restates it. It would be strengthened or weakened by repeating the paired measurement on a second Greenland core, since only one continuous record of this kind exists for the interval.
Sources & supporting material
Explain simply
A link helps you check where a claim came from. It does not automatically confirm the claim. Compare what the source actually says with how it is used here.
Linked responses
No published responses yet.