starplast.claims
Claims: knowledge generated with a measured certainty, then tested by evidence that is measured to be
independent of what generated it (instructions/open/63_claims_generate_and_verify.md).
Every number here comes from held-out genes -- the track record's folds -- never from an assumption. Two of them are about leakage, because "these datasets do not overlap" is a guess: a signal peptide is location information though it shares no dataset with LOPIT.
- :func:
shared_mistakes-- generator vs verifier. When the generator is wrong, how often does the verifier make the SAME wrong call, against how often it would if the two were independent given the truth? 1.0 is independent; a random forest "confirming" kNN repeats its mistakes 3.5x chance. - :func:
family_recovery-- evidence vs label. How much of a label each evidence family recovers on its own. A family that alone recovers the label far better than the rest is either the label's own experiment restated or a strong biological signal -- worth a person's look either way.
And the two that make a claim:
- :func:
certainty_model-- a strategy's support mapped to how often calls with that support were right when the answer was hidden (isotonic, cross-fitted, with its calibration error). - :func:
verification-- what a verifier's agreement and disagreement do to that certainty, measured.
True when the verifier's mistakes are measured to be (nearly) independent of the generator's.
Each evidence family ALONE, held out by orthogroup fold: how much of the label it recovers.
A family is a dataset (datasets.provenance); columns without one form a family of their own.
Reported as accuracy and mean per-class recall with what the commonest-class guess scores, so a
family can be compared with the rest and with chance. Nothing is dropped here: a family that
recovers a label very well is flagged for a person to read, because strong biology and a restated
label look the same from the numbers alone.
A strategy's support mapped to the held-out rate of being right, for one organism and label.
Fit support -> P(right) on held-out calls, and measure its error out of fold.
The calibration error is cross-fitted: each fold's calls are scored by a curve fit on the other folds, so the error is what a new gene would see, not how well the curve memorised its own points.
What the verifier's verdict does to the generator's chance of being right, on held-out genes.
Three outcomes, each with its rate and Wilson interval: the verifier agrees, disagrees, or says
nothing. Measured on the same genes, so any shared information between the two is already in
these numbers -- and shared_mistakes says how much of it there is.
Generator certainty plus independent verdicts -> P(right), fit and checked on held-out genes.
Fit the combination on held-out genes; measure its calibration error out of fold.
Logistic regression on the generator's (cross-fitted) certainty and each verifier's verdict. It never assumes the verifiers are independent of each other or of the generator: their weights are whatever the held-out genes say they are worth, shared information included.
A generator, the verifiers measured to be independent of it, and their fitted combination.
Every generator with the verifiers measured independent of it, and what the pair achieves.
confident_right is the number of held-out genes whose verified certainty reached CONFIDENT and
whose claim was right; confident_precision is how often such claims were right. A generator with
no independent verifier is listed with testable False and is never chosen.
The generator, testable by independent evidence, whose verified claims are most often right.
Chosen among calibrated, testable generators by confident_right; an explicit generator is
honoured, with whatever verifiers are measured independent of it (possibly none).
Claims about every gene with no label: what the generator says, what each verifier says, and the verified certainty -- all labelled genes visible, as in real use.
(distance per gene, limit): mean distance to the RANGE_K nearest labelled genes in the
evidence space, and the RANGE_PERCENTILE of that distance among labelled genes themselves.
The built claims (scripts/build_claims.py), loaded once. Empty where none were built.
Which recipe each label uses, whether it is proven, and how many claims it made.
✓ when every independent check that spoke agreed, ✗ when any disagreed, else the status mark.
Every shipped claim about one gene, one row per label it is claimed for.
The gene card's claims, condensed: one line per label, each a click from its reasoning.
One claim in full: what was claimed, how certain, how it was tested, and what that is worth.
The claims worth a person's time: confident, well above the prior, and (by default) tested.