Strategy calibration

Generated by scripts/calibrate_strategies.py --publish on 2026-09-26 from 7,640 self-test runs (results/calibration_2026-09-26b).

Every strategy carries a self-test that hides information already known -- labels, set members, edges or values -- asks the strategy for it back, and compares the answer with the same procedure on shuffled data. Calibration runs that test over a grid of the strategy's settings, over several held-out labels and five seeds, so each number below is a mean with an interval rather than one draw.

• Skill = (observed - chance) / (1 - chance): 0 is the shuffled-data null, 1 is perfect. Negative means worse than shuffled.
• At defaults: the strategy as it opens in the application.
• Tuned: the setting with the best lower confidence bound on seeds 1-3, reported on seeds 4-5 only.
• 95% CI: two-stage bootstrap (held-out targets, then runs within a target).
• Pass: share of conclusive runs beating the null's 95th percentile by the strategy's minimum effect.

Grades: reliable (defaults beat chance with the interval above 0.05 and pass at least 60%), works when tuned (only the tuned setting does), weak (above chance on average but not reliably), no skill, untestable (fewer than five conclusive runs).

What a high skill here does and does not mean

Each strategy is scored against ITS OWN null, and a null can be too easy. The clearest case is edge prediction: strategy 18 scores its held-out pairs against non-pairs drawn at random, and a random pair of genes is usually a pair of obscure genes, so much of what such a test measures is that well-connected genes are well connected. Its skill here is therefore an upper bound (strategies 16, 33 and 34 use degree-matched non-pairs instead). docs/graphspace.md re-measures the same question against degree-matched and configuration-model nulls, where the honest figure is an AUROC near 0.67 rather than 0.97 -- and it reports the gap between the two nulls as its own quantity. Read a high number here as a reason to look at how the null was built, not as a result.

Toxoplasma gondii

01 · Hold out a category and search for a map that finds it (UMAP + HDBSCAN) -- weak

25% of the label is hidden (whole orthogroups together, so no gene is recovered through a visible paralog). The walk is built as set -- its genes per map (up to 4,000), feature sets and grids (up to three values each); the configuration, and the one cluster that best isolates each label, are both chosen using visible labels only. Metric: for each label, the F1 of its hidden genes against its chosen cluster, weighted by size -- does the structure found on known genes hold the unknown ones? Null: the same chosen clusters scored after permuting the hidden genes' labels, 100 times. (A second search on shuffled labels was the null once; it picks the largest cluster for every label, which scores F1 near 2p by size alone and made the null beat real labels.) Pass: above the null's 95th percentile by at least 0.05.

Metric: F1 of hidden genes in the cluster chosen for their label on known genes. 150 runs, 6 settings, targets: compartment, dtmclass, lopitunified, screenanyphenotype, stageenrichedderived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• min_cluster_size -- 10, 25: 0.06; 20, 50: 0.07
• n_neighbors -- 10, 30: 0.06; 15, 50: 0.06; 25, 100: 0.08
• selection -- eom, leaf: 0.07

02 · Find the map where your gene list is one cluster (UMAP + HDBSCAN) -- weak

30% of the set is hidden; the walk as set (genes per map up to 4,000, feature sets, up to three values of each grid) picks the cluster with the best F1 for the other 70%. Metric: F1 of the hidden members against that cluster's other genes -- precision is the share of the cluster's candidates that are hidden members, recall the share of hidden members among them. Null: 20 random sets of the same size through the same walk. Pass: above the null's 95th percentile by at least 0.05 (an F1 margin; random sets score about 0.04).

Metric: F1 of the hidden members against the best cluster's other genes. 100 runs, 4 settings, targets: compartment, dtmclass, lopitunified, screenanyphenotype, stageenrichedderived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• min_cluster_size -- 10, 25: 0.03; 20, 50: 0.03
• n_neighbors -- 10, 30: 0.03; 15, 50: 0.02

03 · Ask which categories the data can rediscover (UMAP + neighbour AUROC) -- reliable

30% of the label is hidden. The atlas is built from visible genes only (leave-one-out neighbour AUROC per category); hidden genes are then scored by their visible neighbours. Metric: mean hidden-gene AUROC over the categories the atlas ranks in its top half. Null: the same with the visible labels shuffled, 20 times. Pass: above the null's 95th percentile by 0.05. The rank agreement between atlas and hidden recovery is reported.

Metric: hidden-gene AUROC of the categories the atlas ranks in its top half. 75 runs, 3 settings, targets: compartment, dtmclass, lopitunified, screenanyphenotype, stageenrichedderived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• k -- 15: 0.50; 5: 0.45; 50: 0.52

04 · Keep only the modules that survive the whole walk (UMAP + HDBSCAN co-clustering) -- weak

Modules are built from a small walk on 1,500 genes without the label. 25% of the label is hidden; the module that best isolates each label is chosen on the visible genes. Metric: the size-weighted F1 of each label's hidden genes against its chosen module. Null: the same modules scored after permuting the hidden genes' labels, 100 times -- a large module scores the same either way and earns nothing. Pass: above the null's 95th percentile by 0.05.

Metric: F1 of hidden genes in the module chosen for their label on known genes. 75 runs, 3 settings, targets: compartment, dtmclass, lopitunified, screenanyphenotype, stageenrichedderived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• threshold -- 0.3: 0.07; 0.5: 0.07; 0.8: 0.05

05 · Tune a map without labels, then read what it encodes (UMAP + HDBSCAN, chi-square / Kruskal-Wallis) -- reliable

Pattern 5. One label-free map on up to 2,000 genes. Held-out features significant at q < 0.05 on a random half of the genes are the findings; each is re-tested (p < 0.05) on the other half. Metric: share that replicate. Null: the same with the second half's cluster labels permuted, 10 times. Pass: above the null's 95th percentile by 0.2, with at least three findings.

Metric: share of first-half findings that replicate on the second half. 70 runs, 14 settings, targets: (the strategy chooses its own).

Sensitivity (mean skill at each value, all other settings pooled):

• map_from -- PTM: 0.95; chemistry: 0.97; fitness: 0.94; host effect: 0.86; immunity: 0.64; localization: 0.96; metabolism: 0.84; phenotype: 0.80; protein abundance: 0.94; regulation: 0.91; relation: 0.92; sequence: 0.96; transcription: 0.95; translation: 0.97

06 · Find which kind of evidence carries a label (kNN ablation) -- reliable

25% of the label is hidden. Each kind of evidence is ranked by cross-validated accuracy on the visible labels; the top-ranked one then predicts the hidden labels. Metric: its hidden accuracy. Null: the hidden accuracy of every kind of evidence, i.e. choosing at random. Pass: above the null's 80th percentile by 0.02 (with ~15 kinds of evidence the 95th would demand the single best, which asks more than a ranking must deliver).

Metric: hidden accuracy of the evidence ranked first (transcription). 75 runs, 3 settings, targets: compartment, dtmclass, lopitunified, screenanyphenotype, stageenrichedderived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• k -- 15: 0.14; 5: 0.16; 50: 0.10

07 · Call a gene by the genes that behave like it (kNN) -- reliable

Pattern 1: 25% of the label hidden by whole orthogroups; each hidden gene called by its k nearest visible genes with the same vote threshold. Metric: hidden genes called correctly. Null: 10 runs with visible labels shuffled. Pass: above the null's 95th percentile by 0.05.

Metric: correct calls per hidden gene. 225 runs, 9 settings, targets: compartment, dtmclass, lopitunified, screenanyphenotype, stageenrichedderived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• k -- 15: 0.20; 5: 0.27; 50: 0.12
• min_share -- 0.0: 0.22; 0.3: 0.23; 0.6: 0.15

08 · Call a gene by its neighbours on the map (UMAP + kNN) -- weak

Pattern 1 on the genes the map places (up to 2,500): 25% of the label hidden by whole orthogroups; hidden genes called by their k nearest visible genes in the map. Null: 20 runs on shuffled visible labels. Pass: above the null's 95th percentile by 0.05.

Metric: correct calls per hidden gene. 225 runs, 9 settings, targets: compartment, dtmclass, lopitunified, screenanyphenotype, stageenrichedderived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• k -- 15: 0.13; 5: 0.15; 50: 0.13
• n_neighbors -- 10: 0.15; 25: 0.14; 60: 0.13

09 · Name a cluster by the label it is enriched for (UMAP + HDBSCAN, hypergeometric) -- reliable

Pattern 1 scored by precision, on the genes a blind map places (up to 2,500): 25% of the label hidden; the enrichment is recomputed from visible labels, and the calls it makes on hidden genes are scored -- the strategy abstains on noise and unenriched clusters by design, so what matters is how often a call is right. Null: 20 runs on shuffled visible labels. Pass: above the null's 95th percentile by 0.1.

Metric: precision of calls on hidden genes. 300 runs, 12 settings, targets: compartment, dtmclass, lopitunified, screenanyphenotype, stageenrichedderived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• min_cluster_size -- 10: 0.35; 25: 0.31; 60: 0.22
• min_lift -- 1.5: 0.27; 3.0: 0.35
• selection -- eom: 0.21; leaf: 0.34

10 · Find genes whose label their neighbours contradict (kNN + network neighbours) -- reliable

5% of the labels (at least ten) are swapped to a wrong class, drawn in proportion to class size. Surprise is computed with the corrupted labels. Metric: AUROC of surprise for the swapped genes among all labelled genes. Null: 20 random sets of the same size. Pass: above the null's 95th percentile by 0.1.

Metric: AUROC of surprise for the swapped labels. 75 runs, 3 settings, targets: compartment, dtmclass, lopitunified, screenanyphenotype, stageenrichedderived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• k -- 15: 0.64; 5: 0.63; 50: 0.63

11 · Diffuse a label across one measured network (random walk with restart) -- reliable

Pattern 1: 25% of the label hidden by whole orthogroups; the fields are seeded from visible genes only, so a hidden gene never seeds its own call. Metric: hidden genes called correctly (unreached genes count as misses). Null: 10 runs on shuffled labels. Pass: above the null's 95th percentile by 0.05.

Metric: correct calls per hidden gene. 900 runs, 36 settings, targets: compartment, dtmclass, lopitunified, screenanyphenotype, stageenrichedderived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• layer -- coexpression: 0.13; cofitness: 0.07; comention: 0.01; comentionft: 0.06; cotranslation: 0.02; domain: 0.04; ipms: 0.00; orthogroup: 0.00; struct: 0.08; structuralhole: 0.01; unwritteninteraction: 0.07; xlms: 0.07
• restart -- 0.2: 0.05; 0.5: 0.05; 0.8: 0.04

12 · Let every network vote, weighted by what it has earned (chance-weighted ensemble vote) -- weak

Pattern 1: 25% of the label hidden; the weights are learned on an inner holdout of the visible labels only, then hidden genes are called. Null: 5 runs with visible labels shuffled (weights relearned each time). Pass: above the null's 95th percentile by 0.05.

Metric: correct calls per hidden gene. 75 runs, 3 settings, targets: compartment, dtmclass, lopitunified, screenanyphenotype, stageenrichedderived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• k -- 15: 0.30; 5: 0.30; 50: 0.33

13 · Place a protein by the proteins it physically touches (weighted partner vote) -- reliable

Pattern 1 restricted to genes with at least one physical partner: 25% of the label hidden by whole orthogroups; hidden genes called by their visible partners' weighted vote. Null: 20 runs on shuffled labels. Pass: above the null's 95th percentile by 0.05.

Metric: correct calls per hidden gene. 25 runs, 1 settings, targets: compartment, dtmclass, lopitunified, screenanyphenotype, stageenrichedderived.

Tuned setting, per held-out target:

14 · Annotate function through shared fold (TM-score-weighted vote) -- reliable

Pattern 1 restricted to proteins with a structural neighbour: 25% of the annotation (at the chosen level) hidden by whole orthogroups; hidden proteins called by their visible structural neighbours. Null: 20 runs on shuffled annotations. Pass: above the null's 95th percentile by 0.05.

Metric: correct calls per hidden gene. 15 runs, 3 settings, targets: (the strategy chooses its own).

Sensitivity (mean skill at each value, all other settings pooled):

• level -- 1: 0.64; 2: 0.70; 3: 0.71

15 · Find the communities several networks agree on (modularity + Louvain consensus) -- weak

On genes placed in a community: 25% of the label hidden; the community that best isolates each label is chosen on the visible genes (the communities themselves never see labels). Metric: the size-weighted F1 of each label's hidden genes against its chosen community. Null: the same communities scored after permuting the hidden genes' labels, 100 times. Pass: above the null's 95th percentile by 0.05.

Metric: F1 of hidden genes in the community chosen for their label on known genes. 150 runs, 6 settings, targets: compartment, dtmclass, lopitunified, screenanyphenotype, stageenrichedderived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• agreement -- 0.3: 0.04; 0.5: 0.04
• resolution -- 0.5: 0.01; 1.0: 0.05; 2.0: 0.05

16 · Predict the contacts an interactome missed (logistic regression) -- reliable

Pattern 3: 20% of the layer's edges hidden; the model is trained on the rest against DEGREE-MATCHED non-edges -- each with a gene of similar degree at both ends -- and scores the hidden edges against fresh degree-matched non-edges. Against random non-edges this test read AUROC 0.99 on every correlation layer, because a random pair is usually two obscure genes and degree separates them. The layer's source measurements leave the similarity feature with it, and derived, annotation and literature layers are refused as targets. Metric: AUROC. Null: 5 models trained with each gene's evidence read from a random other gene (identities permuted). Pass: above the null's 95th percentile by 0.05.

Metric: AUROC of hidden pairs against degree-matched non-pairs. 10 runs, 2 settings, targets: (the strategy chooses its own).

Sensitivity (mean skill at each value, all other settings pooled):

• layer -- struct: 0.93; xlms: 0.61

17 · Read the literature for biology, not fame (publication-count residual) -- reliable

The label is never used to build the literature layer. Among co-mentioned pairs with both genes labelled, the top k by corrected residual are taken (k = the chosen number, capped at a fifth of the pool). Metric: the share of those pairs sharing a label. Null: 20 random sets of k co-mentioned pairs. Pass: above the null's 95th percentile by 0.05.

Metric: share of the top 50 corrected pairs sharing a compartment label. 75 runs, 3 settings, targets: compartment, dtmclass, lopitunified, screenanyphenotype, stageenrichedderived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• top -- 1000: 0.26; 200: 0.38; 50: 0.41

18 · List what the data says and the literature has not written (multi-layer support count) -- reliable

Pattern 3 with the literature as truth: co-mentioned pairs among genes in the measurement layers against 5 times as many random pairs. Score: number of measurement layers linking the pair. Metric: AUROC. Null: 10 runs with the measurement layers' gene identities permuted. Pass: above the null's 95th percentile by 0.02.

Metric: AUROC of hidden pairs against random non-pairs. 15 runs, 3 settings, targets: (the strategy chooses its own).

Sensitivity (mean skill at each value, all other settings pooled):

• min_layers -- 1: 0.11; 2: 0.11; 3: 0.11

19 · Train a classifier on the known genes and call the rest (logistic regression) -- reliable

Pattern 1: 25% of the label hidden by whole orthogroups; the model is trained on the visible genes and calls the hidden ones. Metric: hidden genes called correctly. Null: the analytic chance level for the same predicted and true class mixes (re-running a multinomial fit on shuffled labels ten times would take longer than it tells). Pass: above that level's 95% bound by 0.05.

Metric: correct calls per hidden gene. 100 runs, 4 settings, targets: compartment, dtmclass, lopitunified, screenanyphenotype, stageenrichedderived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• C -- 0.01: 0.32; 0.1: 0.37; 1.0: 0.38; 10.0: 0.37

20 · Learn what makes your list special, from positives alone (PU bagging, logistic regression) -- reliable

Pattern 2: 30% of the set hidden; the other 70% are the positives. Metric: AUROC of the hidden members against every other non-query gene. Null: 10 random sets of the same size. Pass: above the null's 95th percentile by 0.1.

Metric: AUROC of hidden members against every other gene. 75 runs, 3 settings, targets: compartment, dtmclass, lopitunified, screenanyphenotype, stageenrichedderived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• bags -- 15: 0.79; 40: 0.79; 5: 0.79

21 · Predict a measurement, and find the genes that defy the prediction (gradient boosting / ridge) -- reliable

Pattern 4: 20% of the measured values hidden; the model is trained on the rest. Metric: rank correlation between predicted and hidden values. Null: 3 models trained on the visible values shuffled among the measured genes. Pass: above the null's 95th percentile by 0.1.

Metric: rank correlation of predicted and hidden values. 60 runs, 4 settings, targets: exprtachy, fitinvitrohff, fitinvivo_PE.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• model -- boosted: 0.61; ridge: 0.57
• own_kind -- include: 0.64; leave out: 0.55

22 · Fill in what was never measured, and say where that is honest (soft-impute, low-rank SVD) -- reliable

Pattern 4 across the whole table: 10% of every column's measured entries hidden, the table completed at the chosen rank. Metric: median over columns of the rank correlation on hidden entries. Null: 3 completions of a table whose columns were each shuffled independently. Pass: above the null's 95th percentile by 0.1.

Metric: median per-column rank correlation on hidden entries. 15 runs, 3 settings, targets: (the strategy chooses its own).

Sensitivity (mean skill at each value, all other settings pooled):

• rank -- 20: 0.87; 5: 0.74; 60: 0.90

23 · Find what matters more in one condition, and why (residual + gradient boosting / ridge) -- weak

Pattern 4 on the shift: 20% of genes with both measurements hidden; a model trained on the rest predicts their shift. Metric: rank correlation on hidden genes. Null: 3 models trained on shuffled shifts. Pass: above the null's 95th percentile by 0.1.

Metric: rank correlation of predicted and hidden values. 20 runs, 4 settings, targets: (the strategy chooses its own).

Sensitivity (mean skill at each value, all other settings pooled):

• model -- boosted: 0.10; ridge: 0.08
• own_kind -- include: 0.11; leave out: 0.07

24 · Describe what your gene list has in common (hypergeometric + rank-sum) -- reliable

Pattern 2: 40% of the set hidden; the profile is built from the other 60% and scores every gene. Metric: AUROC of the hidden members against every other non-query gene. Null: 10 random sets of the same size. Pass: above the null's 95th percentile by 0.1.

Metric: AUROC of hidden members against every other gene. 25 runs, 1 settings, targets: compartment, dtmclass, lopitunified, screenanyphenotype, stageenrichedderived.

Tuned setting, per held-out target:

25 · Grow your gene list along the networks (random walk with restart) -- reliable

Pattern 2: 30% of the set hidden; the walk is seeded from the other 70%. Metric: AUROC of the hidden members against every other non-seed gene. Null: 10 random seed sets of the same size. Pass: above the null's 95th percentile by 0.1.

Metric: AUROC of hidden members against every other gene. 150 runs, 6 settings, targets: compartment, dtmclass, lopitunified, screenanyphenotype, stageenrichedderived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• mode -- networks: 0.44; networks + measurements: 0.59
• restart -- 0.1: 0.52; 0.3: 0.52; 0.6: 0.52

26 · Find categories that split in two on another measurement (UMAP + HDBSCAN) -- untestable

Pattern 5: significant findings (q < 0.05) on a random half of the mapped genes; each is checked on the other half -- the minority group enriched again among the cluster's A-matching genes (hypergeometric p < 0.05), or the measurement bimodal again. Metric: share replicating. Null: 20 runs with B shuffled within the second half. Pass: above the null's 95th percentile by 0.2 with at least three findings.

Metric: share of findings that replicate. 15 runs, 3 settings, targets: (the strategy chooses its own).

Sensitivity (mean skill at each value, all other settings pooled):

• min_cluster_size --

27 · Find kinds of gene defined by two labels at once (UMAP + HDBSCAN) -- reliable

Pattern 5: conjunctions found on a random half; each is checked on the other half as the second label's enrichment in the cluster among genes carrying the first label (hypergeometric p < 0.05). Metric: share replicating. Null: 20 runs with the second label shuffled within the second half. Pass: above the null's 95th percentile by 0.2 with at least three findings.

Metric: share of first-half findings that replicate on the second half. 15 runs, 3 settings, targets: (the strategy chooses its own).

Sensitivity (mean skill at each value, all other settings pooled):

• min_cluster_size -- 15: 0.49; 30: 0.66; 8: 0.48

28 · Find paralogs that changed jobs (profile correlation) -- reliable

Paralog pairs with both genes labelled; the labels are withheld from the profiles. Metric: AUROC of divergence for pairs whose labels differ against pairs whose labels match. Null: 20 random reassignments of the divergence values to pairs. Pass: above the null's 95th percentile by 0.05.

Metric: AUROC of profile divergence for paralogs with different compartment. 75 runs, 3 settings, targets: compartment, dtmclass, lopitunified, screenanyphenotype, stageenrichedderived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• min_shared -- 10: 0.16; 30: 0.16; 5: 0.16

29 · Carry what one parasite shows to the other (orthogroup mapping) -- reliable

Numeric target: 25% of the genes measured in both species hidden; the relation is learned on the rest. Metric: rank correlation of transferred and hidden values. Null: 20 runs with the ortholog values permuted among genes. Categorical target: Pattern 1 on genes with an ortholog value. Pass: above the null's 95th percentile by 0.1 (numeric) or 0.05 (categorical).

Metric: rank correlation of transferred and hidden values. 5 runs, 1 settings, targets: (the strategy chooses its own).

30 · Test inference on the genes orthology cannot reach (kNN) -- reliable

Pattern 1 restricted to the stratum: 25% of the label hidden by whole orthogroups; only hidden genes inside the stratum are scored. Null: 10 runs on shuffled labels. Pass: above the null's 95th percentile by 0.05.

Metric: correct calls per hidden gene. 100 runs, 4 settings, targets: compartment, dtmclass, lopitunified, screenanyphenotype, stageenrichedderived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• stratum -- conserved: 0.25; hypothetical protein: 0.21; lineage-specific: 0.26; understudied: 0.21

31 · Call a gene only when independent strategies agree (kNN + logistic + network vote) -- reliable

Pattern 1 scored by precision: 25% of the label hidden; each method is trained on the visible genes and the agreed calls on hidden genes are scored. Metric: share of agreed calls that are correct. Null: 5 runs with visible labels shuffled. Pass: above the null's 95th percentile by 0.1. Single-method precisions are reported.

Metric: precision of calls on hidden genes. 75 runs, 3 settings, targets: compartment, dtmclass, lopitunified, screenanyphenotype, stageenrichedderived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• min_agree -- 1: 0.28; 2: 0.35; 3: 0.60

32 · Put the understudied genes first (kNN + logistic + network vote) -- reliable

Pattern 1 scored by precision and restricted to understudied genes: 25% of the label hidden; agreed calls are scored only on hidden understudied genes. Null: 5 runs with visible labels shuffled. Pass: above the null's 95th percentile by 0.1.

Metric: precision of calls on hidden genes. 75 runs, 3 settings, targets: compartment, dtmclass, lopitunified, screenanyphenotype, stageenrichedderived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• min_agree -- 1: 0.26; 2: 0.32; 3: 0.57

33 · Put every layer into one space and read a gene's neighbourhood (logistic edge model) -- reliable

A layer's edges are hidden by orthogroup -- whole groups at a time, so no hidden edge survives through a visible paralog -- and the layer is removed from its own features. Metric: AUROC of the hidden edges against DEGREE-MATCHED non-pairs, one per hidden edge, matched on connectivity at both ends. Null: 10 configuration-model rewirings of the hidden edges, which keep their degree sequence and destroy their topology, so a model reading fame cannot beat it. Pass: above the null's 95th percentile by 0.05. The AUROC against random non-pairs and the gap between the two are reported as numbers, not as the verdict.

Metric: AUROC of hidden coexpression edges against degree-matched non-pairs. 150 runs, 30 settings, targets: (the strategy chooses its own).

Sensitivity (mean skill at each value, all other settings pooled):

• k -- 10: 0.47; 25: 0.47; 5: 0.47
• knn -- 15: 0.47; 5: 0.47
• layer -- coexpression: 0.59; cofitness: 0.14; cotranslation: 0.89; struct: 0.49; xlms: 0.26

34 · Train on the networks and rank the edges they are missing (logistic / spectral embedding) -- reliable

The chosen layer's edges are hidden by orthogroup and the layer is removed from its own features. Metric: AUROC of the hidden edges against DEGREE-MATCHED non-pairs. Null: 10 configuration-model rewirings of the hidden edges, which preserve their degree sequence, so fame alone cannot clear the bar. Pass: above the null's 95th percentile by 0.05. The random-null AUROC, the fame gap, precision@k, the Brier score and the reliability gap are all reported as numbers beside the verdict.

Metric: AUROC of hidden coexpression edges against degree-matched non-pairs. 100 runs, 20 settings, targets: (the strategy chooses its own).

Sensitivity (mean skill at each value, all other settings pooled):

• fraction -- 0.25: 0.50; 0.4: 0.48
• layer -- coexpression: 0.59; cofitness: 0.14; cotranslation: 0.89; struct: 0.50; xlms: 0.34
• model -- embedding: 0.51; logistic: 0.47

35 · Call genes with a stated error rate (split conformal prediction) -- reliable

25% of the label hidden by whole orthogroups; the model is trained and calibrated on the rest (itself split by orthogroup) and builds a set for every hidden gene. Metric: set efficiency, 1 - (mean set size - 1) / (classes - 1) -- how far the sets narrow the possibilities. Null: 10 runs on shuffled labels, where the model learns nothing and the sets must grow to keep the promise. Pass: above the null's 95th percentile by 0.05. Also reported: set coverage on the hidden genes beside the 1 - alpha promised, and the scorecard of the single-label calls.

Metric: set efficiency: 1 - (mean set size - 1) / (classes - 1). 150 runs, 6 settings, targets: compartment, dtmclass, lopitunified, screenanyphenotype, stageenrichedderived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• alpha -- 0.05: 0.23; 0.1: 0.38; 0.2: 0.55
• model -- kNN: 0.28; logistic: 0.50

36 · Smooth the measurements along the networks, then classify (graph convolution + logistic regression) -- reliable

Pattern 1: 25% of the label hidden by whole orthogroups; the smoothed features are computed from measurements only, and the model is trained on the visible genes. Metric: hidden genes called correctly. Null: the analytic chance level for the same predicted and true class mixes. Pass: above that level's 95% bound by 0.05.

Metric: correct calls per hidden gene. 150 runs, 6 settings, targets: compartment, dtmclass, lopitunified, screenanyphenotype, stageenrichedderived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• C -- 0.1: 0.39; 0.5: 0.40
• hops -- 1: 0.39; 2: 0.39; 3: 0.39

37 · Let a random forest find what defines a label (random forest + permutation importance) -- reliable

Pattern 1: 25% of the label hidden by whole orthogroups; the forest is trained on the visible genes and calls the hidden ones. Metric: hidden genes called correctly. Null: the analytic chance level for the same predicted and true class mixes. Pass: above that level's 95% bound by 0.05.

Metric: correct calls per hidden gene. 150 runs, 6 settings, targets: compartment, dtmclass, lopitunified, screenanyphenotype, stageenrichedderived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• min_leaf -- 1: 0.44; 2: 0.45; 5: 0.46
• trees -- 100: 0.45; 300: 0.45

38 · Learn how much to trust each kind of evidence (stacked logistic regression) -- reliable

Pattern 1: 25% of the label hidden by whole orthogroups; base predictions for the meta-model are out of fold within the visible genes only, so no hidden label reaches either level. Metric: hidden genes called correctly. Null: the analytic chance level for the same predicted and true class mixes. Pass: above that level's 95% bound by 0.05.

Metric: correct calls per hidden gene. 75 runs, 3 settings, targets: compartment, dtmclass, lopitunified, screenanyphenotype, stageenrichedderived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• k -- 10: 0.41; 15: 0.41; 30: 0.41

39 · Predict a value with an interval that holds (gradient boosting / ridge + split conformal) -- weak

Pattern 4: 20% of the measured values hidden; the model is trained and calibrated on the rest (split by orthogroup) and predicts the hidden ones. Metric: rank correlation of predicted and hidden values. Null: the chance distribution of a rank correlation, with refits on shuffled values reported. Pass: above its 95th percentile by 0.1. Also reported: the share of hidden values inside their intervals, beside the coverage promised.

Metric: rank correlation of predicted and hidden values. 60 runs, 4 settings, targets: exprtachy, fitinvitrohff, fitinvivo_PE.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• alpha -- 0.1: 0.54; 0.2: 0.54
• model -- boosted: 0.55; ridge: 0.53

Plasmodium falciparum

01 · Hold out a category and search for a map that finds it (UMAP + HDBSCAN) -- weak

25% of the label is hidden (whole orthogroups together, so no gene is recovered through a visible paralog). The walk is built as set -- its genes per map (up to 4,000), feature sets and grids (up to three values each); the configuration, and the one cluster that best isolates each label, are both chosen using visible labels only. Metric: for each label, the F1 of its hidden genes against its chosen cluster, weighted by size -- does the structure found on known genes hold the unknown ones? Null: the same chosen clusters scored after permuting the hidden genes' labels, 100 times. (A second search on shuffled labels was the null once; it picks the largest cluster for every label, which scores F1 near 2p by size alone and made the null beat real labels.) Pass: above the null's 95th percentile by at least 0.05.

Metric: F1 of hidden genes in the cluster chosen for their label on known genes. 120 runs, 6 settings, targets: isexported, lopitpflocation, pbtransferredphenotype, stageenriched_derived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• min_cluster_size -- 10, 25: 0.06; 20, 50: 0.11
• n_neighbors -- 10, 30: 0.09; 15, 50: 0.07; 25, 100: 0.09
• selection -- eom, leaf: 0.09

02 · Find the map where your gene list is one cluster (UMAP + HDBSCAN) -- reliable

30% of the set is hidden; the walk as set (genes per map up to 4,000, feature sets, up to three values of each grid) picks the cluster with the best F1 for the other 70%. Metric: F1 of the hidden members against that cluster's other genes -- precision is the share of the cluster's candidates that are hidden members, recall the share of hidden members among them. Null: 20 random sets of the same size through the same walk. Pass: above the null's 95th percentile by at least 0.05 (an F1 margin; random sets score about 0.04).

Metric: F1 of the hidden members against the best cluster's other genes. 80 runs, 4 settings, targets: isexported, lopitpflocation, pbtransferredphenotype, stageenriched_derived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• min_cluster_size -- 10, 25: 0.21; 20, 50: 0.21
• n_neighbors -- 10, 30: 0.21; 15, 50: 0.21

03 · Ask which categories the data can rediscover (UMAP + neighbour AUROC) -- reliable

30% of the label is hidden. The atlas is built from visible genes only (leave-one-out neighbour AUROC per category); hidden genes are then scored by their visible neighbours. Metric: mean hidden-gene AUROC over the categories the atlas ranks in its top half. Null: the same with the visible labels shuffled, 20 times. Pass: above the null's 95th percentile by 0.05. The rank agreement between atlas and hidden recovery is reported.

Metric: hidden-gene AUROC of the categories the atlas ranks in its top half. 60 runs, 3 settings, targets: isexported, lopitpflocation, pbtransferredphenotype, stageenriched_derived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• k -- 15: 0.69; 5: 0.61; 50: 0.67

04 · Keep only the modules that survive the whole walk (UMAP + HDBSCAN co-clustering) -- weak

Modules are built from a small walk on 1,500 genes without the label. 25% of the label is hidden; the module that best isolates each label is chosen on the visible genes. Metric: the size-weighted F1 of each label's hidden genes against its chosen module. Null: the same modules scored after permuting the hidden genes' labels, 100 times -- a large module scores the same either way and earns nothing. Pass: above the null's 95th percentile by 0.05.

Metric: F1 of hidden genes in the module chosen for their label on known genes. 60 runs, 3 settings, targets: isexported, lopitpflocation, pbtransferredphenotype, stageenriched_derived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• threshold -- 0.3: 0.13; 0.5: 0.15; 0.8: 0.05

05 · Tune a map without labels, then read what it encodes (UMAP + HDBSCAN, chi-square / Kruskal-Wallis) -- reliable

Pattern 5. One label-free map on up to 2,000 genes. Held-out features significant at q < 0.05 on a random half of the genes are the findings; each is re-tested (p < 0.05) on the other half. Metric: share that replicate. Null: the same with the second half's cluster labels permuted, 10 times. Pass: above the null's 95th percentile by 0.2, with at least three findings.

Metric: share of first-half findings that replicate on the second half. 250 runs, 50 settings, targets: (the strategy chooses its own).

Sensitivity (mean skill at each value, all other settings pooled):

• map_from -- antisense: 0.96; chromprox: 0.94; codon: 0.94; committed: 0.92; complex: 0.90; dnds: 0.95; engaged: 0.96; ev: 0.57; exon: 0.80; export: 0.90; expr: 0.95; febrile: 0.97; fertility: 0.81; field: 0.87; gametocyte: 0.92; has: 0.94; hsp90: 0.93; idc: 0.86; is: 0.93; isoelectric: 0.98; latency: 0.87; length: 0.93; lopit: 0.96; m6a: 0.97; mean: 0.98; melting: 0.93; molecular: 0.95; mrna: 0.92; n: 0.96; novel: 0.92; ortholog: 0.92; paralog: 0.91; pb: 0.95; pf6: 0.97; piggybac: 0.85; plddt: 0.92; polysomal: 0.97; protein: 0.97; proteome: 0.95; resistance: 0.77; riboseq: 0.95; rna: 0.96; sir2: 0.83; sir2a: 0.85; sir2b: 0.87; snp: 0.95; stage: 0.86; steady: 0.96; transcript: 0.93; transcription: 0.88

06 · Find which kind of evidence carries a label (kNN ablation) -- reliable

25% of the label is hidden. Each kind of evidence is ranked by cross-validated accuracy on the visible labels; the top-ranked one then predicts the hidden labels. Metric: its hidden accuracy. Null: the hidden accuracy of every kind of evidence, i.e. choosing at random. Pass: above the null's 80th percentile by 0.02 (with ~15 kinds of evidence the 95th would demand the single best, which asks more than a ranking must deliver).

Metric: hidden accuracy of the evidence ranked first (expr). 60 runs, 3 settings, targets: isexported, lopitpflocation, pbtransferredphenotype, stageenriched_derived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• k -- 15: 0.13; 5: 0.15; 50: 0.14

07 · Call a gene by the genes that behave like it (kNN) -- weak

Pattern 1: 25% of the label hidden by whole orthogroups; each hidden gene called by its k nearest visible genes with the same vote threshold. Metric: hidden genes called correctly. Null: 10 runs with visible labels shuffled. Pass: above the null's 95th percentile by 0.05.

Metric: correct calls per hidden gene. 180 runs, 9 settings, targets: isexported, lopitpflocation, pbtransferredphenotype, stageenriched_derived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• k -- 15: 0.05; 5: 0.14; 50: -0.02
• min_share -- 0.0: 0.08; 0.3: 0.08; 0.6: 0.00

08 · Call a gene by its neighbours on the map (UMAP + kNN) -- weak

Pattern 1 on the genes the map places (up to 2,500): 25% of the label hidden by whole orthogroups; hidden genes called by their k nearest visible genes in the map. Null: 20 runs on shuffled visible labels. Pass: above the null's 95th percentile by 0.05.

Metric: correct calls per hidden gene. 180 runs, 9 settings, targets: isexported, lopitpflocation, pbtransferredphenotype, stageenriched_derived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• k -- 15: -0.01; 5: 0.03; 50: -0.00
• n_neighbors -- 10: 0.01; 25: 0.00; 60: 0.00

09 · Name a cluster by the label it is enriched for (UMAP + HDBSCAN, hypergeometric) -- reliable

Pattern 1 scored by precision, on the genes a blind map places (up to 2,500): 25% of the label hidden; the enrichment is recomputed from visible labels, and the calls it makes on hidden genes are scored -- the strategy abstains on noise and unenriched clusters by design, so what matters is how often a call is right. Null: 20 runs on shuffled visible labels. Pass: above the null's 95th percentile by 0.1.

Metric: precision of calls on hidden genes. 240 runs, 12 settings, targets: isexported, lopitpflocation, pbtransferredphenotype, stageenriched_derived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• min_cluster_size -- 10: 0.47; 25: 0.47; 60: 0.38
• min_lift -- 1.5: 0.48; 3.0: 0.38
• selection -- eom: 0.36; leaf: 0.47

10 · Find genes whose label their neighbours contradict (kNN + network neighbours) -- reliable

5% of the labels (at least ten) are swapped to a wrong class, drawn in proportion to class size. Surprise is computed with the corrupted labels. Metric: AUROC of surprise for the swapped genes among all labelled genes. Null: 20 random sets of the same size. Pass: above the null's 95th percentile by 0.1.

Metric: AUROC of surprise for the swapped labels. 60 runs, 3 settings, targets: isexported, lopitpflocation, pbtransferredphenotype, stageenriched_derived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• k -- 15: 0.74; 5: 0.74; 50: 0.70

11 · Diffuse a label across one measured network (random walk with restart) -- reliable

Pattern 1: 25% of the label hidden by whole orthogroups; the fields are seeded from visible genes only, so a hidden gene never seeds its own call. Metric: hidden genes called correctly (unreached genes count as misses). Null: 10 runs on shuffled labels. Pass: above the null's 95th percentile by 0.05.

Metric: correct calls per hidden gene. 480 runs, 24 settings, targets: isexported, lopitpflocation, pbtransferredphenotype, stageenriched_derived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• layer -- coexpression: 0.22; comention: 0.01; cotranslation: 0.05; domain: 0.11; ip_ms: -0.00; orthogroup: 0.00; struct: 0.03; xlms: 0.01
• restart -- 0.2: 0.05; 0.5: 0.05; 0.8: 0.05

12 · Let every network vote, weighted by what it has earned (chance-weighted ensemble vote) -- reliable

Pattern 1: 25% of the label hidden; the weights are learned on an inner holdout of the visible labels only, then hidden genes are called. Null: 5 runs with visible labels shuffled (weights relearned each time). Pass: above the null's 95th percentile by 0.05.

Metric: correct calls per hidden gene. 60 runs, 3 settings, targets: isexported, lopitpflocation, pbtransferredphenotype, stageenriched_derived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• k -- 15: 0.51; 5: 0.55; 50: 0.33

13 · Place a protein by the proteins it physically touches (weighted partner vote) -- weak

Pattern 1 restricted to genes with at least one physical partner: 25% of the label hidden by whole orthogroups; hidden genes called by their visible partners' weighted vote. Null: 20 runs on shuffled labels. Pass: above the null's 95th percentile by 0.05.

Metric: correct calls per hidden gene. 20 runs, 1 settings, targets: isexported, lopitpflocation, pbtransferredphenotype, stageenriched_derived.

Tuned setting, per held-out target:

14 · Annotate function through shared fold (TM-score-weighted vote) -- reliable

Pattern 1 restricted to proteins with a structural neighbour: 25% of the annotation (at the chosen level) hidden by whole orthogroups; hidden proteins called by their visible structural neighbours. Null: 20 runs on shuffled annotations. Pass: above the null's 95th percentile by 0.05.

Metric: correct calls per hidden gene. 15 runs, 3 settings, targets: (the strategy chooses its own).

Sensitivity (mean skill at each value, all other settings pooled):

• level -- 1: 0.59; 2: 0.66; 3: 0.75

15 · Find the communities several networks agree on (modularity + Louvain consensus) -- weak

On genes placed in a community: 25% of the label hidden; the community that best isolates each label is chosen on the visible genes (the communities themselves never see labels). Metric: the size-weighted F1 of each label's hidden genes against its chosen community. Null: the same communities scored after permuting the hidden genes' labels, 100 times. Pass: above the null's 95th percentile by 0.05.

Metric: F1 of hidden genes in the community chosen for their label on known genes. 120 runs, 6 settings, targets: isexported, lopitpflocation, pbtransferredphenotype, stageenriched_derived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• agreement -- 0.3: 0.03; 0.5: 0.04
• resolution -- 0.5: 0.03; 1.0: 0.03; 2.0: 0.06

16 · Predict the contacts an interactome missed (logistic regression) -- reliable

Pattern 3: 20% of the layer's edges hidden; the model is trained on the rest against DEGREE-MATCHED non-edges -- each with a gene of similar degree at both ends -- and scores the hidden edges against fresh degree-matched non-edges. Against random non-edges this test read AUROC 0.99 on every correlation layer, because a random pair is usually two obscure genes and degree separates them. The layer's source measurements leave the similarity feature with it, and derived, annotation and literature layers are refused as targets. Metric: AUROC. Null: 5 models trained with each gene's evidence read from a random other gene (identities permuted). Pass: above the null's 95th percentile by 0.05.

Metric: AUROC of hidden pairs against degree-matched non-pairs. 5 runs, 1 settings, targets: (the strategy chooses its own).

Sensitivity (mean skill at each value, all other settings pooled):

• layer -- struct: 0.96

17 · Read the literature for biology, not fame (publication-count residual) -- weak

The label is never used to build the literature layer. Among co-mentioned pairs with both genes labelled, the top k by corrected residual are taken (k = the chosen number, capped at a fifth of the pool). Metric: the share of those pairs sharing a label. Null: 20 random sets of k co-mentioned pairs. Pass: above the null's 95th percentile by 0.05.

Metric: share of the top 48 corrected pairs sharing a lopit_pf_location label. 60 runs, 3 settings, targets: isexported, lopitpflocation, pbtransferredphenotype, stageenriched_derived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• top -- 1000: 0.26; 200: 0.26; 50: 0.13

18 · List what the data says and the literature has not written (multi-layer support count) -- reliable

Pattern 3 with the literature as truth: co-mentioned pairs among genes in the measurement layers against 5 times as many random pairs. Score: number of measurement layers linking the pair. Metric: AUROC. Null: 10 runs with the measurement layers' gene identities permuted. Pass: above the null's 95th percentile by 0.02.

Metric: AUROC of hidden pairs against random non-pairs. 15 runs, 3 settings, targets: (the strategy chooses its own).

Sensitivity (mean skill at each value, all other settings pooled):

• min_layers -- 1: 0.11; 2: 0.11; 3: 0.11

19 · Train a classifier on the known genes and call the rest (logistic regression) -- reliable

Pattern 1: 25% of the label hidden by whole orthogroups; the model is trained on the visible genes and calls the hidden ones. Metric: hidden genes called correctly. Null: the analytic chance level for the same predicted and true class mixes (re-running a multinomial fit on shuffled labels ten times would take longer than it tells). Pass: above that level's 95% bound by 0.05.

Metric: correct calls per hidden gene. 80 runs, 4 settings, targets: isexported, lopitpflocation, pbtransferredphenotype, stageenriched_derived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• C -- 0.01: 0.39; 0.1: 0.43; 1.0: 0.43; 10.0: 0.41

20 · Learn what makes your list special, from positives alone (PU bagging, logistic regression) -- reliable

Pattern 2: 30% of the set hidden; the other 70% are the positives. Metric: AUROC of the hidden members against every other non-query gene. Null: 10 random sets of the same size. Pass: above the null's 95th percentile by 0.1.

Metric: AUROC of hidden members against every other gene. 60 runs, 3 settings, targets: isexported, lopitpflocation, pbtransferredphenotype, stageenriched_derived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• bags -- 15: 0.90; 40: 0.90; 5: 0.90

21 · Predict a measurement, and find the genes that defy the prediction (gradient boosting / ridge) -- reliable

Pattern 4: 20% of the measured values hidden; the model is trained on the rest. Metric: rank correlation between predicted and hidden values. Null: 3 models trained on the visible values shuffled among the measured genes. Pass: above the null's 95th percentile by 0.1.

Metric: rank correlation of predicted and hidden values. 60 runs, 4 settings, targets: exprschizont, meanplddt, piggybac_mis.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• model -- boosted: 0.52; ridge: 0.46
• own_kind -- include: 0.49; leave out: 0.49

22 · Fill in what was never measured, and say where that is honest (soft-impute, low-rank SVD) -- reliable

Pattern 4 across the whole table: 10% of every column's measured entries hidden, the table completed at the chosen rank. Metric: median over columns of the rank correlation on hidden entries. Null: 3 completions of a table whose columns were each shuffled independently. Pass: above the null's 95th percentile by 0.1.

Metric: median per-column rank correlation on hidden entries. 15 runs, 3 settings, targets: (the strategy chooses its own).

Sensitivity (mean skill at each value, all other settings pooled):

• rank -- 20: 0.68; 5: 0.50; 60: 0.65

23 · Find what matters more in one condition, and why (residual + gradient boosting / ridge) -- reliable

Pattern 4 on the shift: 20% of genes with both measurements hidden; a model trained on the rest predicts their shift. Metric: rank correlation on hidden genes. Null: 3 models trained on shuffled shifts. Pass: above the null's 95th percentile by 0.1.

Metric: rank correlation of predicted and hidden values. 20 runs, 4 settings, targets: (the strategy chooses its own).

Sensitivity (mean skill at each value, all other settings pooled):

• model -- boosted: 0.64; ridge: 0.59
• own_kind -- include: 0.62; leave out: 0.62

24 · Describe what your gene list has in common (hypergeometric + rank-sum) -- reliable

Pattern 2: 40% of the set hidden; the profile is built from the other 60% and scores every gene. Metric: AUROC of the hidden members against every other non-query gene. Null: 10 random sets of the same size. Pass: above the null's 95th percentile by 0.1.

Metric: AUROC of hidden members against every other gene. 20 runs, 1 settings, targets: isexported, lopitpflocation, pbtransferredphenotype, stageenriched_derived.

Tuned setting, per held-out target:

25 · Grow your gene list along the networks (random walk with restart) -- reliable

Pattern 2: 30% of the set hidden; the walk is seeded from the other 70%. Metric: AUROC of the hidden members against every other non-seed gene. Null: 10 random seed sets of the same size. Pass: above the null's 95th percentile by 0.1.

Metric: AUROC of hidden members against every other gene. 120 runs, 6 settings, targets: isexported, lopitpflocation, pbtransferredphenotype, stageenriched_derived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• mode -- networks: 0.45; networks + measurements: 0.86
• restart -- 0.1: 0.65; 0.3: 0.65; 0.6: 0.66

26 · Find categories that split in two on another measurement (UMAP + HDBSCAN) -- untestable

Pattern 5: significant findings (q < 0.05) on a random half of the mapped genes; each is checked on the other half -- the minority group enriched again among the cluster's A-matching genes (hypergeometric p < 0.05), or the measurement bimodal again. Metric: share replicating. Null: 20 runs with B shuffled within the second half. Pass: above the null's 95th percentile by 0.2 with at least three findings.

Metric: share of findings that replicate. 15 runs, 3 settings, targets: (the strategy chooses its own).

Sensitivity (mean skill at each value, all other settings pooled):

• min_cluster_size --

27 · Find kinds of gene defined by two labels at once (UMAP + HDBSCAN) -- untestable

Pattern 5: conjunctions found on a random half; each is checked on the other half as the second label's enrichment in the cluster among genes carrying the first label (hypergeometric p < 0.05). Metric: share replicating. Null: 20 runs with the second label shuffled within the second half. Pass: above the null's 95th percentile by 0.2 with at least three findings.

Metric: share of findings that replicate. 15 runs, 3 settings, targets: (the strategy chooses its own).

Sensitivity (mean skill at each value, all other settings pooled):

• min_cluster_size --

28 · Find paralogs that changed jobs (profile correlation) -- weak

Paralog pairs with both genes labelled; the labels are withheld from the profiles. Metric: AUROC of divergence for pairs whose labels differ against pairs whose labels match. Null: 20 random reassignments of the divergence values to pairs. Pass: above the null's 95th percentile by 0.05.

Metric: AUROC of profile divergence for paralogs with different lopit_pf_location. 60 runs, 3 settings, targets: isexported, lopitpflocation, pbtransferredphenotype, stageenriched_derived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• min_shared -- 10: 0.11; 30: 0.11; 5: 0.11

29 · Carry what one parasite shows to the other (orthogroup mapping) -- reliable

Numeric target: 25% of the genes measured in both species hidden; the relation is learned on the rest. Metric: rank correlation of transferred and hidden values. Null: 20 runs with the ortholog values permuted among genes. Categorical target: Pattern 1 on genes with an ortholog value. Pass: above the null's 95th percentile by 0.1 (numeric) or 0.05 (categorical).

Metric: rank correlation of transferred and hidden values. 5 runs, 1 settings, targets: (the strategy chooses its own).

30 · Test inference on the genes orthology cannot reach (kNN) -- weak

Pattern 1 restricted to the stratum: 25% of the label hidden by whole orthogroups; only hidden genes inside the stratum are scored. Null: 10 runs on shuffled labels. Pass: above the null's 95th percentile by 0.05.

Metric: correct calls per hidden gene. 80 runs, 4 settings, targets: isexported, lopitpflocation, pbtransferredphenotype, stageenriched_derived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• stratum -- conserved: 0.10; lineage-specific: -0.01; understudied: 0.04

31 · Call a gene only when independent strategies agree (kNN + logistic + network vote) -- works when tuned

Pattern 1 scored by precision: 25% of the label hidden; each method is trained on the visible genes and the agreed calls on hidden genes are scored. Metric: share of agreed calls that are correct. Null: 5 runs with visible labels shuffled. Pass: above the null's 95th percentile by 0.1. Single-method precisions are reported.

Metric: precision of calls on hidden genes. 60 runs, 3 settings, targets: isexported, lopitpflocation, pbtransferredphenotype, stageenriched_derived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• min_agree -- 1: 0.13; 2: 0.25; 3: 0.70

32 · Put the understudied genes first (kNN + logistic + network vote) -- works when tuned

Pattern 1 scored by precision and restricted to understudied genes: 25% of the label hidden; agreed calls are scored only on hidden understudied genes. Null: 5 runs with visible labels shuffled. Pass: above the null's 95th percentile by 0.1.

Metric: precision of calls on hidden genes. 60 runs, 3 settings, targets: isexported, lopitpflocation, pbtransferredphenotype, stageenriched_derived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• min_agree -- 1: 0.07; 2: 0.19; 3: 0.72

33 · Put every layer into one space and read a gene's neighbourhood (logistic edge model) -- reliable

A layer's edges are hidden by orthogroup -- whole groups at a time, so no hidden edge survives through a visible paralog -- and the layer is removed from its own features. Metric: AUROC of the hidden edges against DEGREE-MATCHED non-pairs, one per hidden edge, matched on connectivity at both ends. Null: 10 configuration-model rewirings of the hidden edges, which keep their degree sequence and destroy their topology, so a model reading fame cannot beat it. Pass: above the null's 95th percentile by 0.05. The AUROC against random non-pairs and the gap between the two are reported as numbers, not as the verdict.

Metric: AUROC of hidden coexpression edges against degree-matched non-pairs. 90 runs, 18 settings, targets: (the strategy chooses its own).

Sensitivity (mean skill at each value, all other settings pooled):

• k -- 10: 0.55; 25: 0.55; 5: 0.55
• knn -- 15: 0.55; 5: 0.55
• layer -- coexpression: 0.39; cotranslation: 0.47; struct: 0.79

34 · Train on the networks and rank the edges they are missing (logistic / spectral embedding) -- reliable

The chosen layer's edges are hidden by orthogroup and the layer is removed from its own features. Metric: AUROC of the hidden edges against DEGREE-MATCHED non-pairs. Null: 10 configuration-model rewirings of the hidden edges, which preserve their degree sequence, so fame alone cannot clear the bar. Pass: above the null's 95th percentile by 0.05. The random-null AUROC, the fame gap, precision@k, the Brier score and the reliability gap are all reported as numbers beside the verdict.

Metric: AUROC of hidden coexpression edges against degree-matched non-pairs. 60 runs, 12 settings, targets: (the strategy chooses its own).

Sensitivity (mean skill at each value, all other settings pooled):

• fraction -- 0.25: 0.55; 0.4: 0.54
• layer -- coexpression: 0.38; cotranslation: 0.46; struct: 0.79
• model -- embedding: 0.54; logistic: 0.55

35 · Call genes with a stated error rate (split conformal prediction) -- reliable

25% of the label hidden by whole orthogroups; the model is trained and calibrated on the rest (itself split by orthogroup) and builds a set for every hidden gene. Metric: set efficiency, 1 - (mean set size - 1) / (classes - 1) -- how far the sets narrow the possibilities. Null: 10 runs on shuffled labels, where the model learns nothing and the sets must grow to keep the promise. Pass: above the null's 95th percentile by 0.05. Also reported: set coverage on the hidden genes beside the 1 - alpha promised, and the scorecard of the single-label calls.

Metric: set efficiency: 1 - (mean set size - 1) / (classes - 1). 120 runs, 6 settings, targets: isexported, lopitpflocation, pbtransferredphenotype, stageenriched_derived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• alpha -- 0.05: 0.46; 0.1: 0.51; 0.2: 0.68
• model -- kNN: 0.43; logistic: 0.67

36 · Smooth the measurements along the networks, then classify (graph convolution + logistic regression) -- reliable

Pattern 1: 25% of the label hidden by whole orthogroups; the smoothed features are computed from measurements only, and the model is trained on the visible genes. Metric: hidden genes called correctly. Null: the analytic chance level for the same predicted and true class mixes. Pass: above that level's 95% bound by 0.05.

Metric: correct calls per hidden gene. 120 runs, 6 settings, targets: isexported, lopitpflocation, pbtransferredphenotype, stageenriched_derived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• C -- 0.1: 0.44; 0.5: 0.46
• hops -- 1: 0.45; 2: 0.45; 3: 0.45

37 · Let a random forest find what defines a label (random forest + permutation importance) -- reliable

Pattern 1: 25% of the label hidden by whole orthogroups; the forest is trained on the visible genes and calls the hidden ones. Metric: hidden genes called correctly. Null: the analytic chance level for the same predicted and true class mixes. Pass: above that level's 95% bound by 0.05.

Metric: correct calls per hidden gene. 120 runs, 6 settings, targets: isexported, lopitpflocation, pbtransferredphenotype, stageenriched_derived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• min_leaf -- 1: 0.33; 2: 0.36; 5: 0.42
• trees -- 100: 0.37; 300: 0.37

38 · Learn how much to trust each kind of evidence (stacked logistic regression) -- reliable

Pattern 1: 25% of the label hidden by whole orthogroups; base predictions for the meta-model are out of fold within the visible genes only, so no hidden label reaches either level. Metric: hidden genes called correctly. Null: the analytic chance level for the same predicted and true class mixes. Pass: above that level's 95% bound by 0.05.

Metric: correct calls per hidden gene. 60 runs, 3 settings, targets: isexported, lopitpflocation, pbtransferredphenotype, stageenriched_derived.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• k -- 10: 0.45; 15: 0.45; 30: 0.45

39 · Predict a value with an interval that holds (gradient boosting / ridge + split conformal) -- reliable

Pattern 4: 20% of the measured values hidden; the model is trained and calibrated on the rest (split by orthogroup) and predicts the hidden ones. Metric: rank correlation of predicted and hidden values. Null: the chance distribution of a rank correlation, with refits on shuffled values reported. Pass: above its 95th percentile by 0.1. Also reported: the share of hidden values inside their intervals, beside the coverage promised.

Metric: rank correlation of predicted and hidden values. 60 runs, 4 settings, targets: exprschizont, meanplddt, piggybac_mis.

Tuned setting, per held-out target:

Sensitivity (mean skill at each value, all other settings pooled):

• alpha -- 0.1: 0.48; 0.2: 0.48
• model -- boosted: 0.50; ridge: 0.45