starplast.strategy_catalog
The strategies: thirty-nine ways to use the combined data for inference, each testing itself.
Registered into strategies.REGISTRY on import. Each entry is a runner, a tester, and the prose the
Strategies tab shows -- tooltip, explanation, walkthrough and a description of its self-test. They
are grouped into nine families by MECHANISM rather than by question, because the mechanism is what
decides what a strategy can be trusted for:
1 Search the map space maps, walks and held-out labels 01-06
2 Borrow from neighbours nearest genes in the measurements 07-10
3 Walk the networks the measured edge layers 11-18
4 Learn from examples supervised models, scored out of sample 19-23
5 Start from a gene list a list in, a ranking and a profile out 24-25
6 Contrast and combine layers where two kinds of evidence part or meet 26-28
7 Cross species and strata orthologs, and the genes orthology misses 29-30
8 Combine strategies agreement, and the integrated networks 31-34
9 Advanced models stated error rates, graph convolution, 35-39
forests and stacking
33-34 live in strategy_graph and 35-39 in strategy_learning; every strategy also declares its
scorecard task and its techniques (scorecard, techniques).
Every tester follows one of the five patterns in strategies, so every verdict means the same
thing: the strategy beat the 95th percentile of the same procedure run without the information it
claims to use, by at least the stated margin.
TARGET =
Param(name='target', kind='category', label='Held-out label', tip='The label the strategy is scored against and never allowed to see: the column itself, anything that restates it and the experiment that produced it are removed first, by the same closure the Search tab uses.', default=<function default_category>, lo=0, hi=0, step=1, choices=(), optional=False, space='this')
TEST_MAX_GENES =
4000
TEST_GRID =
3
SAMPLE =
Param(name='sample', kind='int', label='Genes per map', tip='How many genes each map embeds. Zero embeds all of them; a sample of a few thousand makes a walk of many maps finish in minutes rather than an hour, at the cost of saying nothing about the genes left out.', default=3000, lo=0, hi=20000, step=500, choices=(), optional=False, space='this')
NEIGHBORS =
Param(name='n_neighbors', kind='grid', label='UMAP neighbours to try', tip='Comma-separated UMAP n_neighbors values to walk. Small values keep local detail such as complexes; large values keep the global arrangement such as organelles. Every value multiplies the number of maps built.', default='15, 50', lo=0, hi=0, step=1, choices=(), optional=False, space='this')
MIN_DIST =
Param(name='min_dist', kind='grid', label='UMAP min_dist to try', tip='Comma-separated UMAP min_dist values to walk. Zero packs similar genes tightly, which clusters well; larger values spread them out, which reads better and clusters worse. Every value multiplies the number of maps.', default='0.0, 0.25', lo=0, hi=0, step=1, choices=(), optional=False, space='this')
MIN_CLUSTER =
Param(name='min_cluster_size', kind='grid', label='Cluster sizes to try', tip='Comma-separated HDBSCAN minimum cluster sizes. Small values find small complexes and also split organelles; large values find organelles and swallow complexes. Cheap to widen: clustering is fast next to embedding.', default='20, 50', lo=0, hi=0, step=1, choices=(), optional=False, space='this')
GENES =
Param(name='genes', kind='genes', label='Gene list', tip="Gene accessions, one per line or separated by commas or spaces, pasted from anywhere. Matched exactly against this organism's table, ignoring case; anything not found is reported by name rather than silently dropped.", default='', lo=0, hi=0, step=1, choices=(), optional=False, space='this')
EXCLUDE =
Param(name='exclude', kind='category', label='Column the list came from', tip='If the gene list was made from a column of this table, name it here and that column, everything restating it and its experiment are withheld; otherwise the strategy would find the list again by reading the column that defined it.', default=None, lo=0, hi=0, step=1, choices=(), optional=True, space='this')
SELECTIONS =
Param(name='selection', kind='grid', label='Cluster selection to try', tip="HDBSCAN's cluster selection, comma-separated: 'eom' keeps the most persistent clusters -- on the real proteome two to six very large ones -- and 'leaf' every leaf of the cluster tree, dozens of small purer clusters with more genes left as noise. Walking both costs little.", default='eom, leaf', lo=0, hi=0, step=1, choices=(), optional=False, space='this')
SELECTION =
Param(name='selection', kind='choice', label='Cluster selection', tip="HDBSCAN's cluster selection for the one map this strategy reads. 'leaf' gives many small clusters, which is what naming a cluster needs; 'eom' gives a few very large ones, which on this proteome no single label dominates.", default='leaf', lo=0, hi=0, step=1, choices=('leaf', 'eom'), optional=False, space='this')
K =
Param(name='k', kind='int', label='Neighbours to consult', tip="How many labelled genes vote on each call. Few neighbours follow fine local structure and are noisy; many are stable and blur small classes into large ones. Fifteen is the project's default everywhere a neighbourhood is scored.", default=15, lo=3, hi=200, step=1, choices=(), optional=False, space='this')
MIN_SHARE =
Param(name='min_share', kind='float', label='Call when the vote is at least', tip='The share of the neighbourhood vote the winning label must hold before a gene is called. Higher calls fewer genes and fewer of them wrongly; the self-test uses the same threshold, so its number describes exactly these calls.', default=0.3, lo=0.0, hi=1.0, step=0.05, choices=(), optional=False, space='this')
MAP_NN =
Param(name='n_neighbors', kind='int', label='UMAP neighbours', tip='The UMAP n_neighbors of the map the neighbours are read from. Larger values give a map whose distances reflect broad organisation; smaller ones, local detail.', default=25, lo=2, hi=200, step=1, choices=(), optional=False, space='this')
MAP_MD =
Param(name='min_dist', kind='float', label='UMAP min_dist', tip='The UMAP min_dist of that map. Small values pack similar genes together, which is what a neighbourhood vote wants.', default=0.1, lo=0.0, hi=1.0, step=0.05, choices=(), optional=False, space='this')
LAYER =
Param(name='layer', kind='layer', label='Edge layer', tip='The measured network the label is walked across. Each layer is a different kind of evidence -- co-expression, co-fitness, crosslinks, structural similarity -- and a layer built from the held-out label itself is refused.', default=<function _layer_default.<locals>.pick>, lo=0, hi=0, step=1, choices=(), optional=False, space='this')
LINK_CANDIDATES =
300000
LINK_MIN_EDGES =
50
LINK_ANNOTATION_LAYERS =
('orthogroup', 'domain', 'compartment')
LINK_CORRELATION_LAYERS =
('coexpression', 'cofitness', 'cotranslation')
UNWRITTEN_SKIP =
('orthogroup', 'domain', 'compartment')
NUMBER =
Param(name='target', kind='number', label='Measurement to predict', tip="The numeric measurement to predict from everything else. Its own closure -- the same experiment's other outputs and anything restating it -- is withheld first.", default=<function default_numeric>, lo=0, hi=0, step=1, choices=(), optional=False, space='this')
OWN_KIND =
Param(name='own_kind', kind='choice', label='Its own kind of measurement', tip="'leave out' removes every column measured the same way as the target -- all the fitness screens, when the target is a fitness screen -- so the answer says what OTHER evidence knows. 'include' keeps them, which mostly measures how well the screens agree with each other.", default='leave out', lo=0, hi=0, step=1, choices=('leave out', 'include'), optional=False, space='this')
MODEL =
Param(name='model', kind='choice', label='Model', tip="'boosted' is gradient-boosted trees: nonlinear, tolerant of missing values, slower. 'ridge' is a penalised straight-line fit: fast, and its weights can be read.", default='boosted', lo=0, hi=0, step=1, choices=('boosted', 'ridge'), optional=False, space='this')
def
split_replicates(finding: dict, labels, a_vals, b_vals, half, numeric_b: bool) -> bool:
Does one split finding hold among the genes of half?
For a categorical B, the finding's minority group must be enriched again among the cluster's
A-matching genes of this half (hypergeometric p < 0.05); for a measurement, those genes must
again fall into two modes at least discovery.MIN_SPLIT apart.
STRATA =
('lineage-specific', 'hypothetical protein', 'understudied', 'conserved')
def
stratum_mask(ctx, name: str) -> numpy.ndarray:
Which genes belong to a named stratum of the proteome. Unknown membership is False.
CONSERVATION =
('ortho', 'lineage', 'paralog', 'conserv', 'has_pf', 'has_cp', 'has_tg')