starplast.techniques
The techniques the strategies are built from, each explained once.
A strategy's name ends with its method in brackets; its techniques lists the pieces that method is
made of, and every piece is defined here: what it does, and why a strategy would use it. The
Strategies tab shows these beside each strategy, and :func:glossary returns them as a table, so a
reader meets "HDBSCAN" or "random walk with restart" with its explanation attached.
@dataclass(frozen=True)
class
Technique:
One technique: its name, what kind of step it is, what it does, and why a strategy uses it.
TECHNIQUES: dict =
{'umap': Technique(key='umap', name='UMAP', kind='embedding', what='Uniform Manifold Approximation and Projection: places every gene in three dimensions so that genes with similar measurements stay close, preserving local neighbourhoods over global distances. n_neighbors sets how local, min_dist how tightly points may pack.', why="Makes the combined measurements into a map whose structure can be clustered and looked at; the map is only ever built from columns the held-out label's closure permits."), 'hdbscan': Technique(key='hdbscan', name='HDBSCAN', kind='clustering', what="Hierarchical density-based clustering: finds dense groups of any shape and leaves sparse genes unclustered (noise) instead of forcing them into a group. min_cluster_size is the smallest group it reports; 'eom' keeps the most persistent clusters, 'leaf' the finest.", why='Clusters a map without being told how many clusters there are, and admits when a gene belongs nowhere.'), 'settings_walk': Technique(key='settings_walk', name='Settings walk', kind='clustering', what='Builds and clusters the map under every combination of settings on a grid (and optionally each family of measurements alone), and keeps the combination that scores best on labels visible to it.', why='No single map setting is right for every question; the walk searches them, and the self-test then scores the chosen one on labels the choice never saw.'), 'co_association': Technique(key='co_association', name='Co-association consensus', kind='clustering', what='Counts, for every pair of genes, the share of clusterings that put them together, then joins genes by average linkage on one minus that share.', why='Keeps only structure that survives the arbitrary choices a map requires.'), 'map_quality': Technique(key='map_quality', name='Label-free map quality', kind='clustering', what='Scores a clustering by how much of the proteome it places and how evenly, without looking at any label.', why='Lets a map be tuned without the labels it will later be tested against.'), 'knn': Technique(key='knn', name='k-nearest neighbours (kNN)', kind='neighbours', what='Finds the k genes most similar to a gene (Euclidean distance on standardised measurements, or on map coordinates) and lets their labels vote, each weighted by 1 / distance.', why='The plainest guilt by association: no model, no clustering, and every call can be traced to the genes that made it.'), 'knn_graph': Technique(key='knn_graph', name='kNN graph', kind='neighbours', what='Links every gene to its k nearest neighbours in measurement space, symmetrised and degree-normalised, so a table can be walked like a network.', why='Lets measurement similarity and measured networks be combined in one graph.'), 'network_vote': Technique(key='network_vote', name='Network neighbour vote', kind='networks', what="Calls a gene by the labels of its direct partners in one or more edge layers, each partner weighted by the edge's weight (contact counts, correlation strength). Abstains where a gene has no labelled partner.", why='Uses measured relationships -- crosslinks, pulldowns, co-expression -- directly as evidence.'), 'random_walk_restart': Technique(key='random_walk_restart', name='Random walk with restart', kind='networks', what='Diffuses from seed genes along the network: at each step a walker moves to a neighbour or, with the restart probability, jumps back to a seed. The stationary visiting frequency scores every gene.', why='Reaches genes several steps from the seeds while still favouring close ones; restart sets how far the influence spreads.'), 'greedy_modularity': Technique(key='greedy_modularity', name='Greedy modularity communities', kind='networks', what="Merges groups of genes greedily while the network's modularity -- edges inside groups beyond what degree alone predicts -- increases. The resolution parameter sets the community size.", why='Finds communities in each layer separately, before any layer is allowed to dominate.'), 'louvain': Technique(key='louvain', name='Louvain consensus', kind='networks', what='Joins genes that a sufficient share of the layers put together, and splits the joined graph into communities by Louvain modularity optimisation.', why='Keeps groupings several independent kinds of evidence agree on without chaining everything into one component.'), 'tm_score': Technique(key='tm_score', name='Structural similarity (TM-score)', kind='networks', what='Foldseek TM-score between predicted structures: 0 to 1, above about 0.5 usually the same fold, regardless of sequence identity.', why='Fold is conserved long after sequence is not, so it can annotate proteins no sequence search reaches.'), 'chance_weighting': Technique(key='chance_weighting', name='Chance-weighted voting', kind='networks', what='Scores each evidence source on an inner holdout of the known labels and weights its vote by its accuracy minus its own chance level; a source that only guesses the commonest class gets no vote.', why='Lets the evidence that has earned trust for this label count most, and says which that is.'), 'agreement': Technique(key='agreement', name='Agreement of independent callers', kind='networks', what='Calls a gene only where at least a set number of callers built on different evidence give the same label.', why='Trades reach for precision: independent errors rarely agree.'), 'logistic_regression': Technique(key='logistic_regression', name='Logistic regression', kind='models', what='A linear model of the log-odds of a class (or of an edge) from standardised features, with classes balanced and L2 regularisation (C: smaller is stronger).', why='Interpretable -- each feature has a weight -- and hard to overfit; the baseline any fancier model must beat.'), 'gradient_boosting': Technique(key='gradient_boosting', name='Gradient boosting', kind='models', what="An ensemble of shallow decision trees, each fitted to the errors of the ones before (scikit-learn's histogram gradient boosting). Handles missing values and non-linear effects.", why='Captures interactions and thresholds a linear model cannot, at the cost of interpretability.'), 'ridge': Technique(key='ridge', name='Ridge regression', kind='models', what='Linear regression with an L2 penalty that shrinks every coefficient towards zero.', why='A stable linear alternative to boosting, with missing values imputed first.'), 'pu_bagging': Technique(key='pu_bagging', name='Positive-unlabelled bagging', kind='models', what='Trains many classifiers, each on the positives against a small random draw of other genes, and scores every gene only by the models that did not train on it (out of bag).', why='When all you have are positives, treating every other gene as a negative teaches the model to reject the hidden positives you want to find; small random draws rarely contain them.'), 'soft_impute': Technique(key='soft_impute', name='Soft-impute (low-rank SVD)', kind='models', what='Completes a matrix with missing entries by repeatedly filling them from a truncated singular value decomposition of the current estimate, at a chosen rank.', why='Borrows strength across all measurements at once; columns are checked on hidden entries before any imputed value is trusted.'), 'residualisation': Technique(key='residualisation', name='Residualisation on a baseline', kind='models', what="Regresses the condition's measurement on its baseline (both rank-scaled) and keeps the residual: the part the baseline does not explain.", why='Separates what matters only in one condition from what matters everywhere.'), 'spectral_embedding': Technique(key='spectral_embedding', name='Spectral embedding', kind='models', what="Node vectors from a truncated SVD of a layer's adjacency matrix; a pair is described by the elementwise product of its two vectors.", why='Captures network position beyond direct evidence; kept only where it beats the interpretable baseline on a degree-matched null.'), 'triadic_closure': Technique(key='triadic_closure', name='Shared partners (triadic closure)', kind='networks', what='Counts the partners two genes share in a layer; two genes linked to the same partners are often linked themselves.', why='The classic link-prediction feature, used alongside support from other layers.'), 'degree_matched': Technique(key='degree_matched', name='Degree-matched non-pairs', kind='statistics', what="Negative pairs drawn so each end has a degree similar to the positives' ends.", why='Against random non-pairs, a model scores highly just by knowing which genes are well connected; matched non-pairs leave only what the evidence says about the pair.'), 'attention_residual': Technique(key='attention_residual', name='Attention-corrected co-mention', kind='statistics', what="Replaces each pair's co-mention count by its residual over what the two genes' own publication counts predict (log2 observed / expected).", why='Stops famous genes being linked just because they are famous.'), 'support_count': Technique(key='support_count', name='Multi-layer support count', kind='networks', what='Counts, for each gene pair, how many independent measurement layers link it.', why='Independent kinds of evidence agreeing on a pair is stronger than any one of them.'), 'profile_correlation': Technique(key='profile_correlation', name='Profile correlation', kind='statistics', what="One minus the correlation of two genes' profiles across every permitted measurement both have.", why='Measures how differently two paralogs behave, whatever they are called.'), 'orthogroup_mapping': Technique(key='orthogroup_mapping', name='Orthogroup mapping', kind='statistics', what="Aggregates a measurement or label over each orthogroup in the other species (median, or the commonest label) and learns its relation to this species' values on genes measured in both.", why='Carries evidence across species without merging the two tables.'), 'hypergeometric': Technique(key='hypergeometric', name='Hypergeometric enrichment', kind='statistics', what="The probability of drawing at least the observed number of a label's genes into a group of that size by chance (one-sided Fisher's exact test).", why='The standard test that a cluster or gene list holds more of a label than chance.'), 'rank_sum': Technique(key='rank_sum', name='Rank-sum test', kind='statistics', what="Mann-Whitney U: whether a measurement's values are higher (or lower) in one group of genes than in the rest, using ranks only.", why='Tests a measurement against a gene list without assuming a distribution.'), 'chi_square': Technique(key='chi_square', name="Chi-square test (Cramer's V)", kind='statistics', what="Tests whether a categorical label is distributed differently across clusters; Cramer's V is the effect size, 0 to 1.", why='Asks what a map built from one kind of measurement encodes about another kind.'), 'kruskal_wallis': Technique(key='kruskal_wallis', name='Kruskal-Wallis test (eta-squared)', kind='statistics', what='Rank-based test that a measurement differs between clusters; eta-squared is the effect size.', why='The numeric counterpart of the chi-square test above.'), 'bh_fdr': Technique(key='bh_fdr', name='Benjamini-Hochberg FDR', kind='statistics', what='Adjusts many p-values together so the expected share of false discoveries among the significant ones stays below the chosen rate (q).', why='Every strategy that tests many clusters, labels or measurements at once corrects for it.'), 'grouped_cv': Technique(key='grouped_cv', name='Orthogroup-grouped cross-validation', kind='statistics', what='Splits genes into folds by whole orthogroups, so a gene is never predicted by a model that saw it or its paralog.', why='Paralogs share measurements; without grouping, a model can look accurate by recognising family members.'), 'ablation': Technique(key='ablation', name='Evidence ablation', kind='statistics', what='Scores each kind of measurement alone and the combination without it, out of fold.', why='Says which experiments carry which biology and which could be dropped without loss.'), 'bimodal_split': Technique(key='bimodal_split', name='Bimodal split', kind='statistics', what="Divides a cluster's values on a second measurement into two modes and requires them to be at least a set distance apart.", why='Finds categories whose members fall into two kinds on another measurement.'), 'split_conformal': Technique(key='split_conformal', name='Split conformal prediction', kind='models', what='Calibrates any model on genes it did not train on: the errors (or the scores of the true labels) on those genes set a threshold so that the returned sets or intervals contain the truth for at least 1 - alpha of new genes.', why='Turns scores into a stated error rate that holds whatever the model, and says which genes the data cannot decide.'), 'sgc': Technique(key='sgc', name='Simplified graph convolution', kind='networks', what="Multiplies the measurement matrix by the degree-normalised network (with self-loops) once per step, so each gene gains features that average its neighbours' measurements.", why="Lets a linear model learn from a gene's network neighbourhood -- the core of a graph neural network -- while staying transparent and leak-free."), 'random_forest': Technique(key='random_forest', name='Random forest', kind='models', what='Hundreds of decision trees, each grown on a bootstrap sample of genes with a random subset of measurements at every split, voting on the class; classes are re-weighted to balance.', why='Finds thresholds and interactions a linear model cannot, and is robust to scale and outliers.'), 'permutation_importance': Technique(key='permutation_importance', name='Permutation importance', kind='statistics', what='Shuffles one measurement among held-out genes and records how much balanced accuracy drops; repeated to give a mean and a spread.', why='Measures what a model actually relies on for new genes, unlike impurity importance, which rewards measurements with many distinct values.'), 'stacking': Technique(key='stacking', name='Stacking', kind='models', what='Trains a meta-model on the out-of-fold predictions of several base models, so the combination is learned on genes no base model trained on.', why='Learns per label and per class how far to trust each kind of evidence, without rewarding a base model for memorising its training genes.')}
def
glossary(keys=None) -> pandas.DataFrame:
The techniques (all, or keys in that order) as a table: name, kind, what and why.