starplast.methods
Ways of answering a recipe other than clustering a UMAP, judged by the same control.
Instruction 47. Until now every claim this program made came out of one procedure: embed a feature matrix, cluster it, and see whether a held-out label comes back. That is a deliberately conservative design and it stays. But it has no BASELINE -- "mean F1 0.20" cannot be called good or bad by anyone -- and it flattens away 302,756 edges that never inform a prediction.
The unification that makes the comparison honest: every method produces a PARTITION. For
UMAP + HDBSCAN the partition is the clustering. For a classifier it is the out-of-fold predicted
class. For propagation it is the label whose diffusion score is highest. Once a method has produced
one, everything downstream is unchanged -- score_recovery measures it per label, infer names the
unlabelled genes in concentrated groups, and dominant_by_cluster asks whether the independent
control corroborates them. One question, several methods, one control judging all of them.
Supervised methods must be scored OUT OF FOLD, and this is the whole difficulty. A classifier's partition is derived from the label it is being scored against, so scoring its training predictions would report memorisation as recovery -- a mistake far easier to make here than in the clustering, which never sees the label at all. Every supervised partition below is assembled from cross-validated predictions, each gene classified by a model that never saw it.
The leakage closure matters MORE here, not less. A UMAP blurs a 0.7-associated column; a penalised regression puts a coefficient on it and a boosted tree splits on it twice.
Cross-validated predictions for the labelled genes, and a full-data model for the rest.
Returns (partition, fitted, classes). partition carries the out-of-fold predicted class for
every labelled gene and the full model's prediction for every unlabelled one, as integer codes --
the same shape a clustering produces, so the scoring path does not know which method made it.
The unlabelled genes are predicted by a model trained on ALL the labelled ones, which is correct and is worth saying plainly: they are the answer, they were never in any fold, and holding a fold out of the model that names them would only make that answer worse.
L1-penalised logistic regression: the baseline this project has never had.
Chosen as the first alternative method for three reasons, and the third is the one that matters. It yields a COEFFICIENT PER COLUMN, so an answer reads "this label is predicted by these six measurements" -- something a cluster can never say. It has no cluster-size floor, so labels too rare to cluster become answerable. And it is a yardstick: if a penalised linear model recovers a label better than the tuned UMAP and its tuned clustering, then the map is a picture rather than an inference engine, and everything more elaborate should be judged against this number rather than against nothing.
L1 rather than L2 because the answer wanted is WHICH measurements, not a weight on all 361 of
them; class_weight="balanced" because these labels are wildly unequal and an unweighted fit
predicts the common class and calls it accuracy.
Which measurements the model actually used, per class, strongest first.
The reason a linear baseline earns its place. A cluster says "these genes go together"; this says "this label is predicted by these columns, in this direction", which is a sentence a biologist can disagree with -- and disagreement is what makes a claim testable.
One edge layer as a symmetric, degree-normalised sparse operator.
Symmetric normalisation (D^-1/2 W D^-1/2) rather than row-stochastic, because a random walk on the row-stochastic matrix concentrates on hubs -- and the hubs of the two co-mention layers are the genes people write about most. Attention bias is a correctness problem in this project, not a cosmetic one, and the normalisation is the first place it gets in.
How many nodes the shipped graph was built over, or None when there is no graph here.
Edge endpoints are stored as positions in the node table, which makes them meaningless against any other table. This is what lets a caller be told that rather than shown a confident answer about the wrong genes.
Random walk with restart over one edge layer, wearing a classifier's interface.
Deliberately shaped like an estimator so it can go through out_of_fold unchanged: the honesty
problem is identical to the classifier's and the solution has to be too. A gene's own seed makes
its own score enormous, so scoring a gene the walk was seeded from would report the seeding, not
the biology. Every gene is therefore scored by a walk seeded only from the OTHER folds.
X here is a column of node positions rather than features -- the walk's evidence is the graph,
not the table -- which is what lets the same cross-validation machinery drive both methods.
Answer by diffusing the label across one edge layer.
The reason this is worth having is the constraint it removes. A clustering needs fifteen labelled genes inside one cluster before it can say anything, which is why several questions in the shipped catalogue return nothing and why a 221-gene control cannot corroborate anything. Propagation has no such floor: seeded with the genes that ARE labelled, it scores all 8,140 and ranks them.
One layer, named. The thirteen layers mean thirteen different things and merging them is the mistake this project's second design decision exists to prevent -- and two of them are attention-biased, so a walk over the merged graph would carry the literature's popularity contest into every answer.
The chosen columns as numbers, with missing values LEFT MISSING.
Every other method in this project goes through build_matrix, whose four missing-value policies
all resolve absence one way or another: impute the median, drop the column, drop the gene, or add
an indicator. Each is a decision made before the model sees the data, and one of them -- dropping
genes -- changes WHICH GENES the labels describe, a bug class this project has already been
bitten by.
A histogram-boosted tree needs none of that: it learns a split for the missing branch directly, so "not measured" becomes evidence rather than a value invented to stand in for it. That is the reason this method is here and not a marginal accuracy gain, so the matrix is built raw.
Histogram gradient boosting, scored out of fold like every other supervised method here.
HistGradientBoostingClassifier rather than a new dependency: it is already in scikit-learn,
which this project already requires, and it takes NaN natively. Adding xgboost or lightgbm for
the same capability would put two gigabytes of wheels behind a method whose whole argument is
that it needs less preprocessing, not more.
Which measurements the tree relied on, by permutation rather than by split count.
Split counts are the cheap answer and a misleading one: a column with many distinct values offers more places to split and accumulates a high count without predicting anything. Permutation importance asks the question that matters -- how much worse does the model get when this column is shuffled -- and is measured on the fitted model rather than read off its structure.
Communities of ONE layer, as a partition over all nodes. Unreached nodes get -1.
Greedy modularity via networkx rather than Leiden: leidenalg and igraph are not installed here and neither is worth two more dependencies for a method whose purpose is to be COMPARED against the clustering rather than to win. Resolution is exposed because modularity has a resolution limit and a single value would silently decide how big a community is allowed to be.
Communities that several layers AGREE on, built as a consensus rather than a merged graph.
Design decision 2 of this project says the thirteen edge types are not one graph and are never merged silently. Summing adjacency matrices would do exactly that, and would let the two attention-biased layers -- which reproduce the literature's popularity contest -- pull every community toward the well-published genes.
So each layer is clustered ON ITS OWN, and the consensus is over the resulting PARTITIONS: two
genes are joined when at least min_agreement of the layers that can see them both put them
together. A merge is then a statement several independent measurements make, and the layers that
cannot see a pair abstain rather than voting no -- which matters here, because layer sizes span
four orders of magnitude and ip_ms has 64 edges against compartment's 118,712.
join decides how the agreed pairs become groups. "components" joins every chain of agreed
pairs, which is exact when agreement is rare and chains everything into one group when two
layers each agree with a third about different genes -- on a planted table with four layers
and a 0.5 threshold it put all 480 genes in one component. "louvain" instead finds
communities in the agreement graph weighted by how many layers agree, which a chain of single
agreements cannot merge.
Score fixed class identities on held-out predictions; abstentions are errors.
Returns the legacy summary/table shape for recipe consumers, with an explicit score_kind. Class labels are never remapped by looking at evaluation answers.
Attach class-level CV evidence to unknown genes without changing model calls.
CV class precision describes performance among labelled validation genes; it is not a calibrated probability for an individual, potentially shifted gene.