starplast.graphspace
One integrated neighbour space over the genes, and the nulls that decide whether it means anything.
The thirteen measured edge layers each answer a different question and are deliberately never merged
in this project (build_graph). That is right for reading a layer, and wrong for one question a user
keeps asking: which genes are near this one, and on what evidence? Answering it needs every
permitted layer and the measurement table in one graph, with a single number per pair and the
per-source breakdown that produced it. This module builds that graph.
It is a CLAIM, not a measurement. Nothing here writes to ctx.graph, replaces a measured layer or
adds an edge to one: a :class:GraphSpace is a separate object with its own pairs, and every table
it produces says which kind each pair is -- a kind of measured or inferred, and a measured_in
naming the layers that record the pair, empty for exactly the inferred ones. An integrated edge is a
hypothesis with a probability attached; a crosslink is an experiment. Anything that loses that
distinction turns a model's output into a dataset, which is the one failure this project is
organised to prevent.
The evaluation was written before the model, because the easy number here is worthless. Strategy
16 (link_prediction) reports AUROC 0.97 for held-out crosslinks against random non-pairs, and most
of that is the discovery that well-connected genes are well-connected: a random pair of genes is
usually a pair of obscure genes, and any feature correlated with degree separates it from an edge
between two hubs. So every number here is reported against three negative sets at once:
random non-pairs drawn uniformly from every gene in the table;
degree_matched each positive paired with a non-pair of similar evidence degree at BOTH ends;
configuration non-pairs drawn from a configuration model -- the same degree sequence,
stub-paired at random -- so the negatives are degree-plausible by design.
The gap between the first and the second is reported as its own quantity, fame_gap: it is how much
of the performance is fame rather than biology. The shipped model's gap is SMALL -- +0.008 AUROC on
the Toxoplasma table -- and that is the result, not a disappointment: this model is given no degree
and is trained against degree-matched non-pairs, so there was little fame in it to remove. The same
features plus degree, trained the usual way against random non-pairs, have a gap of +0.024 here and
of +0.216 on a preferential-attachment graph (tests/test_graphspace.py), which is what the number
looks like when fame IS the model. That diagnostic is fitted on every run, beside the real one, and
never ranks anything. docs/graphspace.md carries the full measured table.
Edges are held out by NODE GROUP (Context.groups, which is the orthogroup), never by random edge.
A randomly held-out edge can be recovered through a paralog that stayed visible, which scores the
resemblance of a gene to its own copy rather than the inference being tested.
Self-exclusion is the other rule that cannot be relaxed: when the evidence is scored on layer l,
l is not one of its features. It is enforced in one place -- :meth:Evidence.features with
skip=l, which drops l's two columns and rebuilds the shared-partner count on the union of the
other layers -- and the integrated weight of a source is the mean of what it was worth in every
fold, counting as zero the fold in which it was the target. A source that can only predict itself
therefore ends at weight zero as arithmetic rather than as good intentions.
The leakage guard is the project's own: Context.banned for columns and Context.banned_layers for
layers, reached through Context.measurement_layers and Context.features, so holding out
compartment removes the compartment columns from the measurement similarity AND the compartment
layer from the sources, with no second copy of the closure living here.
Typical use::
from starplast import graphspace, strategies
ctx = strategies.Context.shipped("Tg")
sp = graphspace.space(ctx)
sp.neighbours("TGME49_233460", 10) # who is near this gene, and on what evidence
sp.evidence("TGME49_233460", "TGME49_269010")
sp.gaps(50) # what the evidence implies and no layer records
print(sp.report.summary()) # the three nulls, side by side
Every permitted source of evidence about a pair of genes, as one matrix with named columns.
One object per (context, held-out label), because every fold of the evaluation, every candidate pair and every question asked of a finished space must read the same features from the same closure. Two implementations of "the features of a pair" is how a reported AUROC stops describing the model that is shipped.
The sources are: for each permitted layer, the pair's weight as a percentile among that layer's own edge weights, and an indicator that the layer does not record the pair at all; the cosine similarity of the two genes' permitted measurements; the Adamic-Adar shared-partner count on the union of the permitted layers; and, where the graph carries no such layer and the closure permits the column, a shared-orthogroup or shared-Pfam indicator.
Two choices are worth the words. Weights are replaced by percentiles because a crosslink count of
5 and a co-expression correlation of 0.95 are not on one scale, and a single weight per source is
meaningless until they are. Similarity is a cosine on Context.matrix, whose columns are already
rank-scaled to [-0.5, 0.5] with missing values at the median, so it is a rank correlation across
the measured columns in all but name -- and it is read through Context.features, which applies
the closure, rather than from ctx.nodes, which would not.
Degree is deliberately NOT a source. It is the strongest predictor of a measured edge and the worst one: a model given degree learns which genes are famous. The degree-matched null exists to measure that contamination, and putting degree in the features would be paying for the null and then defeating it.
One layer as a symmetric sparse matrix of percentile weights in (0, 1].
The permitted layers as one binary graph, without skip: what a shared partner means.
Each gene's degree in the union without skip: the fame the hard null removes.
Which permitted layers record this pair. Empty means the pair is inferred, not measured.
The same for many pairs at once, as one comma-joined string per pair.
Nine scalar lookups per row turns a 276,000-row table into minutes; nine vectorised lookups over the whole column turns it into a second. The single-pair version above stays for the one pair a user asks about.
(matrix, row norms, dropped columns): the measurements a similarity may use with skip out.
A layer computed FROM measurements is those measurements' correlation structure, drawn as
edges: co-translation is the top correlations among the ribosome-footprint columns, and
co-expression the same among one transcript series. Hiding such a layer's edges while keeping
its source columns in the cosine hands the model the layer back by a second route. The
calibration sweep found exactly that -- held-out co-translation "recovered" at AUROC 0.94
against degree-matched non-pairs -- so the columns a layer was built from
(search.LAYER_SOURCES), and every other column of the experiments they came from, leave
the similarity along with the layer's edges.
One row of sources per pair, computed in chunks. skip is the self-exclusion rule.
With skip=l the returned matrix has no column for l, its shared-partner count is rebuilt
on the union of the other layers -- a shared partner reached through l is l's own
evidence arriving by a second route -- and the measurement similarity is computed without
the columns l was built from (measurement), which is the same evidence arriving by a
third. All three together are the rule.
Full features with skip's columns set to "this layer does not record the pair".
Used to score a held-out edge on the INTEGRATED weights, which are defined over every source:
dropping the columns would change the length of the vector, so instead the pair is presented
as one its own target layer never saw. The shared-partner count is still rebuilt without
skip, so the substance of the self-exclusion is unchanged; what the masking costs is a
constant per fold (the weight of one absence indicator), which the calibration absorbs.
(pairs, note): every gene's neighbours in every permitted layer, plus its nearest genes.
What the cap costs, said plainly: a pair that is in no layer and in neither gene's knn nearest
neighbours in measurement space is never scored, and so can never appear as a gap. The space is
therefore not "every pair ranked" but "every pair with a reason to be looked at, ranked" -- on
the shipped Toxoplasma table 276,384 of the 33 million pairs, which is the difference between
half a minute and several hours. per_layer bounds the contribution of a dense layer (the
compartment layer alone would otherwise supply 119,000 pairs, all of them the same claim).
Non-pairs drawn uniformly from nodes: the easy null, and the one that flatters a model.
Kept because the gap between it and the degree-matched one is the quantity worth reporting. The evaluation passes every gene here, including the ones no layer touches, since that is the null a published AUROC is usually against.
One non-pair per positive, with a gene of similar degree at BOTH ends.
The point of the whole module. Genes are put into quantile bins of log degree in the evidence union; for a positive (a, b) a partner is drawn from a's bin and one from b's bin until the draw is not a real edge. What survives this null is what the model knows beyond which genes are well connected -- and on real data that is a much smaller number than the usual one.
Non-pairs from a configuration model on the positives: the same degrees, rewired at random.
Every endpoint of every positive becomes a stub; the stubs are shuffled and paired. The result has (up to the self-loops and duplicates it loses) the positives' own degree sequence and none of their topology, so a model that scores these as highly as the real edges is reading the degree sequence. Degree-preserving rewiring and stub pairing are the same null; stub pairing is the one that is a single shuffle rather than a chain of swaps whose mixing has to be argued for.
(mean absolute gap, table): are the predicted probabilities right, decile by decile?
A ranking can be perfect and the probabilities still wrong, and a probability is what a gap table invites a reader to act on. The gap is the mean over bins of |predicted - observed|.
AUROC, precision@k, the Brier score and the reliability gap for one scored pair set.
k defaults to the number of positives, capped at 100: precision@k with k the length of the list
is just the class balance, which is a property of the sampling and not of the model.
One held-out layer: what was hidden, what each model scored, and what each source was worth.
Every model against every null, per held-out layer and pooled: the honest comparison table.
fame_gap is the quantity the module exists to report: the baseline's AUROC against random
non-pairs minus its AUROC against degree-matched ones. It is not an error term. It is the share
of an impressive-looking link-prediction number that is the observation that hubs are hubs.
One pooled number: the mean over scored folds, weighted by held-out edges.
Any of the metrics, pooled the same way.
What the learned embedding adds on the degree-matched null. Negative means it costs.
The embedding's gain on the degree-matched null, one number per held-out layer.
The pooled gain hides the thing worth knowing. On the shipped table the embedding gains +0.022 on crosslinks and +0.023 on structure, +0.002 on co-expression, and LOSES 0.005 on co-translation: the profile of a model that finds something in the sparse layers and nothing in the dense ones. A user whose table is mostly crosslinks should test it themselves.
The same gap for the diagnostic fame model: the usual recipe's own distance from honesty.
This is the number to quote when someone shows an AUROC of 0.97 for link prediction. It is the same data and the same held-out edges; all that changes is that the model was given degree and trained against random non-pairs, which is how such figures are made.
Whether the embedding may be offered for ranking at all.
Two conditions, both with a reason. The gain on the DEGREE-MATCHED null must exceed 0.01 AUROC, because a smaller gain is not worth a model that cannot say why it ranked a pair where it did -- the attribution a gap table needs is exactly what an embedding does not have. And it must exceed twice its own standard error across the held-out layers, because a mean gain that is inside its own spread is a coin landing heads. Measured on the shipped table: +0.0075 pooled, per-layer gains of +0.002, +0.010, -0.005, +0.023, +0.022, a standard error of 0.005 and so a bar of 0.011 -- it fails both conditions and the baseline ships. A tie is a loss: the interpretable model keeps the place by default.
One row per model and null: the table docs/graphspace.md prints.
The same numbers per held-out layer, because one layer can carry a pooled average.
A boolean per gene: whether its NODE GROUP is held out. Paralogs are held out together.
Hold out each layer in turn by node group, and score every model against all three nulls.
One fold per layer worth holding out. In a fold the layer is removed from the features entirely
(Evidence.features(skip=layer)), its edges between held-out groups become the positives, its
edges between visible groups become the training positives, and the negatives are drawn three
ways. Edges that straddle the split are used for neither: they have one endpoint the model has
seen, which is neither a fair test nor a wasted edge worth arguing about.
One integrated graph over genes: a probability per pair, and the evidence that produced it.
Not a layer. pairs are candidate pairs, probability is a claim about each, and measured says
whether any permitted layer records it -- the distinction every table here keeps, because an
inferred edge read as a measurement is a fabricated experiment. The measured layers in
ctx.graph are untouched by everything in this class.
The probability answers one question: given the permitted evidence, is this pair a real
relationship of the kind the measured layers record, rather than a degree-matched non-pair? It is
calibrated on held-out edges against degree-matched negatives at one negative per positive, so it
is conditional on that comparison and not an absolute posterior over all 33 million pairs. The
honest reading is a relative one, and report.metric("reliability_gap") says how far even that
is from perfect.
A gene id or a position, as a position. A name this table does not have is an error.
The row holding this pair, or -1 when the pair was never a candidate.
The probability for one pair, computed on demand when it was never a candidate.
A pair outside the candidate set is not refused: the same weights are applied to its own
features. What the candidate set decides is which pairs the space RANKS, not which it can
score, so neighbours and gaps see the candidates and this sees anything.
Which sources spoke for this pair and how much each moved the answer.
contribution is on the calibrated logit scale and sums, with intercept, to the logit of
the probability -- so a reader can see that a pair is high because three layers record it, or
because nothing records it and the measurements are nearly identical. measured marks the
sources that are a layer recording the pair, as against a similarity or a shared partner.
The genes nearest gene in the integrated space, strongest first.
measured_in names the layers that record the pair and is empty for an inferred neighbour,
and top_evidence names the three sources that moved the probability most. Only candidate
pairs are ranked, so a neighbour that shares no layer and is outside either gene's nearest
neighbours in measurement space will not appear -- see :func:candidates.
Every candidate pair as a table, strongest first. measured filters to one kind.
The pairs the evidence implies and NO permitted layer records: the gaps, ranked.
A gap's probability rests only on the sources that are not a layer edge -- measurement
similarity, shared partners, a shared orthogroup or domain -- since every layer weight is
zero for a pair no layer records. That is the right way round: a gap is an inference from
everything except a direct observation, and the table says so in top_evidence.
Build the integrated neighbour space, evaluating it on the way: space(ctx) and read it.
The order matters and is not an implementation detail: the candidate pairs are collected, the standardisation is fixed, the evaluation runs (one fold per layer, each blind to its own layer), and only then are the folds' coefficients averaged into the weights that score every candidate. A source's weight is the mean of what it was worth in each fold, counting as zero the fold where it was the target -- so the shipped weights are a held-out quantity, not a fit to everything.
exclude is a held-out label: its closure removes columns (Context.banned) and layers
(Context.banned_layers) before anything is computed, which is what makes a space safe to use
while scoring that label.