Edit on GitHub

starplast.umap_gallery

Pregenerated maps, and how well every label maps onto each of them.

The central map is one answer to "which measurements should place a gene": all of them, balanced by block. It is not the only reasonable answer, and a label that fails to separate on it may separate cleanly on a map built from one kind of evidence -- stage transcription, knockout fitness, protein abundance. So a gallery of maps is built ahead of time for each organism and shipped, in six groups (GROUPS, and recipes builds them in this order):

all measurements at three n_neighbors settings (10, 25, 60), because the neighbourhood size decides whether a map shows many small islands or a few continents, and which one a label prefers is itself informative. These three are the gallery's fixed reference maps: their settings are declared, not searched, so the same three maps ship whatever a structure search would have picked;
all but one family -- every measurement except localization, except transcription, except fitness, and so on -- so a label can always be scored on a map that never saw the kind of measurement that restates it;
one map per evidence family -- the slot catalogue's axis (strategies.Context.family_of): transcription, translation, protein abundance, fitness, sequence, and so on;
one map per individual experiment block -- each screen, each expression atlas, each proteomics set, the structure and sequence blocks -- wherever the block has at least MIN_BLOCK_COLUMNS columns and MIN_SET_GENES genes measured in enough of them. A block with one or two columns has no 3D map worth drawing and is covered by its family instead;
pairs and triples of families that make a biological question (COMBINATIONS): transcription + translation is expression end to end, fitness + protein abundance asks whether what a knockout costs tracks how much protein there is, and so on. Each carries the reason it is in the gallery.

Every map is built from measurements only. No categorical label column is an input (labels are what is scored), and the recipe of each map is stored beside it. Each map is clustered with HDBSCAN through clustering.cluster, as the Clusters tab does (see CLUSTERING for why "leaf" and not the Clusters tab's opening 25/25 "eom").

Maximising structure. Except for the three fixed reference maps, every candidate feature set is SEARCHED: MAP_GRID UMAP settings x GALLERY_CLUSTER_GRID clusterings, and the combination with the best label-free structure score (structure) is the one that ships. No label is consulted, so the search cannot tune a map into recovering the thing it will be scored on. The search is successive halving, as search.tune_umap does it -- every UMAP setting ranked on a SEARCH_SAMPLE sample, only the best SEARCH_KEEP rebuilt over all the recipe's genes -- because an embedding costs about a hundred times a reclustering. Every setting tried, and its score, is stored in the manifest beside the map that won.

The scores. For a label (a categorical column, e.g. compartment) and a map:

Categories map to clusters -- for each category of the label, the cluster that best recovers it is found and its F1 taken (precision: share of the cluster's labelled genes in the category; recall: share of the category's genes in that cluster). The label's score is the mean of these F1 values weighted by category size, as search.score_recovery does. One difference, stated: genes HDBSCAN leaves unclustered still count in a category's size, so a map that calls most genes noise cannot score high recall on the few it clustered. Mean precision and mean recall are reported beside it.
Best single category -- the one category that maps best, scored on the F1 of the Wilson 95% lower bounds of its precision and recall. A 2-gene category that sits alone in one cluster has precision 2/2 and recall 2/2, but lower bounds of 0.34 each, so it scores 0.34 rather than 1.0; a 200-gene category at 0.9/0.9 keeps about 0.86. Size therefore counts through evidence, not through an arbitrary weight. The category chosen is the one whose bound beats its OWN shuffled-label chance by the most: on a map with one giant cluster, the majority value of a yes/no label has a high bound by sheer size and exactly as high a bound when shuffled, and it must not win for that.
Chance -- both scores recomputed with the label's values shuffled over the same genes (five shuffles, fixed seeds) and averaged. "Skill" is (score - chance) / (1 - chance): 0 is what the map's cluster sizes give any label, 1 is perfect.
Circular -- whether the map was built from a column in the label's leakage closure (search.excluded_for): a map built from hyperLOPIT probabilities recovering compartment is expected, not discovered.

The same scoring runs live for a map built in the application (score_map).

COORDS_FILE = 'umap_gallery.npz'

MANIFEST_FILE = 'umap_gallery.json'

SCORES_FILE = 'umap_gallery_scores.tsv'

ALL_NEIGHBORS = (10, 25, 60)

LEFT_OUT_FAMILY = 'localization'

MIN_FAMILY_COLUMNS = 4

MIN_FAMILY_GENES = 400

MIN_GENE_COVERAGE = 0.5

MIN_BLOCK_COLUMNS = 3

MIN_SET_GENES = 400

COVERAGE_STEPS = (0.5, 0.3, 0.15)

GROUPS = ('All measurements', 'All but one kind', 'Evidence families', 'Single experiments', 'Pairs of families', 'Triples of families')

COMBINATIONS = ((('transcription', 'translation'), 'Expression end to end: what is transcribed together with what reaches the ribosome, so a gene regulated at translation separates from one regulated at transcription.'), (('transcription', 'protein abundance'), 'Message against protein: genes whose transcript and protein levels agree sit apart from those where one is buffered against the other.'), (('translation', 'protein abundance'), 'Ribosome occupancy against how much protein is actually there, which is where turnover and stability show up.'), (('transcription', 'fitness'), 'When a gene is expressed against what losing it costs -- the classic pairing for finding stage-specific essential genes.'), (('fitness', 'protein abundance'), 'Whether what a knockout costs tracks how much of the protein there is: abundant-and-dispensable and scarce-and-essential are both interesting neighbourhoods.'), (('fitness', 'localization'), 'Where a protein is together with what losing it costs, the pairing that separates the essential machinery of one compartment from its dispensable passengers.'), (('protein abundance', 'localization'), 'Abundance and compartment together: the axes a spatial proteomics experiment actually measures, without the transcriptional variation on top.'), (('sequence', 'localization'), "What a protein's sequence and fold say, beside where it goes -- the signal a targeting prediction lives on."), (('sequence', 'fitness'), 'Conservation, domains and disorder against what losing the gene costs: essentiality against how constrained the sequence is.'), (('sequence', 'transcription'), 'Sequence and fold beside expression, so a conserved housekeeping gene separates from a variable, stage-restricted one.'), (('regulation', 'transcription'), 'The regulatory layer -- chromatin, RNA stability, splicing -- with the transcription it produces, so a gene held down by its chromatin separates from one that is simply off.'), (('regulation', 'PTM'), 'Two layers of control that are not the message itself: chromatin and RNA on one side, post-translational modification on the other.'), (('relation', 'protein abundance'), "Who a protein is found with, beside how much of it there is: the pairing a complex should show up on, since a complex's members are usually present in proportion."), (('relation', 'localization'), 'Interaction evidence with compartment, so a complex confined to one organelle separates from a promiscuous hub.'), (('PTM', 'protein abundance'), 'Modification against abundance: whether a heavily modified protein is a scarce regulator or an abundant substrate.'), (('fitness', 'relation'), 'What losing a gene costs, beside who it works with -- the evidence a genetic-interaction argument is made from.'), (('chemistry', 'fitness'), 'Drug response and target engagement beside genetic essentiality: whether a chemically vulnerable gene is also a genetically required one.'), (('immunity', 'localization'), "Antigenicity beside compartment, because what the host's immune system sees is largely a question of where the protein ends up."), (('host effect', 'localization'), 'What a gene does to the host cell, beside where its product goes: the effector question.'), (('transcription', 'translation', 'protein abundance'), 'The whole expression cascade in one map -- transcript, ribosome, protein -- so a gene regulated at any one step separates from one regulated at all three.'), (('transcription', 'translation', 'fitness'), 'Expression at both levels against cost, the fullest picture of when a gene is needed.'), (('fitness', 'protein abundance', 'localization'), 'Cost, amount and place: the three questions a functional genomics screen is usually read for.'), (('sequence', 'relation', 'localization'), 'What a protein is, who it is with and where it is -- structure-first evidence with no expression in it at all, so an expression label scored here cannot be reading itself back.'), (('regulation', 'transcription', 'translation'), 'The whole message layer: how it is regulated, what is made and what is read.'), (('sequence', 'transcription', 'fitness'), 'Conservation, expression and cost, the three axes a candidate gene is usually triaged on.'))

MAP_GRID = {'n_neighbors': (10, 25, 60), 'min_dist': (0.0, 0.25)}

GALLERY_CLUSTER_GRID = {'min_cluster_size': (15, 25, 40), 'min_samples': (5, 10)}

SEARCH_SAMPLE = 1500

SEARCH_KEEP = 2

SILHOUETTE_SAMPLE = 3000

MIN_CLUSTER_SIZE = 25

CLUSTERING = {'min_cluster_size': 25, 'min_samples': 5, 'cluster_selection_method': 'leaf'}

MIN_CATEGORY = 2

PERMUTATIONS = 5

Z95 = 1.959963984540054

NOISE = -1

SCORE_COLUMNS = {'map': ('Map', 'The pregenerated map (or the map on screen) the label was scored on.'), 'label': ('Label', "The categorical column whose categories are compared with the map's clusters."), 'category_f1': ('Categories → clusters', 'For each category, the F1 of the cluster that best recovers it; averaged over categories weighted by their size. 1 means every category is its own cluster.'), 'category_skill': ('Skill', 'Categories → clusters, rescaled against the same score with the label shuffled over the same genes: 0 is chance, 1 is perfect.'), 'category_precision': ('Precision', "Size-weighted mean share of each best cluster's labelled genes that belong to the category."), 'category_recall': ('Recall', "Size-weighted mean share of each category's genes that fall in its best cluster. Unclustered genes count as missed."), 'best_category': ('Best category', 'The single category that maps onto one cluster furthest above chance, judged on the size-aware score.'), 'best_n': ('n', 'Genes in the best category on this map. Small categories are scored on lower bounds, so they need to map very cleanly to lead.'), 'best_f1_lower': ('Best category (size-aware)', "F1 of the Wilson 95% lower bounds of the best category's precision and recall. A 2-gene category alone in a cluster scores 0.34, not 1."), 'best_skill': ('Best skill', 'The size-aware best-category score, rescaled against shuffled labels: 0 is chance, 1 is perfect.'), 'best_precision': ('Best precision', "Raw precision of the best category's cluster."), 'best_recall': ('Best recall', "Raw recall of the best category's cluster."), 'coverage': ('Coverage', "Share of the label's labelled genes that are placed on this map. A family map covers only the genes that family measured."), 'genes_scored': ('Genes', 'Labelled genes placed on the map: the genes the scores are over.'), 'categories': ('Categories', 'Categories scored (at least 2 genes on the map).'), 'circular': ('Circular', 'Yes when the map was built from a column that restates this label (its leakage closure): recovering it is then expected, not found.')}

def wilson_lower(k, n, z: float = 1.959963984540054):

The Wilson score interval's lower bound for k successes in n trials (0 where n is 0).

def skill(score, chance):

(score - chance) / (1 - chance): 0 at chance, 1 at perfect, negative below chance.

Works elementwise on arrays; NaN where either is missing or chance is already perfect.

def per_category( truth, clusters, min_category: int = 2, permutations: int = 5, seed: int = 0) -> pandas.DataFrame:

Every category of a label against one clustering: best cluster, P, R, F1, bounded F1, skill.

truth and clusters are aligned per gene (the genes placed on one map); -1 in clusters is HDBSCAN noise. Unlabelled genes are ignored; unclustered labelled genes count as missed. Sorted by skill, the size-aware score against its own shuffled-label chance level -- the order the best single category is chosen in.

def label_scores( truth, clusters, min_category: int = 2, permutations: int = 5, seed: int = 0) -> dict:

The scores of one label on one clustered map, with the shuffled-label chance level.

Returns category_f1 / category_precision / category_recall (size-weighted over categories), the chance level and skill of category_f1, and the best single category: the one whose size-aware score (best_f1_lower, the F1 of the Wilson lower bounds) beats its OWN chance level by the most. Chosen on skill rather than on the raw bound, because on a map with one giant cluster the majority value of a yes/no label scores a high bound by sheer size, and exactly as high with the labels shuffled.

def separation(xyz, labels, sample: int = 3000, seed: int = 0) -> float:

Mean silhouette of the clustered points, rescaled to 0..1 (0.5 is no separation at all).

On the map's own coordinates, over clustered points only -- HDBSCAN noise is not a cluster and scoring it as one would punish exactly the maps that correctly refuse to place a gene. NaN when there is nothing to measure (fewer than two clusters).

def structure(xyz, labels) -> dict:

How much structure a map has, WITHOUT looking at any label. The gallery's search ranks on this.

Two things are measured, because either alone is gameable:

search.map_quality -- the project's own measure, and the one strategy 05 tunes on: the share of genes clustered times the evenness of the cluster sizes, gated by clustering.degenerate so a configuration that merely bisected the cloud scores zero however confident it looked. It is about the PARTITION and says nothing about the geometry: clusters can be even and cover everything while overlapping completely.
separation -- the mean silhouette of the clustered points, which is about the geometry and says nothing about coverage: two crisp clusters holding 5% of the proteome score beautifully.

structure is their GEOMETRIC mean, so a map has to be good at both to win, and a zero in either is a zero overall. It is not a label score: no label is read here, which is what lets the search tune a map that will afterwards be scored against labels honestly.

def block_title(block: str) -> str:

A block key as a readable phrase, with the space's own prefix removed.

The prefix comes from the registry (organisms), never from a typed species code: slot keys are f"{code}_{name}" by construction, so every registered code is tried.

def recipes(ctx) -> list:

The gallery for one organism's context: a list of dicts (id, title, blocks, group, ...).

Blocks are the display recipe's slot blocks (embedding.default_spec), grouped into families by the slot catalogue's axis. Only measurements are inputs: the slot catalogue's feature role already excludes label columns, and any categorical column that slipped through is refused.

Six groups, in GROUPS order: the three fixed "all measurements" reference maps, an "all but" map per family, a map per family, a map per individual experiment block, and the pairs and triples of COMBINATIONS. Every recipe but the three reference maps carries tune: True, which asks build_map to search MAP_GRID x GALLERY_CLUSTER_GRID for the best-structured version.

def genes_for(ctx, recipe: dict) -> numpy.ndarray:

Positions of the genes a recipe places: all of them, or those measured often enough.

def search_settings( ctx, recipe: dict, log=<built-in function print>, sample: int = 1500, keep: int = 2, seed: int = 0) -> dict:

Search MAP_GRID x GALLERY_CLUSTER_GRID for the best-structured map of one recipe.

Successive halving, as search.tune_umap does it and for the same reason: an embedding of a whole proteome costs several times one of 1,500 genes, and most settings are not worth paying the full price for twice. Every UMAP setting is ranked on the sample, the best keep are rebuilt over all the recipe's genes, and the clustering grid is searched on each finished embedding.

Ranked on structure alone -- no label is read -- and returns the winner together with every setting tried and its score, so the manifest can record what the choice was made against.

def build_map(ctx, recipe: dict, log=<built-in function print>) -> dict:

Embed and cluster one recipe. Returns rows, xyz (float32), clusters (int16) and provenance.

A recipe with tune searches MAP_GRID x GALLERY_CLUSTER_GRID and ships the best-structured combination (search_settings); one without is built exactly at its declared settings. Either way the map's structure and every setting tried are recorded in info.

def score_map( ctx, rows, clusters, labels=None, map_id: str = 'on screen', columns=None, permutations: int = 5) -> pandas.DataFrame:

Every label's scores on one clustered map (the gallery's, or one built in the app).

rows are the positions in ctx.nodes the map places, clusters one label per placed gene. columns, when known, are the map's source columns, used for the circularity flag; None leaves it unknown.

def score_gallery( ctx, maps: dict, labels=None, log=<built-in function print>) -> pandas.DataFrame:

The label x map table for one organism. maps is map id -> {rows, clusters, info}.

def write( built: dict, scores: pandas.DataFrame, gene_ids: dict, directory: str | None = None, extra: dict | None = None) -> dict:

Write the gallery: coordinates (npz), manifest (json), scores (tsv). Returns the paths.

built is organism -> map id -> build_map() result; gene_ids organism -> the node table's gene ids, stored so a loader can refuse a gallery built over a different table.

class Gallery:

The shipped gallery, loaded lazily: manifest, per-map coordinates and clusters, scores.

Gallery(directory: str | None = None)

Point at a directory holding the three gallery files (the package data by default).