Edit on GitHub

starplast.runs

Clusterings, kept: named, dated, saved with their recipe, and knowing which genes they cover.

A clustering used to be computed, drawn, and lost. The second run replaced the first with no way back, which makes the one comparison this application exists for -- does this structure hold under different settings -- impossible to make by looking. Runs are kept here instead.

Three things a kept run has to carry, and each of them was missing:

  • A name. Auto-generated from the timestamp to the second, so two runs a minute apart are distinguishable without anyone typing anything, and renameable afterwards because "hdbscan_60 on expression" is what a person will actually look for.
  • Its recipe. An embedding spec and the clustering parameters. Without them a name is a label on nothing: nobody can rebuild the run, and two runs cannot be told apart except by their numbers.
  • Which genes it applies to. Not how many -- which. A clustering of a subsample has fewer labels than the map has genes, and lining them up by position would put cluster 3's color on whichever gene happens to sit at that index. The mask travels with the labels for exactly that reason.

Saved as one .npz of arrays plus one .json of the recipe, in the same shape as EmbeddingStore, so a run and the embedding it came from are read the same way and neither is a special case.

NOISE = -1
NOISE_NAME = 'unclustered'
@dataclass
class Run:

One clustering, with everything needed to redraw it, name it, and rebuild it.

Run( name: str, labels: numpy.ndarray, genes: numpy.ndarray, recipe: dict = <factory>, created: str = '')
name: str
labels: numpy.ndarray
genes: numpy.ndarray
recipe: dict
created: str = ''
n_clusters: int
noise_frac: float
n_genes: int
def describe(self) -> str:

One line for a list: the name, how many clusters, and how much it left unclustered.

def values(self, n_nodes: int) -> pandas.Series:

The run as a categorical column over the WHOLE node table.

Genes the run does not cover get "", which every part of this application already treats as absence -- they are not unclustered, they were not in the map. Conflating the two would put 6,000 genes into a category that means something quite different.

def timestamp_name(when=None, prefix: str = 'run') -> str:

A name from the clock, to the second, so two runs a minute apart are distinguishable.

class RunStore:

Kept clusterings, in memory and on disk, in the order they were made.

In memory as well as on disk because the panel lists them constantly and the disk copy exists so a run survives the window; load_all puts the file back into the list on the next start.

RunStore(root: str | None = None)
root
runs: list
def add( self, labels, genes, recipe=None, name: str = '', created: str = '') -> Run:

Keep a clustering. Unnamed, it takes a timestamp name.

def get(self, name: str):

The run of that name, or None.

def rename(self, old: str, new: str) -> bool:

Rename a run, refusing a name already in use.

Refused rather than silently suffixed: two runs called the same thing in a list you choose from is worse than being told to pick another name.

def names(self) -> list:

Every kept run's name, in the order they were made -- what the color-by list shows.

def save(self, run: Run) -> str:

Write a run and its recipe. Without the recipe the name is a label on nothing.

def load_all(self) -> list:

Read every saved run back into the list, newest last, skipping any that will not open.

def bin_column(values, bins: int = 5, log=None) -> pandas.Series:

A numeric column as a categorical one, labeled by its ranges.

Coloring a quantity by a ramp is already offered; binning makes it behave like a category, which is what makes it comparable with a clustering -- the comparison this panel exists for.

Quantile bins, not equal-width: nearly every quantity in this table is heavy-tailed (the fitness screens span 64x within themselves), and equal-width bins put 95% of the genes in one of them and call the result a coloring. Missing values become "", which is absence everywhere else -- they are not a low bin.

Fewer bins than asked for is an answer, not a failure, and it is reported rather than worked around. n_publications is zero for most of this proteome, so its quartile edges are all zero and quantile binning yields one bin. Splitting the tie anyway -- by rank, or by equal width -- would draw four colors over a column that has one level, which is a picture of a distinction that does not exist. The honest coloring of a mostly-constant quantity is mostly one color.