Edit on GitHub

starplast.leakage

Is the slot grouping tight enough that holding one thing out holds out its information? Measured.

Every held-out analysis in this project rests on one promise: when a target is withheld, everything that restates it is withheld too. search.excluded_detail keeps that promise with five mechanisms -- the target, measured association above a threshold, declared families, shared provenance, the same quantity measured another way -- and this module checks them against the data rather than against intentions.

Three questions, each answered with a number:

  1. How associated can two columns be when they measure DIFFERENT things? The closure's cutoff (0.8) was chosen by hand. Here it is compared with the empirical distribution of association between columns from different axes of the slot catalogue -- expression against fitness, localization against sequence -- whose upper tail is the most that genuinely different measurements reach on this proteome. :func:calibrate reports that tail with a bootstrap interval, and a column pair above it is as associated as nothing measuring different biology gets.
  2. Does the closure's own measure see every near-copy? It uses Pearson correlation between numeric columns, and a log-transformed copy of a column correlates perfectly on ranks and far less on values. :func:association_matrix reports both.
  3. After the closure, does anything left predict the target like a copy would? For each target, :func:residual_leaks scores every PERMITTED slot on how well it alone predicts the target, out of fold, and flags the ones that stand out from the rest -- the undeclared derivations that no pairwise statistic can see, like a label that is the argmax of several columns.

Nothing here changes a closure; the audit reports, and fixes are made where the mechanisms are declared (datasets provenance and derived_from, search.SAME_QUANTITY, slot families), so the reason for each exclusion stays readable.

CATEGORICAL_MAX_UNIQUE = 12
MIN_OVERLAP = 50
SKIP = ('gene_id', 'sequence', 'product', 'alphafold_accession', 'interpro_desc', 'interpro_id', 'pfam_id', 'orthogroup', 'ec_number')
def is_categorical(s: pandas.Series) -> bool:

The closure's rule: non-numeric, boolean, or numeric with a dozen values or fewer.

def measured_columns(nodes: pandas.DataFrame, min_values: int = 50) -> list:

Every column that is a measurement: not an identifier, measured on enough genes, varying.

def association_matrix(nodes: pandas.DataFrame, columns=None, max_categories: int = 30) -> tuple:

(on ranks, on values): column-by-column association in [0, 1], pairwise complete.

Numeric pairs are |Spearman| in the first matrix and |Pearson| -- the closure's own measure -- in the second; a pair involving a category is Cramer's V or the correlation ratio in both, as the closure computes them. Pairs measured on fewer than MIN_OVERLAP genes are NaN.

def slot_of_columns(nodes: pandas.DataFrame, organism: str) -> dict:

column -> the catalogue slot that declares it (the first, if several do).

def relation(a: str, b: str, owner: dict, organism: str | None = None) -> str:

How the catalogue relates two columns: the same slot, family, quantity, experiment, or none.

different axis pairs are the reference for what genuinely different measurements look like; same axis pairs measure related things (two expression series) and are reported, not used to calibrate.

def pair_table( ranks: pandas.DataFrame, values: pandas.DataFrame, owner: dict, organism: str | None = None) -> pandas.DataFrame:

Every column pair once, with both associations, their slots and their declared relation.

def calibrate( pairs: pandas.DataFrame, quantile: float = 0.999, n_boot: int = 1000, seed: int = 0) -> dict:

The most two columns from different axes are associated, with a bootstrap interval.

The quantile of the association between columns whose slots sit on different axes of the catalogue: pairs measuring genuinely different things. Bootstrapped over SLOT PAIRS rather than column pairs, because the columns of one slot are not independent draws and resampling them as if they were would make the interval falsely narrow.

def closure_gaps( nodes: pandas.DataFrame, pairs: pandas.DataFrame, threshold: float, targets=None) -> pandas.DataFrame:

Pairs the closure lets through although they are more associated than threshold.

For each target (every measured column by default), the columns search.excluded_for removes are compared with the columns associated with it at or above threshold; what is associated and NOT removed is a gap -- a copy of the target a held-out analysis would still be allowed to read.

def residual_leaks( nodes: pandas.DataFrame, target: str, organism: str = 'Tg', z: float = 4.0, floor: float = 0.5, log=None) -> pandas.DataFrame:

Every slot the closure PERMITS for target, scored on how well it alone predicts it.

A slot that restates the target -- even as a joint function no pairwise statistic can see -- predicts it far better than the other permitted slots do. Flagged when it is BOTH a robust outlier -- more than z median absolute deviations above the median slot -- AND at least floor (kappa or correlation): halfway to a copy. The outlier test alone flags ordinary biology whenever every other slot sits near zero -- expression predicts compartment at kappa 0.1, six deviations above a median of 0.01 -- and the floor alone would flag any strong but genuine predictor. A flag says "look"; the table says what to look at.