spacr.parameter_sweep

Parameter Sweep: vary regression settings and see which change the answer.

A pooled screen has no single correct analysis. It has a model family, an aggregation rule, a unit of analysis, a set of nuisance effects, a multiple-testing correction, and three or four filtration cutoffs – and a hit list that is stable across all of them is a very different claim from one that appears under exactly one combination.

This module makes that comparison a command rather than a week. It builds a trial list from a search space, runs each trial into its own folder, and returns one tidy row per trial: the settings, whether it ran, how many wells and guides survived, how many hits it called, where the named controls landed, and how long it took.

Two things make it fast enough to be worth having:

• Preparation is shared. Loading 226k score rows and 642k count rows, aggregating per well, thresholding and joining costs about twenty seconds and depends only on the FILTRATION settings. Every trial sharing those reuses one prepared frame, so a sweep over models and corrections pays that cost once per filtration cell rather than once per trial.

• A failed trial is a result. Many combinations are illegal by construction – quantile refuses alpha, the penalised families refuse cov_type, random_row_column_effects replaces the backend entirely. Those are recorded with their reason and the sweep continues, because “this combination is not allowed” is information about the design space, not a crash.

The search space is declared as data (DEFAULT_SWEEP_SPACE), so adding an axis is adding a key.

Classes

SweepSpace

Define variable and fixed settings for a parameter sweep.

Functions

build_trials(→ list[dict])	Enumerate accepted trials from a sweep space.
rank_trials(→ pandas.DataFrame)	Order trials by recovery of a named control role.
recommended_workers(*[, measured_gib, requested])	Choose a worker count from available memory and CPU capacity.
run_sweep(→ pandas.DataFrame)	Run parameter-sweep trials sequentially.
run_sweep_parallel(→ pandas.DataFrame)	Run parameter-sweep trials concurrently in spawned processes.
summarise_sweep() → dict)	Summarize sweep completion, controls, and hit-count sensitivity.

Module Contents

class spacr.parameter_sweep.SweepSpace

Define variable and fixed settings for a parameter sweep.

Parameters:

• axes (dict of str to list, optional) – Settings to vary. Each trial receives one value from every list. Defaults to a copy of DEFAULT_SWEEP_SPACE.

• fixed (dict, optional) – Settings copied into every trial after the Cartesian product is formed and before filters are evaluated.

• filters (list of callable, optional) – Predicates called with a complete trial dictionary. A predicate returns None to accept a trial or a reason string to reject it. An empty list selects the built-in compatibility filters when trials are built.

size() → int

Return the raw Cartesian-product size before filtering.

spacr.parameter_sweep.build_trials(space: SweepSpace, *, mode: str = 'grid', max_trials: int = 5000, seed: int = 0) → list[dict]

Enumerate accepted trials from a sweep space.

Parameters:

• space (SweepSpace) – Variable axes, fixed settings, and optional rejection predicates.

• mode ({'grid', 'random'}, default='grid') – 'grid' visits the Cartesian product in axis order. 'random' shuffles that product without replacement before applying the limit.

• max_trials (int, default=5000) – Maximum number of accepted trials to return. Rejected combinations do not count toward this limit.

• seed (int, default=0) – Seed used to shuffle combinations in 'random' mode. It has no effect in 'grid' mode.

Returns:

list of dict – Complete setting dictionaries with one-based trial_id values. Fixed settings are present when filters are evaluated.

Raises:

ValueError – If mode is not 'grid' or 'random'.

Notes

Random mode materializes the full Cartesian product before shuffling it. Use narrower axes when the unfiltered product is very large.

spacr.parameter_sweep.rank_trials(results: pandas.DataFrame, *, role: str = 'positive') → pandas.DataFrame

Order trials by recovery of a named control role.

Parameters:

• results (pandas.DataFrame) – Sweep results containing <role>_control_percentile and optionally status.

• role ({'positive', 'negative'}, default='positive') – Control role whose percentile determines the ordering.

Returns:

pandas.DataFrame – Copy ordered by increasing control percentile, with missing values and failed trials last. The input object is returned unchanged if the percentile column is absent or contains no finite values.

Notes

Percentile is used instead of raw rank so trials that fit different numbers of coefficients remain comparable. Trials that did not recover the control remain in the table rather than being dropped.

spacr.parameter_sweep.recommended_workers(*, measured_gib=None, requested=None)

Choose a worker count from available memory and CPU capacity.

Parameters:

• measured_gib (float or None, optional) – Peak resident memory for one representative trial, in GiB. None uses ASSUMED_TRIAL_GIB.

• requested (int or None, optional) – Preferred maximum worker count. The result is still limited by memory, available CPU cores, and MAX_WORKERS.

Returns:

• workers (int) – Recommended number of worker processes, always at least one.

• reason (str) – Explanation of the memory estimate and any reduction from the requested count, suitable for logs. The Qt screen renders the same budget with localized templates.

Notes

The calculation budgets MEMORY_BUDGET_FRACTION of currently available memory. If memory cannot be measured, at most two workers are recommended.

spacr.parameter_sweep.run_sweep(base_settings: Mapping[str, Any], destination, space: SweepSpace | None = None, *, mode: str = 'grid', max_trials: int = 5000, seed: int = 0, controls: Mapping[str, str] | None = None, progress_every: int = 10, learn_from_failures: int = 2, corrections: Sequence[str] | None = None, contained: bool = True, qc: bool = False, memory_floor_gb: float = FREE_MEMORY_FLOOR_GB, runner: Callable | None = None) → pandas.DataFrame

Run parameter-sweep trials sequentially.

Parameters:

• base_settings (mapping) – Regression settings shared by every trial, including the score and count inputs.

• destination (path-like) – Directory for trial folders, sweep_trials.json, and sweep_results.csv.

• space (SweepSpace or None, optional) – Search space. None uses DEFAULT_SWEEP_SPACE and the built-in compatibility filters.

• mode ({'grid', 'random'}, default='grid') – Trial enumeration order passed to build_trials().

• max_trials (int, default=5000) – Maximum number of accepted trials.

• seed (int, default=0) – Random-order seed used when mode='random'.

• controls (mapping or None, optional) – Mapping from control aliases to identifiers. Control recovery metrics are added to each successful result row.

• progress_every (int, default=10) – Print and attempt an incremental CSV write after this many trials. Use zero to disable periodic progress messages; a CSV is still written after each completed in-process trial.

• learn_from_failures (int, default=2) – Skip later trials with the same model, inference, analysis-unit, and penalty signature after this many matching failures. Use zero to run every trial.

• corrections (sequence of str or None, optional) – Multiple-testing methods to apply to the p-values from one fitted model, producing one row per method. This option applies to the in-process path only; contained children return summary rows rather than coefficient frames.

• contained (bool, default=True) – Run each trial through run_trial_contained(). Hard resource limits are conditional on a usable systemd user scope; otherwise the child is uncapped but uses reduced priority and thread limits.

• qc (bool, default=False) – Generate the full regression diagnostic figure suite for every trial.

• memory_floor_gb (float, default=FREE_MEMORY_FLOOR_GB) – Stop starting contained trials when available memory falls below this threshold, in GB.

• runner (callable or None, optional) – In-process regression callable. None uses contained child trials when contained=True and spacr.ml.perform_regression() otherwise. Injected callables bypass child containment and the memory floor.

Returns:

pandas.DataFrame – Trial settings, status, timing, fit metrics, and control metrics. The frame is also written to sweep_results.csv as trials finish.

spacr.parameter_sweep.run_sweep_parallel(base_settings: Mapping[str, Any], destination, space: SweepSpace | None = None, *, mode: str = 'random', max_trials: int = 1000, seed: int = 0, controls: Mapping[str, str] | None = None, n_jobs: int = 8, contained: bool = True, qc: bool = False, progress_every: int = 25) → pandas.DataFrame

Run parameter-sweep trials concurrently in spawned processes.

Parameters:

• base_settings (mapping) – Regression settings shared by every trial, including the score and count inputs.

• destination (path-like) – Directory for trial folders, sweep_trials.json, and the incremental sweep_results.csv table.

• space (SweepSpace or None, optional) – Search space. None uses DEFAULT_SWEEP_SPACE and the built-in compatibility filters.

• mode ({'grid', 'random'}, default='random') – Trial enumeration order passed to build_trials().

• max_trials (int, default=1000) – Maximum number of accepted trials.

• seed (int, default=0) – Random-order seed used when mode='random'.

• controls (mapping or None, optional) – Mapping from control aliases to identifiers. Control recovery metrics are added to each result row.

• n_jobs (int, default=8) – Requested pool size. recommended_workers() may reduce it based on memory and CPU capacity.

• contained (bool, default=True) – Run each fit in a separate child through run_trial_contained(). Hard memory, swap, task, and CPU limits are enforced only when a usable systemd user scope is available; otherwise the child is uncapped but uses reduced priority and thread limits. Set to False only when trial resource use is known.

• qc (bool, default=False) – Generate the full regression diagnostic figure suite for every trial.

• progress_every (int, default=25) – Print progress after this many completed trials. Use zero to disable periodic progress messages.

Returns:

pandas.DataFrame – One status row per trial, sorted by trial_id. The same rows are written incrementally to sweep_results.csv.

Raises:

RuntimeError – If called from a worker process. Scripts must call this function under an if __name__ == '__main__': guard.

Notes

The pool uses the spawn start method because fitted models may import torch and OpenMP runtimes. Only a bounded number of jobs are submitted at once, allowing new submissions to pause when available memory is low.

spacr.parameter_sweep.summarise_sweep(results: pandas.DataFrame, *, controls: Sequence[str] = ('gra14', 'eaf1')) → dict

Summarize sweep completion, controls, and hit-count sensitivity.

Parameters:

• results (pandas.DataFrame) – Result rows produced by run_sweep() or run_sweep_parallel().

• controls (sequence of str, default=('gra14', 'eaf1')) – Control aliases whose presence and rank columns should be summarized when available.

Returns:

dict – Trial counts, elapsed minutes, failure categories, control recovery, and hit-count ranges. Median hit counts are also grouped by correction, regression family, analysis unit, and inference mode when those columns are present. An empty input returns {'trials': 0}.

Notes

The summary emphasizes consistency across defensible analysis choices. Hit counts alone should not be used to select a model or correction.