Resumable multi-objective UMAP search

Image UMAP can search n_neighbors, min_dist and the other structural UMAP parameters using one criterion or a multi-objective mode. Open UMAP settings… from the Hyperparameter search panel on the Image UMAP screen; multi_objective is the first and default criterion in that panel. The Python function spacr.hyperparam.umap_search() defaults to metric="trustworthiness".

Why several objectives?

No score proves that a UMAP contains biologically meaningful structure. The multi-objective mode therefore retains three separate measurements:

neighborhood_preservation

The geometric mean of trustworthiness and continuity. Trustworthiness penalizes neighbours invented by the two-dimensional embedding; continuity penalizes feature-space neighbours that the embedding places far apart. Because the geometric mean is low when either term is low, a high value requires both.

stability

The mean fraction of nearest neighbours shared by embeddings fitted with different reproducible seeds. It is invariant to rotations and reflections. The stability neighbourhood is capped at one fewer than the row count; comparing every other row gives a score of 1 regardless of local structure. Use a smaller neighbourhood to assess repeat-to-repeat local agreement.

cluster_structure

The silhouette coefficient clipped to the range 0 to 1, so negative values score 0. Labels supplied through the Python API define the partition only when their length matches the rows and they contain at least two classes but fewer classes than rows. Otherwise spaCR uses seeded K-means partitions with 2 through 8 clusters (at most one fewer than the number of rows), requiring at least four rows, and selects the highest silhouette among valid partitions. If no partition resolves, the score is zero and cluster_structure_method is no_resolved_clusters. Check that field and the selected cluster counts to see which partition was used. This measures geometric structure, not biological identity.

The three weights are normalized to sum to one. Their weighted geometric mean is called multi_objective and guides the grid or adaptive search. The geometric mean penalizes a collapsed objective more strongly than an arithmetic average.

Pareto front

The composite score gives one ranking of the trials. spaCR also marks the Pareto front in the result table. A configuration is Pareto-optimal when no other tested configuration improves one objective without making another worse. Inspect the embedding panels and objective tooltips for every non-dominated row before propagating a configuration.

The neighborhood, stability and cluster objectives, normalized weights, repeat count, raw silhouette, cluster source and discovered cluster counts are stored in each trial’s extra_metrics.

Repeated fits and runtime

stability repeats controls how many seeded embeddings are fitted per configuration. The minimum is 2 and the default is 3. Runtime scales roughly as:

configurations × stability repeats × one UMAP fit

Use three repeats for routine searches and increase the count when the leading Pareto configurations have similar stability or when a final analysis must be especially reproducible.

Walk search and stopping

Switch on Walk to search locally from the values in the parameter fields instead of sweeping a grid. Each round scores the neighbourhood around the current centre and moves to the best configuration in it; with the default axes, n_neighbors and min_dist at resolution 2, that is the four diagonal corners. Axes… adds other structural parameters and sets each axis’s step and resolution. The weighted multi-objective score chooses the direction of the next move. Search stops at maximum rounds, when a round fails to exceed minimum improvement, or after Stop is requested. A stopped result is explicitly marked partial.

Resume behavior

Enable Resume checkpoint to continue from results/.spacr_checkpoints/umap_search.json under the current project (or from an explicit checkpoint_path in the Python API). Each completed trial is written atomically and its primary embedding is stored beside the JSON. An interrupted Walk round evaluates only its missing candidates before choosing a direction.

Resume refuses to combine incompatible work. Feature and label hashes, search space, criterion, seed, output dimensions, neighborhood size, the n_neighbors/min_dist increments, stopping threshold, stability repeat count, objective weights, embedder identity, backend and the cluster-during-search settings must match the checkpoint.

Python API

Use spacr.hyperparam.umap_search() for a full search, spacr.hyperparam.umap_objective_scores() to score repeated embeddings, spacr.hyperparam.embedding_stability() for stability alone, and spacr.hyperparam.SearchResult.pareto_front() to retrieve non-dominated trials.

from spacr.hyperparam import SearchSpace, umap_search

result = umap_search(
    features,
    SearchSpace({"n_neighbors": [5, 15, 50], "min_dist": [0.05, 0.2]}),
    metric="multi_objective",
    stability_repeats=3,
    objective_weights={
        "neighborhood_preservation": 0.4,
        "stability": 0.3,
        "cluster_structure": 0.3,
    },
    checkpoint_path="results/.spacr_checkpoints/umap_search.json",
    resume=True,
)
for trial in result.pareto_front():
    print(trial.params, trial.extra_metrics)