spacr.cli_download¶

spacr-download — fetch spaCR’s published example data, with no GUI.

Every example dataset spaCR publishes has, until now, been reachable only by pressing a button inside the application: “Load test data”, “Load example data”, the screen-data picker. That is fine on a laptop and useless on a cluster, where the data has to be on disk BEFORE a batch job starts and there is no display to press a button on. This is that download as a command.

WHAT IT WILL AND WILL NOT DO WITHOUT BEING ASKED. With no arguments it fetches the three example sets – Mask, Measure, Annotate/Classify – which come to about 1.1 GB. It does NOT fetch the published TSG101 screen, which is 33 GB. A command that spent 33 GB of somebody’s quota because they typed its name with no arguments would be a bug however well documented, so the screen is opt-in, is asked for in pieces, and is confirmed before it starts. The pieces are spacr.screen_data’s, unchanged: four measurement databases of about 0.5 GB and four crop folders of about 8 GB, any subset of which can be named.

NOTHING IS DOWNLOADED THAT CANNOT FIRST BE PRICED. --list prints every piece with its size and whether it is already on disk, and the total the current selection would cost, without opening a socket. --dry-run is the same listing for the same reason: a user deciding what to spend an hour of network on should be able to see the bill first.

Importing this module must stay light – no Qt, no torch, no matplotlib – so spacr-download --help answers instantly on a login node with a cold NFS cache. That is why the download primitives live in spacr.example_archives rather than in spacr.qt.hf_download, which imports PySide6 at module scope. tests/test_cli_download.py pins it.

Usage:

spacr-download                            # every example set (~1.1 GB)
spacr-download --list                     # what exists, what is here
spacr-download measure annotate           # two of the three
spacr-download --screen measurements      # the four databases (~2.1 GB)
spacr-download --screen crops --plate 1   # one plate of crops (~8.9 GB)
spacr-download all --yes                  # everything, screen included

Exit codes (a job that exits 0 having downloaded nothing is the classic footgun, so these are exact):

everything asked for is on disk, or a confirmation was declined
a download failed; the pieces that succeeded are still on disk
bad arguments, not enough disk space, or a large download that could not
   be confirmed because nothing was there to confirm it

Exceptions¶

SelectionError

A name, kind or plate number the user got wrong.

Classes¶

Piece

One archive this command can fetch, priced and located.

Functions¶

`build_parser`(→ argparse.ArgumentParser)	Return the `spacr-download` argument parser.
`build_plan`(→ List[Piece])	One `Piece` per thing to fetch, with its size and its folder.
`default_destination`(→ pathlib.Path)	`~/.cache/spacr/example_data` -- where the GUI already looks.
`example_folder`(→ pathlib.Path)	The one plate folder all three example sets unpack into.
`main`(→ int)	`spacr-download` entry point.
`render_listing`(→ str)	Every piece, its size, whether it is here, and what the total would be.
`resolve_selection`(, *, screen, plates, ...)	Turn what the user typed into the exact list of things to fetch.
`screen_folder`(→ pathlib.Path)	One folder per screen plate, and it has to be.

Module Contents¶

exception spacr.cli_download.SelectionError[source]¶

Bases: Exception

A name, kind or plate number the user got wrong.

Always exit code 2: nothing was attempted, so it is an argument problem rather than a failed download.

Initialize self. See help(type(self)) for accurate signature.

class spacr.cli_download.Piece[source]¶

One archive this command can fetch, priced and located.

Examples and screen pieces are described by two different dataclasses – ExampleSet and ScreenAsset – because they answer to two different publishers. This is the one shape the listing, the size arithmetic and the download loop all work in, so none of them has to know which kind of thing it is holding.

Parameters:

key – how a selection names it, unique across both publishers.
name – what the listing’s first column shows.
detail – the rest of the row – a summary for an example set, the archive name for a screen piece.
repo – the dataset repository it is published in.
archive – the .tar to fetch from it.
folder – where it unpacks.
bytes – the archive’s size, so a selection can be priced before it is paid.
present – whether it is already unpacked where it would go. Computed when the plan is built rather than looked up later, so the listing and the download agree about what will happen.
expands_npz – whether the unpacked arrays have to be rewritten as .npy afterwards.

spacr.cli_download.build_parser() → argparse.ArgumentParser[source]¶

Return the spacr-download argument parser.

Building it imports nothing beyond the standard library and two dependency-light spaCR modules, so --help is instant.

spacr.cli_download.build_plan(examples: Sequence[spacr.example_archives.ExampleSet], assets: Sequence[spacr.screen_data.ScreenAsset], dest) → List[Piece][source]¶

One Piece per thing to fetch, with its size and its folder.

Parameters:

examples – the example sets to include.
assets – the screen pieces to include.
dest – the root everything unpacks under.

spacr.cli_download.default_destination() → pathlib.Path[source]¶

~/.cache/spacr/example_data – where the GUI already looks.

Chosen so that a plate fetched by this command is the plate the application’s own “Load example data” buttons point at: spacr.example_archives.example_plate_folder() is <this>/plate1. Someone who ran spacr-download before opening the GUI should find the data already there, not download it twice.

spacr.cli_download.example_folder(dest) → pathlib.Path[source]¶

The one plate folder all three example sets unpack into.

ONE FOLDER because the sets compose: the Measure example’s merged/, the Annotate example’s data/ and measurements/, and the Mask demo’s raw images are three stages of the same plate, and spaCR expects to be pointed at a plate.

Parameters:: dest – the root everything unpacks under.

spacr.cli_download.main(argv: Sequence[str] | None = None) → int[source]¶

spacr-download entry point.

Parameters:: argv – argument list; sys.argv[1:] when None.
Returns:: 0 done, 1 a download failed, 2 bad arguments or no room.

spacr.cli_download.render_listing(plan: Sequence[Piece], chosen: Sequence[Piece], dest, *, force: bool = False) → str[source]¶

Every piece, its size, whether it is here, and what the total would be.

plan is the whole inventory and chosen is the selection, so the listing answers both “what is there?” and “what would this command do?” at once. A * marks the selected rows.

Parameters:

plan – every piece there is.
chosen – the pieces this run would fetch.
dest – the root everything unpacks under.
force – whether pieces already on disk count towards the total.

spacr.cli_download.resolve_selection(what: Sequence[str] = (), *, screen: str | None = None, plates: Sequence[int] = ()) → Tuple[List[spacr.example_archives.ExampleSet], List[spacr.screen_data.ScreenAsset]][source]¶

Turn what the user typed into the exact list of things to fetch.

Kept apart from argparse so the rules can be read – and tested – as rules rather than as a parser’s side effects.

Parameters:

what – names and groups: an example key, examples, screen or all. Empty means the default.
screen – measurements, crops, all or None. Naming a kind is itself a request for the screen, so --screen crops needs no positional argument to go with it.
plates – which screen plates; empty means all of them.

Returns:

(example sets, screen assets) in listing order.

Raises:

SelectionError – on a name, kind or plate that does not exist. A typo must not quietly select nothing and then report success.

spacr.cli_download.screen_folder(dest, plate: int) → pathlib.Path[source]¶

One folder per screen plate, and it has to be.

Every plate’s measurements archive unpacks to measurements/measurements.db and every plate’s crop archive unpacks to data/ – the same two paths, four times over. Unpacked into one folder the fourth plate would silently overwrite the third, and spacr.screen_data.ScreenAsset.is_present() would report a plate as downloaded because a DIFFERENT plate’s file is sitting where its own would go. So the plate number is in the path, and a selection of all four is four plate folders that can each be opened as itself.

Parameters:

dest – the root everything unpacks under.
plate – which screen plate.