spacr.projects

N4 — every project on disk in one place: stage, size, last run, staleness.

Until now the only way to see the shape of your work was to navigate by folder. Which plates have been measured? Which finished last week and have since had their masks re-run underneath them? Which one is the 400 GB that filled the disk? Each of those questions had an answer somewhere in spaCR and none of them had a list.

This module is that list, and it is deliberately thin: everything it reports already existed, and re-deriving any of it would be how the browser and the rest of spaCR start disagreeing.

What the browser shows	Where the answer comes from
stage reached	spacr.ports.declared_outputs() — which of a module’s declared outputs are on disk. Not a hand-written ladder of stage names: the order is topologically sorted out of the port graph, so a module registered by a plugin takes its place in it without this file being edited.
size, and what of it	spacr.data_manager.scan_project(). ONE walk of
is unaccounted for	the tree, per-kind attribution, and the registry reconciled against the filesystem. There is no second walker here, and there must not be one.
last run	the registry’s newest created_ns for the project; failing that, the mtime of the outputs themselves. Which of the two answered is reported, because “spaCR recorded this run” and “something wrote into this folder” are different claims.
what is stale	spacr.artifacts.Registry.is_stale(), whose Staleness already carries the reasons, the machine cause codes and the separate missing flag.
what to run next	spacr.chaining.next_steps(), so an offer here is the same offer, with the same readiness check, that the module’s own screen makes.

The project the registry has never seen

This is the case the browser exists for and the one that is easy to get wrong. A user copies a colleague’s plate folder onto the machine and opens spaCR: a browser that lists only what spaCR itself recorded shows them nothing, which is worse than useless because it looks like an answer.

So a project is found on disk (discover(), looks_like_project()) rather than read out of a registry, and an unrecorded one appears with everything the filesystem can answer — stage, size, files, last write — and with ProjectSummary.known False. What it must NOT do is report “0 stale”, which reads as clean. With no provenance there is nothing to compare against, so staleness is unknown, and ProjectSummary.staleness_known is the flag that says so. ProjectSummary.staleness_note() puts it in words for a table cell.

Nothing here imports Qt, so the same summary is available from a notebook and from a test, and the browser screen is a renderer of it.

Classes

ModuleState	Whether one module has run in a project, judged by what it declares.
ProjectSummary	One project, as the browser lists it.
StaleArtifact	One registered result that no longer matches what made it.

Functions

browse(→ Tuple[ProjectSummary, ...])	Find every project under roots and summarise each. The browser.
discover(→ Tuple[str, ...])	Find project folders under roots.
evidence_ports(→ Tuple[spacr.ports.Port, ...])	The produced ports whose existence PROVES module ran here.
format_project(→ str)	Render one ProjectSummary as a block of text.
format_projects(→ str)	Render a whole browse as a table, one row per project.
looks_like_project(→ bool)	Whether a folder is a spaCR project, judged without the registry.
module_states(, records, ...])	Judge every producing module against one project folder.
pipeline_order(→ Tuple[str, ...])	Producing modules, upstream first.
producing_modules(→ Tuple[str, ...])	Every declared module that writes something, sorted.
scan(→ ProjectSummary)	Summarise one project. The whole of spacr.projects in one call.

Module Contents

class spacr.projects.ModuleState

Whether one module has run in a project, judged by what it declares.

Parameters:

• module – the module key.

• state – STATE_DONE, STATE_PARTIAL or STATE_ABSENT.

• found – roles whose declared output is on disk — or, when the answer came from the registry, the roles it recorded.

• missing – required roles whose declared output is not there. An optional output that was cleaned up (masks/) is not missing — the declaration already says it may legitimately be absent, and reporting it would make every tidied project look broken.

• optional_missing – optional roles that are absent, kept separate so the difference stays visible without being alarming.

• newest_ns – when the newest of the found outputs was written.

• evidence – what answered — SOURCE_FILESYSTEM, SOURCE_REGISTRY, or "" when nothing did.

• detectable – whether this module has any on-disk signature at all. See evidence_ports(). False plus STATE_ABSENT means “unknown”, not “no”.

describe() → str

One line: the module, its state and what is short.

property ran: bool

True when this module left anything at all behind.

class spacr.projects.ProjectSummary

One project, as the browser lists it.

Parameters:

• root – absolute project root.

• name – its folder name — what the table’s first column shows.

• exists – the folder is there. A registry can outlive the data.

• known – the registry holds at least one artifact for it. False for the folder a user just copied in, and the reason staleness_known exists.

• has_registry – a registry file sits in the project. True with known False when the file exists but records another project — the shared-registry case (spacr.artifacts.ARTIFACTS_DB_ENV).

• stage – the furthest module in pipeline_order() that left anything behind, or "" for a project nothing has been run on.

• modules – every producing module’s ModuleState, in pipeline order.

• size_bytes – every byte under the root, from spacr.data_manager.scan_project().

• n_files – how many files that is.

• unregistered_bytes – of those, how many nothing claims.

• unregistered_files – and how many files.

• n_artifacts – registry rows for this project.

• last_run_ns – when it last produced something.

• last_run_utc – the same instant, ISO-8601, or "".

• last_run_source – SOURCE_REGISTRY or SOURCE_FILESYSTEM — which question was answered. They are not the same claim and a browser that blurs them is lying by omission.

• stale – results that no longer match their inputs.

• missing – registered results whose file is gone.

• next_steps – modules that could run next, ready ones first, as (module, blocked_reason); the reason is "" when it can run.

• errors – paths that could not be read.

• scanned_utc – when this summary was taken.

• usage – the full spacr.data_manager.ProjectUsage, for a detail pane that wants the per-kind breakdown.

describe() → str

The full report; see format_project().

note() → str

The one thing about this project worth saying in a table.

Ordered by what a user has to act on: a folder that is gone, then a project spaCR has no record of, then results that no longer match, then a large pile of bytes nobody claims.

staleness_note() → str

What the stale column should say, in words.

property n_stale: int

How many recorded results are out of date.

property ran: Tuple[str, ...]

Every module that left something behind, in pipeline order.

property stage_label: str

the module, and whether it finished.

Type:

The stage as a table cell

property staleness_known: bool

Whether “is anything stale?” has an answer at all.

False for a project the registry has never seen. Reporting such a project as having zero stale artifacts would read as clean, and it is not clean — it is unexamined.

class spacr.projects.StaleArtifact

One registered result that no longer matches what made it.

A thin projection of spacr.artifacts.Staleness onto the artifact it describes, so a table row has the kind and the producing module beside the reason without the caller re-querying the registry.

Parameters:

• artifact_id – the registry id.

• kind – a spacr.ports kind.

• module – the module that produced it.

• role – its port role.

• path – where it is, or was.

• reasons – the registry’s own sentences.

• causes – the machine cause codes, e.g. "upstream-newer".

• missing – the file is gone. An availability problem, not a provenance one — spacr.artifacts.Staleness keeps the two apart and so does this.

describe() → str

One line, fit for a list under a project.

explain() → str

The causes as one readable clause, via spacr.chaining.

spacr.projects.browse(roots: Iterable[Any], *, depth: int = DEFAULT_DEPTH, registry: spacr.artifacts.Registry | None = None, limit: int = 500, with_next_steps: bool = True, on_progress: Any | None = None) → Tuple[ProjectSummary, ...]

Find every project under roots and summarise each. The browser.

Parameters:

• roots – folders to search, or the projects themselves.

• depth – how deep to look; see discover().

• registry – an open registry to read every project through — the shared-registry case. Omit and each project’s own is used.

• limit – stop after this many projects.

• with_next_steps – passed to scan().

• on_progress – optional fn(done, total, root), called on the calling thread after each project. A GUI passes something that only touches its own counters — this runs on a worker thread.

Returns:

summaries, most recently run first, then by name. A project nobody has run sorts last, which is where a browser wants it.

spacr.projects.discover(roots: Iterable[Any], *, depth: int = DEFAULT_DEPTH, limit: int = 500) → Tuple[str, ...]

Find project folders under roots.

Descent stops at a project: a project’s merged/ holds .npy files and would otherwise be reported as a project of its own, which is both wrong and the kind of wrong that fills a table with noise.

Parameters:

• roots – folders to search. A root that is itself a project is returned as one.

• depth – how many levels below each root to look. 0 means “test the roots themselves and nothing under them”.

• limit – stop after this many projects. A browser pointed at a home directory must return something rather than walk a filesystem.

Returns:

absolute paths, sorted, de-duplicated.

spacr.projects.evidence_ports(module: str) → Tuple[spacr.ports.Port, ...]

The produced ports whose existence PROVES module ran here.

The naive reading of “stage reached” — every declared output that is on disk — is wrong, and wrong in a way a user would notice immediately. ml_analyze declares one output, measurements/measurements.db, which mask created and measure filled. On that reading, a project that has only been segmented reports classical ML as complete.

So a produced port counts as evidence only when nothing EARLIER in pipeline_order() already explains the same location with a different kind. An earlier writer’s file being there says the earlier module ran, and nothing more. This is the same distinction spacr.artifacts draws when it keeps object-counts and measurements-db apart in one file — “mask creating the file does not make the measurements in it current”.

Two consequences worth stating rather than discovering:

• measure is left with only its optional data/ crops, because mask wrote the database first. A measured project whose crops were never saved therefore reports its stage as mask from the filesystem alone. The registry answers it exactly when it has a record — see module_states().

• A module with no evidence ports at all (ml_analyze) has no on-disk signature whatsoever. ModuleState.detectable is False for it, so the browser can say “no way to tell without a run record” instead of the flatly wrong “not run”.

Modules that declare an identical set of outputs are interchangeable (mask and timelapse share theirs literally — ports.py passes the same tuple to both). Only the earlier of such a pair carries filesystem evidence, or one segmented project would report that both had run.

Parameters:

module – module key or alias.

Returns:

the subset of module_ports(module).produces that is discriminating, in declaration order.

spacr.projects.format_project(summary: ProjectSummary, *, limit: int = 6) → str

Render one ProjectSummary as a block of text.

Parameters:

• summary – what scan() returned.

• limit – how many stale entries to name.

spacr.projects.format_projects(summaries: Sequence[ProjectSummary]) → str

Render a whole browse as a table, one row per project.

Parameters:

summaries – project summaries to render in their existing order.

spacr.projects.looks_like_project(root: Any) → bool

Whether a folder is a spaCR project, judged without the registry.

Parameters:

root – folder to inspect for registry, artifact, input, or output evidence of a spaCR project.

True when a registry file sits in it, or when any declared output of any producing module is on disk, or when the mask pipeline’s declared input finds raw images there. That last clause is what makes a plate folder somebody just copied in a project: nothing has been run on it yet, and it is exactly the folder a user wants the browser to list.

Reusing the port declarations rather than testing for merged/ by name means a plugin’s module makes its own outputs count as evidence.

spacr.projects.module_states(root: Any, *, modules: Sequence[str] = (), records: Sequence[spacr.artifacts.Artifact] = ()) → Tuple[ModuleState, ...]

Judge every producing module against one project folder.

Two sources, in order of authority. A run record in records settles the question outright — the registry saw the run happen. Without one, the answer comes from the discriminating outputs on disk (see evidence_ports()), which is what makes a project spaCR has never seen readable at all.

Parameters:

• root – the project root.

• modules – which modules to judge, in the order to report them. Defaults to pipeline_order().

• records – artifacts the registry holds for this project.

Returns:

one ModuleState per module, in that order.

spacr.projects.pipeline_order() → Tuple[str, ...]

Producing modules, upstream first.

A topological sort of spacr.ports.PORTS using spacr.ports.upstream_modules(), with an alphabetical tie-break so the answer is stable between runs. Derived, not written down: a module that registers its ports through spacr.ports.register_module_ports() — a plugin, or a module written after this one — takes its place in the ladder without this file changing.

A cycle cannot happen with the declared graph and is not an error worth refusing over, so one is broken alphabetically and logged: a browser that raised rather than showing a slightly odd ordering would be a worse tool.

spacr.projects.producing_modules() → Tuple[str, ...]

Every declared module that writes something, sorted.

A module that only reads — the analysis apps that open measurements.db — cannot be a stage a project has reached, because it leaves nothing behind for anyone to find.

spacr.projects.scan(root: Any, *, registry: spacr.artifacts.Registry | None = None, usage: spacr.data_manager.ProjectUsage | None = None, with_next_steps: bool = True) → ProjectSummary

Summarise one project. The whole of spacr.projects in one call.

Parameters:

• root – the project root. It does not have to be a project, and it does not have to be in the registry — that is the point.

• registry – an open registry to read through. Omit and the project’s own is opened when a file exists; a project with none is summarised anyway, with ProjectSummary.known False.

• usage – a spacr.data_manager.ProjectUsage already taken for this root, to be reused rather than re-walked. The browser has none; the Data Manager screen, which has just walked the same project, does.

• with_next_steps – compute what could run next. Off skips one spacr.ports.check_ready() per successor, which is the only part of this that globs a second time.

Returns:

a ProjectSummary. Never raises for a folder that is missing, unreadable or not a project: “there is nothing here” is an answer a browser has to be able to show.