Edit on GitHub

starplast.results

Saving what the analysis tabs computed, and loading it back so it is still clickable.

A search is minutes to hours and every tab's table is lost when the window closes. What makes that worth fixing here rather than by exporting a CSV is what a row in these tables IS: not a summary but a recipe -- blocks, missing-value policy, scaling, both UMAP hyperparameters, the clustering size, the seed, the subsample and the excluded columns. That is why clicking a row rebuilds its map.

So a loaded table has to be as clickable as a computed one. Load a search from last week, click a row, and the map rebuilds. A loader that produced a table you could only read would be a screenshot with extra steps.

That requires one thing the plain CSV export does not do: recording WHICH table a file came from. A validation table dropped into the walk tab makes rows nobody can rebuild and an error message about the wrong thing, so the file names its table and loading refuses a mismatch instead of guessing.

Two shapes, the same format underneath:

one table   a CSV whose first line is `# starplast-table: recovery_search`
every tab   a zip of those CSVs plus a manifest saying what is in it

A zip of CSVs rather than a pickle or an HDF5: these are results, someone will want to open one in a spreadsheet or read it on a machine that has never had this program installed, and a format that requires the program to read it is a format that loses the results the day the program stops building.

MARKER = '# starplast-table:'

MANIFEST = 'manifest.json'

def save_table(path: str, kind: str, df: pandas.DataFrame) -> str:

Write one table, naming which table it is on the first line.

def table_kind(path: str) -> str:

Which table a file holds, or "" if it does not say.

A file that does not say is still loadable -- somebody's own CSV, or one saved before this existed -- but the caller is the one that decides whether to accept it, because only the caller knows which table it is being dropped into.

def load_table(path: str) -> pandas.DataFrame:

Read a saved table, with or without the marker line.

def save_bundle(path: str, tables: dict, meta: dict | None = None) -> str:

Write every tab's results into one file.

Empty tables are skipped rather than written as headers: a bundle that lists eight tables of which six are empty invites the reader to think six analyses returned nothing, when they were never run.

def load_bundle(path: str) -> tuple:

Read a bundle: (tables, manifest).

A member that will not parse costs that table and says so, rather than the whole file: a bundle is a session's work and losing seven tables to one bad row would be its own kind of failure.

def is_bundle(path: str) -> bool:

Whether a path is a bundle rather than a single table.

class RowLog:

A results table written to disk one row at a time, as the run produces them.

A search is minutes to hours, and until now nothing of it reached disk except the embeddings: the rows lived in the panel's frame, so stopping was safe but quitting, crashing or a power cut was not, and "save results" was a step the user had to remember AFTER waiting an hour.

Written in the same format save_table produces -- the marker line, then CSV -- so an autosaved file loads back through the ordinary Load, and its rows are as clickable as any other. Flushed after every row on purpose: a buffer that loses the last twenty rows of an interrupted run is the failure this exists to prevent.

RowLog(path: str, kind: str)

def append(self, row: dict) -> None:

Write one row, opening the file and writing its header on the first.

def close(self) -> str:

Close the file and return its path, or "" if nothing was ever written.

def autosave_path(root: str, kind: str, stamp: str) -> str:

Where a run's rows are written as it computes them.