API reference

The public API is small: a recompress() conversion/recompression entry point, a read-only audit_path() archive assessor, a file-format detector, and the exception hierarchy. Everything below is imported directly from the top-level ncarnate package.

Conversion and recompression

ncarnate.recompress(src, dst=None, zlib=True, shuffle=True, complevel=7, overwrite=True, geolocation=True)[source]

Rewrites the file at src as a netCDF4 file with the given compression settings, losslessly, and returns the output path.

For netCDF3/netCDF4/HDF5 input the output target is resolved as:

  • dst given: the output is written to dst and the source is left untouched (overwrite is ignored).

  • dst omitted, overwrite true (the default): the source file is replaced in place — but only after the new file has been fully written and verified lossless against it.

  • dst omitted, overwrite false: the output is written next to the source with a _recompressed suffix.

HDF4/HDF-EOS2 input is a format conversion: the output is written to dst (default: the source path with a .nc extension) and the source file is never replaced, regardless of overwrite. When geolocation is true (the default), HDF-EOS2 grid/swath coordinates are reconstructed additively per the geolocation design; geolocation=False converts the SDS payload only.

The new file is always written to a temporary path in the target’s directory, verified value-for-value against the source, and only then atomically moved onto the target. On any failure the source file is untouched and the temporary file is removed.

Parameters:
Return type:

str

Read-only audit

audit_path assesses an archive without modifying it: it discovers files, detects formats, inspects metadata (never reading science arrays), classifies each file into a stable status taxonomy, and returns an AuditReport. The per-file JSONL output is a versioned migration-manifest contract.

ncarnate.audit_path(path, options)[source]

Audits path (a file or an archive root) read-only and returns an AuditReport. Never opens science arrays, never touches the network, never writes to audited files.

Parameters:
Return type:

AuditReport

class ncarnate.AuditOptions(recursive=False, mode='metadata', checksum=None)[source]

How an audit run behaves: recursion, audit depth, and opt-in hashing.

Parameters:

  • recursive (bool)

  • mode (str)

  • checksum (str | None)

recursive: bool = False
mode: str = 'metadata'
checksum: str | None = None

Manifest-driven conversion

convert_manifest executes a migration manifest produced by audit_path (or ncarnate audit --output): for each record whose status is selected it re-verifies the recorded sha256 before touching the file, confines the source and output paths, and drives recompress() into a mirrored output tree. Per-record failures are isolated and tallied into a ConvertResult — the archive is never mutated unless in_place is set.

Because a manifest is untrusted input, the read containment base must be operator-controlled: set ConvertOptions.root (the CLI --root) to anchor source resolution to a directory you control, or allow_manifest_root (--allow-manifest-root) to explicitly trust the manifest’s own recorded root. With neither, convert_manifest refuses the run rather than trust an attacker-controllable base.

ncarnate.convert_manifest(manifest_path, options)[source]

Execute a migration manifest (design §The per-record loop). For each record whose status is selected, confine and re-verify the source, then drive ncarnate.core.recompress() into a mirrored out_dir tree. Per-record failures are isolated — one bad file never aborts the run — and tallied into the returned ConvertResult; a blocker is skipped with a counted reason and never converted (KD6). Sources are never mutated.

Parameters:
Return type:

ConvertResult

class ncarnate.ConvertOptions(out_dir=None, statuses=<factory>, allow_unverified=False, in_place=False, skip_existing=False, root=None, allow_manifest_root=False, zlib=True, shuffle=True, complevel=7, geolocation=True)[source]

How a manifest-driven convert run behaves.

out_dir is the mirrored output root (the non-destructive default, KD3); in_place opts into recompressing sources where they sit. statuses selects which audited statuses to act on and defaults to {"ready"} only (KD8) — the operator widens it after reading the report. allow_unverified relaxes the mandatory sha256 gate (KD2) for a null-hash record; skip_existing makes an out_dir run resumable; root is the operator-supplied containment base a source path resolves under (§Risks); allow_manifest_root opts into trusting the manifest’s own recorded root as that base when root is not given — untrusted by default, since the manifest is untrusted input. The encoding flags (zlib/shuffle/complevel/geolocation) share recompress’s defaults.

Parameters:
out_dir: str | None = None
statuses: set[str]
allow_unverified: bool = False
in_place: bool = False
skip_existing: bool = False
root: str | None = None
allow_manifest_root: bool = False
zlib: bool = True
shuffle: bool = True
complevel: int = 7
geolocation: bool = True

Format detection

class ncarnate.FileFormat(*values)[source]

The input container formats ncarnate distinguishes between.

NETCDF3 = 'netCDF3 (classic)'
HDF5 = 'HDF5 / netCDF4'
HDF4 = 'HDF4 / HDF-EOS2'
UNKNOWN = 'unknown'
ncarnate.detect_format(path)[source]

Identifies the container format of the file at path from its magic bytes.

Parameters:

path (str)

Return type:

FileFormat

Exceptions

All errors ncarnate raises deliberately derive from NcarnateError.

Defines the exception hierarchy for the ncarnate package.

Copyright (c) 2020-2026 Erick Edward Shepherd. MIT License — see the top-level LICENSE file.

exception ncarnate.errors.NcarnateError(*args, code=None)[source]

Bases: Exception

The base class for all errors raised deliberately by ncarnate.

Carries an optional structured code (a ncarnate.audit.codes registry string) so the audit can disambiguate raise sites that share one exception type — e.g. UnsupportedGeolocationError is both a packed-geolocation blocker and a name collision. The code is set at the raise site, never intrinsic to the type; code defaults to None and the message is unchanged.

Parameters:

code (str | None)

exception ncarnate.errors.UnsupportedFormatError(*args, code=None)[source]

Bases: NcarnateError

Raised when an input file is not a format ncarnate can read.

Parameters:

code (str | None)

exception ncarnate.errors.UnsupportedTypeError(*args, code=None)[source]

Bases: NcarnateError

Raised when a variable uses a netCDF4 user-defined type (compound, VLen, enum, or opaque) that is outside the v2 fidelity guarantee. ncarnate fails loud rather than guessing at a lossy copy.

Parameters:

code (str | None)

exception ncarnate.errors.EosParseError(*args, code=None)[source]

Bases: NcarnateError

Raised when a file’s StructMetadata text cannot be parsed as the ODL structure HDF-EOS2 defines.

Parameters:

code (str | None)

exception ncarnate.errors.UnsupportedProjectionError(*args, code=None)[source]

Bases: NcarnateError

Raised when an HDF-EOS2 grid uses a GCTP projection ncarnate has not verified against a fixture. A wrong coordinate is worse than a refused conversion; --no-geolocation converts the SDS payload without reconstruction.

Parameters:

code (str | None)

exception ncarnate.errors.UnsupportedGeolocationError(*args, code=None)[source]

Bases: NcarnateError

Raised when an HDF-EOS2 structure uses a geolocation construct outside the v2 scope (index dimension maps, merged fields, missing geolocation fields). --no-geolocation converts the SDS payload without reconstruction.

Parameters:

code (str | None)

exception ncarnate.errors.VerificationError(*args, code=None)[source]

Bases: NcarnateError

Raised when the post-write verification pass finds any difference between the source file and the recompressed copy. The source file is never replaced when this is raised.

Parameters:

code (str | None)

exception ncarnate.errors.AllocationTooLargeError(*args, code=None)[source]

Bases: NcarnateError

Raised when a file’s declared array (from untrusted metadata) would exceed the allocation safety ceiling — a tiny-file-declares-terabytes size bomb. A dedicated type lets the audit’s classifier map it to the unsafe status by type as well as by code.

Parameters:

code (str | None)