Skip to content

climval check

The climval check validation gate.

A configuration file declares checks (reference, candidate, variable, alignment options and thresholds). :func:run_file runs them and returns a :class:GateReport with an exit code: 0 when every check passes, 1 when a threshold fails, 2 when the configuration or the data could not be used.

from climval.gate import run_file report = run_file("climval.yml") report.exit_code 0

load_config

load_config(path: str | PathLike[str]) -> GateConfig

Read and validate a configuration file.

Relative data paths in the file are read relative to the file's directory.

Raises:

Type Description
ConfigError

The file is missing, cannot be parsed, or fails validation.

parse_config

parse_config(data: Any, base_dir: str | PathLike[str] | None = None) -> GateConfig

Validate an already parsed configuration (a dict) and return a GateConfig.

run_file

run_file(path: str | Path, only: list[str] | None = None, verbose: bool = False) -> GateReport

Load a configuration file and run it. A configuration error gives exit code 2.

run

Run the checks of a :class:~climval.gate.config.GateConfig.

Each check runs :func:climval.compare once per variable and evaluates its thresholds with :func:climval.evaluate_thresholds. A check that cannot run (missing file, failed catalogue query, alignment error) is recorded as an error and the remaining checks still run.

Exit codes: 0 when every check passes, 1 when a threshold fails and nothing else went wrong, 2 when the configuration is invalid or any check could not run.

CheckOutcome dataclass

CheckOutcome(check: str, variable: str, status: str, reference: str, candidate: str, description: str | None = None, comparison: Comparison | None = None, thresholds: ThresholdResult | None = None, error: str | None = None, error_type: str | None = None, detail: str | None = None, seconds: float = 0.0, inputs: dict[str, Any] = dict())

The result of one check for one variable.

GateReport dataclass

GateReport(outcomes: list[CheckOutcome], config_path: str | None = None, config_sha256: str | None = None, created: str = timestamp(), config_error: str | None = None)

Everything a climval check run produced.

open_input

open_input(spec: InputSpec, check_kwargs: Mapping[str, Any], catalogs: _Catalogs) -> tuple[xr.Dataset, dict[str, Any]]

Open one side of a check lazily; returns the Dataset and a description.

run_check

run_check(check: CheckSpec, catalogs: _Catalogs | None = None, verbose: bool = False) -> list[CheckOutcome]

Run one check: one outcome per variable.

run

run(config: GateConfig, verbose: bool = False, progress: Callable[[str], None] | None = None) -> GateReport

Run every check of config and return a :class:GateReport.

run_file

run_file(path: str | Path, only: list[str] | None = None, verbose: bool = False) -> GateReport

Load a configuration file and run it. A configuration error gives exit code 2.

GateReport dataclass

GateReport(outcomes: list[CheckOutcome], config_path: str | None = None, config_sha256: str | None = None, created: str = timestamp(), config_error: str | None = None)

Everything a climval check run produced.

CheckOutcome dataclass

CheckOutcome(check: str, variable: str, status: str, reference: str, candidate: str, description: str | None = None, comparison: Comparison | None = None, thresholds: ThresholdResult | None = None, error: str | None = None, error_type: str | None = None, detail: str | None = None, seconds: float = 0.0, inputs: dict[str, Any] = dict())

The result of one check for one variable.

GateConfig dataclass

GateConfig(path: str | None, sha256: str | None, checks: list[CheckSpec])

A parsed configuration file.

select

select(names: Sequence[str]) -> GateConfig

Keep only the named checks; unknown names are a configuration error.

CheckSpec dataclass

CheckSpec(name: str, reference: InputSpec, candidate: InputSpec, variables: list[str | tuple[str, str] | None], options: dict[str, Any], thresholds: list[dict[str, Any]], description: str | None = None)

One check: a comparison per variable and the thresholds that gate it.

threshold_mapping

threshold_mapping(item: dict[str, Any]) -> dict[str, dict[str, Any]]

One threshold in the form :func:climval.evaluate_thresholds takes.

InputSpec dataclass

InputSpec(path: str | None = None, catalog: str | None = None, query: dict[str, Any] = dict(), sel: dict[str, Any] = dict(), open_kwargs: dict[str, Any] = dict(), name: str | None = None, given: str | None = None)

Where one side of a check comes from: a path or URL, or a catalogue query.