Python API

The package root re-exports the supported API. Prefer from benchbro import ... unless you are extending an implementation module.

Definition API

benchbro.api.Case dataclass

Configure a group of benchmarks with shared measurement settings.

Attributes:

Name	Type	Description
name	str	Stable case name used in reports and baseline matching.
case_type	str	Free-form workload category, such as "cpu".
metric_type	MetricType	Measure elapsed time or Python allocations.
tags	list[str]	Labels used by command-line filtering.
warmup_iterations	int	Unmeasured invocations before sampling.
min_iterations	int	Invocations combined into each repeat sample.
repeats	int	Fixed repeat count or adaptive maximum.
comparison_metric	str | None	Metric used for baseline comparison. The metric-type default is used when this is None.
adaptive	bool	Stop sampling when the precision target is reached.
isolation	IsolationMode	Run in this process or a dedicated worker process.

input()

Register the callable that supplies input for this case's benchmarks.

Returns:

Type	Description
Callable[[F], F]	A decorator that preserves the decorated callable.

parametrize(name, values, *, ids=None)

Add named parameter values to a benchmark.

Multiple parameter decorators form a Cartesian product. The generated identifier is recorded in result artifacts and benchmark names.

Parameters:

Name	Type	Description	Default
name	str	Function parameter to inject.	required
values	Iterable[Any]	Values to register.	required
ids	Sequence[str] | None	Optional stable display identifier for each value.	None

Returns:

Type	Description
Callable[[F], F]	A decorator that records the parameter specification.

benchmark(name=None, comparison_metric=None, regression_threshold_pct=None, warning_threshold_pct=None, *, warmup_iterations=None, min_iterations=None, repeats=None, adaptive=None, timeout_s=None, isolation=None, setup_timing=None, teardown_timing=None, profiler=None, profile_output=None)

Register a Python callable as a benchmark.

Optional arguments override the corresponding case settings for this callable only. Synchronous and asynchronous callables are supported.

Returns:

Type	Description
Callable[[F], F]	A decorator that registers and preserves the callable.

subprocess(name=None, comparison_metric=None, regression_threshold_pct=None, warning_threshold_pct=None, *, warmup_iterations=None, min_iterations=None, repeats=None, adaptive=None, timeout_s=None, isolation=None, setup_timing=None, teardown_timing=None, profiler=None, profile_output=None)

Register a callable that returns a :class:CommandSpec.

Subprocess benchmarks support time metrics only. The command callable must be synchronous and returns an explicit argument vector rather than a shell command string.

Returns:

Type	Description
Callable[[F], F]	A decorator that registers and preserves the callable.

benchbro.api.system(*, scope='function')

Register an injectable dependency with a managed lifecycle.

Systems can be synchronous, asynchronous, generators, or async generators, and can depend on other systems by parameter name.

Parameters:

Name	Type	Description	Default
scope	SystemScope	"iteration", "benchmark", or "session". The legacy aliases "function" and "module" are also accepted.	'function'

Returns:

Type	Description
Callable[[F], F]	A decorator that registers and preserves the callable.

benchbro.models.CommandSpec dataclass

Describe one command invocation for a subprocess benchmark.

Attributes:

Name	Type	Description
argv	tuple[str, ...] | list[str]	Executable and arguments. Commands never pass through a shell.
cwd	str | Path | None	Optional child working directory.
env	dict[str, str] | None	Environment values merged into the child environment.
timeout_s	float | None	Optional hard timeout in seconds.
expected_exit_codes	tuple[int, ...] | list[int]	Exit codes considered successful.
stdin_bytes	bytes | None	Optional fixed standard input.
stdout	CommandStream	Inherit standard output or redirect it to the null device.
stderr	CommandStream	Inherit standard error or redirect it to the null device.

Execution API

benchbro.runner.run_cases(cases, repeats=None, warmup=None, min_iterations=None, *, adaptive=None, min_repeats=None, min_time_s=None, max_time_s=None, target_relative_margin_pct=None, noise_threshold_pct=None, isolation=None, cpu_affinity=None, stabilization_delay_s=None)

Execute registered cases and return a complete benchmark run.

Explicit function arguments override the settings stored on each case. Scoped systems and the shared async event loop are finalized even when execution fails.

Parameters:

Name	Type	Description	Default
cases	list[BenchmarkCase]	Concrete benchmark cases to execute in order.	required
repeats	int | None	Fixed repeat count or adaptive maximum.	None
warmup	int | None	Unmeasured invocations before sampling.	None
min_iterations	int | None	Invocations combined into each repeat sample.	None
adaptive	bool | None	Enable precision-aware early stopping.	None
min_repeats	int | None	Minimum samples collected in adaptive mode.	None
min_time_s	float | None	Minimum measurement duration.	None
max_time_s	float | None	Maximum measurement duration.	None
target_relative_margin_pct	float | None	Target relative margin for the 95% interval.	None
noise_threshold_pct	float | None	Coefficient-of-variation threshold for noisy results.	None
isolation	IsolationMode | None	Override in-process or worker-process execution.	None
cpu_affinity	tuple[int, ...] | None	Linux CPU identifiers to use during measurement.	None
stabilization_delay_s	float | None	Delay before benchmark measurement begins.	None

Returns:

Type	Description
BenchmarkRun	The environment metadata and result for every executed benchmark.

benchbro.api.list_cases()

Return all benchmark cases currently registered by decorators.

benchbro.api.clear_registry()

Remove all registered benchmarks and systems.

Comparison API

benchbro.comparison.compare_runs(baseline, current, *, allow_environment_mismatch=False, confidence_threshold_pct=95.0)

Compare matching results and classify their performance change.

Parameters:

Name	Type	Description	Default
baseline	BenchmarkRun	Reference run.	required
current	BenchmarkRun	Candidate run.	required
allow_environment_mismatch	bool	Compare even when runtime or machine fields differ.	False
confidence_threshold_pct	float	Minimum confidence required for a regression.	95.0

Returns:

Type	Description
list[Regression]	One classification for each current result that has a baseline match and
list[Regression]	a usable comparison metric.

Raises:

Type	Description
ValueError	If environments differ and the override is not enabled.

benchbro.comparison.comparison_environment_mismatches(baseline, current)

Return environment fields that make two runs unsafe to compare.

Result models

benchbro.models.BenchmarkSettings dataclass

Validated measurement, lifecycle, isolation, and profiling settings.

benchbro.models.BenchmarkResult dataclass

Measurements and effective metadata for one registered benchmark.

benchbro.models.BenchmarkRun dataclass

A complete schema-versioned collection of benchmark results.

benchbro.models.Regression dataclass

Classification produced by comparing one result with its baseline.

Serialization

benchbro.serialization.read_json(path)

Read a run, migrating supported older payloads to the current schema.

benchbro.serialization.write_json(path, run)

Write a complete schema-versioned run as JSON.

benchbro.serialization.write_csv(path, run)

Write one long-form CSV row per benchmark metric.

benchbro.serialization.write_markdown(path, run)

Write a human-readable Markdown result summary.

Profiling

benchbro.profiling.ProfilerHook

Bases: Protocol

Lifecycle implemented by built-in and user-registered profilers.

benchbro.profiling.register_profiler(name, factory)

Register a factory that creates a fresh profiler hook for each benchmark.