Calibration Pipeline¶

Note

For optimizer implementation details, see Optimizer Pipeline.

The calibration package implements a four-stage pipeline for refractive multi-camera calibration.

Pipeline¶

End-to-end calibration pipeline orchestration.

aquacal.calibration.pipeline.calibrate_from_detections(detections, intrinsics, board, *, reference_camera=None, n_air=1.0, n_water=1.333, loss='huber', loss_scale=1.0, min_corners=4, verbose=0)[source]¶

Run Stages 2-3 on pre-computed detections and return a CalibrationResult.

This is a high-level convenience function that takes detections and intrinsics (typically from synthetic data or a previous detection step) and runs the extrinsic initialization (Stage 2) and joint refractive optimization (Stage 3).

Parameters:

detections (DetectionResult) – Detected corners across all cameras and frames.
intrinsics (dict[str, CameraIntrinsics]) – Per-camera intrinsic parameters.
board (BoardGeometry) – Board geometry used for detection.
reference_camera (str | None) – Name of the reference camera (identity extrinsics). Defaults to the first camera in sorted order.
n_air (float) – Refractive index of air.
n_water (float) – Refractive index of water.
loss (str) – Robust loss function for Stage 3 (‘huber’, ‘cauchy’, etc.).
loss_scale (float) – Scale parameter for the robust loss.
min_corners (int) – Minimum corners per detection to use.
verbose (int) – Verbosity level (0=silent, 1=summary, 2=per-iteration).

Returns:

Tuple of (CalibrationResult, board_poses) where board_poses maps frame index to the optimized BoardPose.

Return type:

tuple[CalibrationResult, dict[int, BoardPose]]

Example

>>> from aquacal.datasets import create_scenario, generate_synthetic_detections
>>> from aquacal.calibration import calibrate_from_detections
>>> scenario = create_scenario("minimal")
>>> board = BoardGeometry(scenario.board_config)
>>> detections = generate_synthetic_detections(
...     scenario.intrinsics, scenario.extrinsics, scenario.water_zs,
...     board, scenario.board_poses, noise_std=scenario.noise_std,
... )
>>> result, poses = calibrate_from_detections(
...     detections, scenario.intrinsics, board,
... )
>>> print(f"RMS: {result.diagnostics.reprojection_error_rms:.3f} px")

aquacal.calibration.pipeline.load_config(config_path)[source]¶

Load calibration configuration from YAML file.

Parameters:

config_path (str | Path) – Path to config.yaml file

Returns:

CalibrationConfig populated from file

Raises:

FileNotFoundError – If config file doesn’t exist
ValueError – If config is invalid or missing required fields

Return type:

CalibrationConfig

aquacal.calibration.pipeline.split_detections(detections, holdout_fraction, seed=42)[source]¶

Split detections into calibration and validation sets.

Randomly assigns entire frames to either set (not individual detections).

Parameters:

detections (DetectionResult) – Full detection result
holdout_fraction (float) – Fraction of frames for validation (0.0 to 1.0)
seed (int) – Random seed for reproducibility

Returns:

Tuple of (calibration_detections, validation_detections)

Return type:

tuple[DetectionResult, DetectionResult]

aquacal.calibration.pipeline.run_calibration(config_path, verbose=False)[source]¶

Run complete calibration pipeline from config file.

Loads configuration from YAML and delegates to run_calibration_from_config().

Parameters:

config_path (str | Path) – Path to config.yaml file
verbose (bool) – If True, enable per-iteration progress output from optimizers

Returns:

Complete CalibrationResult

Raises:

FileNotFoundError – If config or video files not found
CalibrationError – If any calibration stage fails

Return type:

CalibrationResult

Example

>>> from aquacal import run_calibration
>>> result = run_calibration("config.yaml", verbose=True)
>>> print(f"Calibrated {len(result.cameras)} cameras")
>>> print(f"Water surface at Z = {result.cameras['cam0'].water_z:.3f} m")

Note

For details on the optimizer pipeline, see the Optimizer Guide guide.

aquacal.calibration.pipeline.run_calibration_from_config(config, verbose=False)[source]¶

Run complete calibration pipeline from configuration object.

Pipeline stages: 1. Detect ChArUco in intrinsic (in-air) videos 2. Run Stage 1: Intrinsic calibration 3. Detect ChArUco in extrinsic (underwater) videos 4. Split underwater detections into calibration/validation sets 5. Run Stage 2: Extrinsic initialization 6. Run Stage 3: Interface and pose optimization 7. Optionally run Stage 4: Joint refinement 8. Run validation on held-out data 9. Generate and save diagnostics 10. Save final calibration result

Parameters:

config (CalibrationConfig) – Complete calibration configuration
verbose (bool) – If True, enable per-iteration progress output from optimizers

Returns:

CalibrationResult with all calibrations and diagnostics

Raises:

CalibrationError – If any stage fails
InsufficientDataError – If not enough detections
ConnectivityError – If pose graph is disconnected

Return type:

CalibrationResult

Stage 1: Intrinsics¶

Stage 1: Per-camera intrinsic calibration.

aquacal.calibration.intrinsics.validate_intrinsics(intrinsics, camera_name='', max_roundtrip_error_px=1.0, expected_fx=None, fx_tolerance_fraction=0.3)[source]¶

Validate intrinsic calibration quality with post-calibration sanity checks.

Detects bad intrinsic calibrations (broken undistortion, unstable distortion model, implausible focal length) before they cascade into wrong extrinsics.

Parameters:

intrinsics (CameraIntrinsics) – The calibration result to validate
camera_name (str) – For warning messages (default “”)
max_roundtrip_error_px (float) – Maximum acceptable undistortion roundtrip error (default 1.0 px)
expected_fx (float | None) – If provided, check that calibrated fx is within tolerance of this value (default None = skip check)
fx_tolerance_fraction (float) – Fractional tolerance for fx check (default 0.3 = 30%)

Returns:

List of warning strings (empty = all checks passed)

Return type:

list[str]

Example

>>> warnings = validate_intrinsics(intrinsics, camera_name="cam0")
>>> for w in warnings:
...     print(f"WARNING: {w}")

aquacal.calibration.intrinsics.calibrate_intrinsics_single(video_path, board, max_frames=100, min_corners=8, frame_step=1, rational_model=False, fisheye=False)[source]¶

Calibrate intrinsics for a single camera from in-air video.

Detects ChArUco corners in video frames, selects a subset with good spatial coverage, and runs OpenCV camera calibration.

Parameters:

video_path (str | Path) – Path to calibration video (in-air, no refraction)
board (BoardGeometry) – Board geometry
max_frames (int) – Maximum number of frames to use for calibration (default 100)
min_corners (int) – Minimum corners required per frame (default 8)
frame_step (int) – Process every Nth frame from video (default 1)
rational_model (bool) – If True, use 8-coefficient rational distortion model instead of the standard 5-coefficient model. Use for wide-angle lenses.
fisheye (bool) – If True, use equidistant fisheye calibration (cv2.fisheye.calibrate). Returns 4 distortion coefficients (k1-k4). Incompatible with rational_model.

Returns:

CameraIntrinsics with K and dist_coeffs (5, 8, or 4 coefficients), and image_size
RMS reprojection error in pixels

Return type:

Tuple of (CameraIntrinsics, reprojection_error_rms)

Raises:

ValueError – If no valid frames found or calibration fails

Example

>>> board = BoardGeometry(config)
>>> intrinsics, error = calibrate_intrinsics_single("cam0_inair.mp4", board)
>>> print(f"Reprojection error: {error:.3f} pixels")

aquacal.calibration.intrinsics.calibrate_intrinsics_all(video_paths, board, max_frames=100, min_corners=8, frame_step=1, rational_model_cameras=None, fisheye_cameras=None, progress_callback=None)[source]¶

Calibrate intrinsics for multiple cameras.

Parameters:

video_paths (dict[str, str | Path]) – Dict mapping camera_name to video file path
board (BoardGeometry) – Board geometry (same for all cameras)
max_frames (int) – Maximum frames per camera (default 100)
min_corners (int) – Minimum corners per frame (default 8)
frame_step (int) – Process every Nth frame (default 1)
rational_model_cameras (list[str] | None) – List of camera names that should use the 8-coefficient rational distortion model (default None = all use 5-coeff)
fisheye_cameras (list[str] | None) – List of camera names that should use the equidistant fisheye model (default None = none use fisheye)
progress_callback (Callable[[str, int, int], None] | None) – Optional callback(camera_name, current_cam, total_cams)

Returns:

Dict mapping camera_name to (CameraIntrinsics, reprojection_error_rms)

Raises:

ValueError – If calibration fails for any camera

Return type:

dict[str, tuple[CameraIntrinsics, float]]

Example

>>> paths = {'cam0': 'cam0.mp4', 'cam1': 'cam1.mp4'}
>>> board = BoardGeometry(config)
>>> results = calibrate_intrinsics_all(paths, board)
>>> for name, (intrinsics, error) in results.items():
...     print(f"{name}: error={error:.3f}px")

Stage 2: Extrinsics¶

Stage 2: Extrinsic initialization via pose graph.

class aquacal.calibration.extrinsics.Observation(camera, frame_idx, corner_ids, corners_2d)[source]¶

Bases: object

A single camera’s observation of the board in one frame.

Parameters:

camera (str)
frame_idx (int)
corner_ids (ndarray[tuple[int, ...], dtype[int32]])
corners_2d (ndarray[tuple[int, ...], dtype[float64]])

class aquacal.calibration.extrinsics.PoseGraph(camera_names, frame_indices, observations, adjacency=<factory>)[source]¶

Bases: object

Graph connecting cameras through shared board observations.

Cameras connect indirectly: if cameras A and B both see the board in frame F, they are linked through that frame’s board pose node.

Parameters:

camera_names (list[str])
frame_indices (list[int])
observations (list[Observation])
adjacency (dict[str, set[str]])

camera_names¶

List of all camera names in the graph

Type:: list[str]

frame_indices¶

List of frame indices with 2+ camera observations

Type:: list[int]

observations¶

All camera-board observations

Type:: list[aquacal.calibration.extrinsics.Observation]

adjacency¶

Dict mapping each node to its neighbors for connectivity analysis. Node names: camera names (str) and frame indices prefixed with “f” (e.g., “f42”)

Type:: dict[str, set[str]]

aquacal.calibration.extrinsics.estimate_board_pose(intrinsics, corners_2d, corner_ids, board)[source]¶

Estimate board pose relative to camera using PnP.

Parameters:

intrinsics (CameraIntrinsics) – Camera intrinsic parameters
corners_2d (ndarray[tuple[int, ...], dtype[float64]]) – Detected corner positions, shape (N, 2)
corner_ids (ndarray[tuple[int, ...], dtype[int32]]) – Corner IDs corresponding to corners_2d, shape (N,)
board (BoardGeometry) – Board geometry for 3D corner positions

Returns:

Tuple of (rvec, tvec) representing board pose in camera frame, or None if PnP fails (e.g., too few points). - rvec: Rodrigues rotation vector, shape (3,) - tvec: Translation vector, shape (3,)

Return type:

tuple[ndarray[tuple[int, …], dtype[float64]], ndarray[tuple[int, …], dtype[float64]]] | None

Example

>>> result = estimate_board_pose(intrinsics, corners, ids, board)
>>> if result is not None:
...     rvec, tvec = result

aquacal.calibration.extrinsics.refractive_solve_pnp(intrinsics, corners_2d, corner_ids, board, water_z, interface_normal=None, n_air=1.0, n_water=1.333)[source]¶

Estimate board pose with refractive correction using LM refinement.

Uses standard solvePnP as initial guess, applies rough depth correction, then refines by minimizing refractive reprojection error.

The key trick: use an identity-extrinsics camera so camera frame = world frame. Board corners transformed by the candidate pose become “world points” that get projected through the refractive interface.

Parameters:

intrinsics (CameraIntrinsics) – Camera intrinsic parameters
corners_2d (ndarray[tuple[int, ...], dtype[float64]]) – Detected corner positions, shape (N, 2)
corner_ids (ndarray[tuple[int, ...], dtype[int32]]) – Corner IDs corresponding to corners_2d, shape (N,)
board (BoardGeometry) – Board geometry for 3D corner positions
water_z (float) – Distance from camera to water surface in meters
interface_normal (ndarray[tuple[int, ...], dtype[float64]] | None) – Interface normal vector. If None, uses [0, 0, -1].
n_air (float) – Refractive index of air (default 1.0)
n_water (float) – Refractive index of water (default 1.333)

Returns:

Tuple of (rvec, tvec) representing board pose in camera frame, or None if PnP fails.

Return type:

tuple[ndarray[tuple[int, …], dtype[float64]], ndarray[tuple[int, …], dtype[float64]]] | None

aquacal.calibration.extrinsics.build_pose_graph(detections, min_cameras=2)[source]¶

Build pose graph from detection results.

Creates a bipartite graph where cameras and frames are nodes, connected by observation edges. Only includes frames where at least min_cameras see the board.

Parameters:

detections (DetectionResult) – Detection results from detect_all_frames
min_cameras (int) – Minimum cameras per frame to include (default 2)

Returns:

PoseGraph with adjacency structure for connectivity analysis

Raises:

ConnectivityError – If the graph is not connected (some cameras cannot be linked to others through shared observations). Error message includes details about disconnected components.

Return type:

PoseGraph

Example

>>> detections = detect_all_frames(videos, board)
>>> pose_graph = build_pose_graph(detections, min_cameras=2)
>>> print(f"Graph has {len(pose_graph.frame_indices)} usable frames")

aquacal.calibration.extrinsics.estimate_extrinsics(pose_graph, intrinsics, board, reference_camera=None, water_zs=None, interface_normal=None, n_air=1.0, n_water=1.333, progress_callback=None)[source]¶

Estimate camera extrinsics by chaining poses through the graph.

Uses BFS traversal from reference camera, computing each camera’s pose relative to world frame (centered at reference camera).

The algorithm fixes the reference camera at world origin (R=I, t=0), then runs BFS through the pose graph. When visiting a frame node from a known camera, it computes the board pose via PnP. When visiting a camera node from a known frame, it computes the camera pose via PnP and inversion.

Parameters:

pose_graph (PoseGraph) – Pose graph from build_pose_graph
intrinsics (dict[str, CameraIntrinsics]) – Dict mapping camera names to intrinsics
board (BoardGeometry) – Board geometry
reference_camera (str | None) – Camera to place at world origin. If None, uses first camera name (sorted).
water_zs (dict[str, float] | None) – Optional dict mapping camera names to interface distances in meters. When provided, uses refractive PnP for cameras with known distances.
interface_normal (ndarray[tuple[int, ...], dtype[float64]] | None) – Interface normal vector. If None, uses [0, 0, -1].
n_air (float) – Refractive index of air (default 1.0)
n_water (float) – Refractive index of water (default 1.333)
progress_callback (Callable[[str, int, int], None] | None) – Optional callback(camera_name, cameras_located, total_cameras) called after each camera is located during BFS traversal

Returns:

Dict mapping camera names to CameraExtrinsics. Reference camera has R=I, t=0.

Raises:

ValueError – If reference_camera not in pose_graph
ValueError – If intrinsics missing for any camera in graph

Return type:

dict[str, CameraExtrinsics]

Example

>>> extrinsics = estimate_extrinsics(pose_graph, intrinsics, board)
>>> cam0_pose = extrinsics['cam0']
>>> print(f"cam0 at world origin: {np.allclose(cam0_pose.t, 0)}")

Stage 3: Interface Estimation¶

Stage 3 refractive optimization for interface estimation.

This module implements joint optimization of camera extrinsics, per-camera interface distances, and board poses using underwater ChArUco detections.

aquacal.calibration.interface_estimation.optimize_interface(detections, intrinsics, initial_extrinsics, board, reference_camera, initial_water_zs=None, interface_normal=None, n_air=1.0, n_water=1.333, loss='huber', loss_scale=1.0, min_corners=4, use_sparse_jacobian=True, verbose=1, normal_fixed=True)[source]¶

Jointly optimize camera extrinsics, interface distances, and board poses.

This is Stage 3 of the calibration pipeline. It refines the initial estimates from Stage 2 by accounting for refraction at the air-water interface.

Internally, a single global water_z parameter replaces N per-camera interface distances. Per-camera distances are derived as d_i = water_z - C_z_i, where C_z_i is the camera center’s Z coordinate. This eliminates the degeneracy between camera height and interface distance by construction.

Note

For details on the optimizer implementation and sparse Jacobian structure, see the Optimizer Guide guide.

Parameters:

detections (DetectionResult) – Underwater ChArUco detections from detect_all_frames
intrinsics (dict[str, CameraIntrinsics]) – Per-camera intrinsic parameters (fixed during optimization)
initial_extrinsics (dict[str, CameraExtrinsics]) – Initial camera extrinsics from Stage 2
board (BoardGeometry) – ChArUco board geometry
reference_camera (str) – Camera name to fix at origin (extrinsics not optimized)
initial_water_zs (dict[str, float] | None) – Optional initial distances per camera. If None, defaults to 0.15m for all cameras.
interface_normal (ndarray[tuple[int, ...], dtype[float64]] | None) – Interface normal vector. If None, uses [0, 0, -1]. Normal is fixed during optimization.
n_air (float) – Refractive index of air (default 1.0)
n_water (float) – Refractive index of water (default 1.333)
loss (str) – Robust loss function (“linear”, “huber”, “soft_l1”, “cauchy”)
loss_scale (float) – Scale parameter for robust loss in pixels
min_corners (int) – Minimum corners per detection to include in optimization
use_sparse_jacobian (bool) – Use sparse Jacobian structure (default True). Dramatically improves performance for large camera arrays.
verbose (int) – Verbosity level for scipy.optimize.least_squares (default 1). 0 = silent, 1 = one-line per iteration, 2 = full per-iteration report.
normal_fixed (bool) – If False, estimate reference camera tilt (2 DOF) to account for non-perpendicular camera-to-water-surface alignment.

Returns:

dict[str, CameraExtrinsics]: Optimized extrinsics for all cameras
dict[str, float]: Optimized interface distances per camera (derived from water_z)
list[BoardPose]: Optimized board poses for each frame used
float: Final RMS reprojection error in pixels

Return type:

Tuple of

Raises:

InsufficientDataError – If no valid frames for optimization
ConvergenceError – If optimization fails to converge
ValueError – If reference_camera not in initial_extrinsics

aquacal.calibration.interface_estimation.register_auxiliary_camera(camera_name, intrinsics, detections, board_poses, board, water_z, interface_normal=None, n_air=1.0, n_water=1.333, min_corners=4, refine_intrinsics=False, verbose=0)[source]¶

Estimates the camera’s extrinsics by minimizing refractive reprojection error against known board poses from Stage 3. The interface distance is derived from the known global water_z and the camera’s Z position: d = water_z - C_z. This is a 6-parameter problem (extrinsics only) with no degeneracy. Optionally refines intrinsics (fx, fy, cx, cy) for a 10-parameter optimization.

Parameters:

camera_name (str) – Name of the auxiliary camera
intrinsics (CameraIntrinsics) – Camera intrinsic parameters (initial values, refined if refine_intrinsics=True)
detections (DetectionResult) – Full detection results (must contain this camera’s detections)
board_poses (dict[int, BoardPose]) – Fixed board poses from Stage 3 (frame_idx -> BoardPose)
board (BoardGeometry) – Board geometry
water_z (float) – Global water surface Z from Stage 3 (required)
interface_normal (ndarray[tuple[int, ...], dtype[float64]] | None) – Interface normal (default [0, 0, -1])
n_air (float) – Refractive index of air
n_water (float) – Refractive index of water
min_corners (int) – Minimum corners per detection
refine_intrinsics (bool) – If True, also optimize fx, fy, cx, cy (10 params total). Distortion coefficients are kept fixed.
verbose (int) – Verbosity level

Returns:

Tuple of (extrinsics, water_z, rms_error) When refine_intrinsics=True: Tuple of (extrinsics, water_z, rms_error, refined_intrinsics)

Return type:

When refine_intrinsics=False

Raises:

InsufficientDataError – If no usable frames found

Calibration Pipeline¶

Pipeline¶

Stage 1: Intrinsics¶

Stage 2: Extrinsics¶

Stage 3: Interface Estimation¶

Stage 4: Refinement¶