Troubleshooting

This page provides practical solutions to common calibration issues.

Reference Camera Choice

Problem: Stage 3/4 optimization converges but produces poor extrinsic estimates or high validation errors.

Why it happens: The reference camera (first camera in your config’s cameras list) defines the world coordinate origin. If the reference camera has poor intrinsic calibration, all other cameras inherit these errors during extrinsic initialization.

Solution:

  1. Check Stage 1 RMS reprojection errors (printed during aquacal calibrate)

  2. Set the camera with the lowest Stage 1 RMS as your reference camera

  3. Edit your config: move that camera to the first position in the cameras list

  4. Re-run calibration

Example:

Stage 1 complete.
  cam0: RMS = 0.42 pixels  ← good
  cam1: RMS = 1.89 pixels  ← poor intrinsics
  cam2: RMS = 0.38 pixels  ← best, use as reference

In this case, set cam2 as the reference camera (first in config).

High RMS in Stage 1 (Intrinsic Calibration)

Problem: Stage 1 reports RMS reprojection error > 1.0 pixels for one or more cameras.

Possible causes:

  • Too few calibration frames

  • Poor spatial coverage (board always in center of frame)

  • Board measurements in config don’t match physical board

  • Low-quality video (motion blur, compression artifacts)

Solutions:

1. Lower frame_step for more data

Edit your config:

detection:
  frame_step: 5  # Change to 2 or 1 for more frames

This uses more frames from your in-air videos, improving intrinsic estimates.

2. Verify board measurements

Measure your physical ChArUco board with calipers:

  • square_size: Distance between inner corners of adjacent squares (in meters)

  • marker_size: Black marker side length (in meters)

Even 1mm error can degrade calibration.

3. Check video quality

  • Ensure board is fully visible and in focus

  • Move board to different positions and orientations (not just center)

  • Avoid motion blur (hold board steady or use faster shutter speed)

4. Use max_calibration_frames to speed up iterations

If Stage 1 is good but Stage 3/4 are slow, limit optimization frames:

optimization:
  max_calibration_frames: 150  # Use subset for faster Stage 3/4

This speeds up iteration time while you debug other issues.

Bad Round-Trip Errors or Diverging Optimization

Problem: Stage 3 or 4 fails to converge, or validation errors are much higher than training errors.

Possible causes:

  • Disconnected pose graph (no overlapping camera views)

  • Too few shared observations between cameras

  • Initial water_z estimates are far from true value

  • Overfitted intrinsic model (rare, but see next section)

Solutions:

1. Check for pose graph connectivity

All cameras must observe the calibration board in at least some shared frames. If cameras have completely non-overlapping views, extrinsic initialization will fail.

Fix: Ensure underwater videos include frames where the board is visible to multiple cameras simultaneously.

2. Improve initial water_z estimates

Add initial_water_z (approximate camera-to-water Z-distances) to your config:

interface:
  initial_water_z:
    cam0: 0.20  # Measure with a ruler (meters)
    cam1: 0.22
    cam2: 0.18

Better initialization helps Stage 3 converge faster and more reliably.

3. Check water_z bounds

Ensure the final water_z value is within the optimization bounds [0.01, 2.0] meters. If your cameras are farther than 2m from the water surface, you’ll need to modify the bounds in the source code.

Camera Models and Overfitting

Problem: Stage 1 RMS is very low (< 0.2 pixels), but Stage 3/4 validation errors are high or undistortion produces artifacts.

Possible cause: Overfitted distortion model. With few calibration frames, higher-order distortion coefficients can fit noise rather than true lens distortion, causing the model to “blow up” outside the calibrated image region.

Camera models in AquaCal:

Model

Distortion Coeffs

When to Use

Auto-Simplification

Standard (pinhole)

5 params: k1, k2, p1, p2, k3

Most cameras, moderate distortion

✅ Yes (automatic)

Rational

8 params: k1-k6, p1, p2

Wide-angle lenses, extreme barrel/pincushion distortion

❌ No

Fisheye

4 params: k1-k4 (equidistant)

Ultra-wide or fisheye lenses (> 120° FOV)

❌ No

Auto-simplification (standard model only): If the full 5-parameter model produces a bad undistortion roundtrip (monotonicity check fails), AquaCal automatically retries with simpler models:

  1. Fix k3 to 0 (3-parameter model: k1, k2, p1, p2)

  2. Fix k3 and k2 to 0 (1-parameter model: k1, p1, p2)

This prevents overfitting when calibration data is sparse.

Solutions:

If using rational or fisheye models:

These models do NOT auto-simplify. If you suspect overfitting:

  1. Remove the camera from rational_model_cameras or fisheye_cameras in your config

  2. Let it use the standard 5-parameter model (with auto-simplification)

  3. Re-run calibration

Signs of overfitting:

  • Stage 1 RMS < 0.2 pixels but high round-trip error warnings in console

  • Distortion correction produces visible artifacts at image edges

  • Validation RMS >> training RMS in Stage 3/4

When to use each model:

  • Standard: Default for most lenses (up to moderate wide-angle)

  • Rational: Use only if standard model gives RMS > 1.0 pixels and you have 50+ calibration frames

  • Fisheye: Use only for true fisheye lenses (GoPro, ultra-wide security cameras)

For detailed theory, see the Camera Models section in the Optimizer Pipeline guide.

Allowed Camera Combinations

Problem: Config validation fails with ValueError about camera lists.

Rules:

  • fisheye_cameras must be a subset of auxiliary_cameras

  • fisheye_cameras must not overlap with rational_model_cameras

  • auxiliary_cameras must not overlap with cameras (primary cameras)

Example (valid):

cameras:
  - cam0  # Reference camera
  - cam1  # Primary camera

auxiliary_cameras:
  - cam2  # Auxiliary (registered post-hoc)
  - cam3  # Auxiliary (fisheye)

rational_model_cameras:
  - cam1  # Wide-angle primary camera

fisheye_cameras:
  - cam3  # Subset of auxiliary_cameras

Example (invalid):

cameras:
  - cam0

auxiliary_cameras:
  - cam1

fisheye_cameras:
  - cam0  # ERROR: cam0 is in cameras, not auxiliary_cameras

rational_model_cameras:
  - cam1  # ERROR: cam1 is in both rational and fisheye

fisheye_cameras:
  - cam1

Memory and Performance Issues

Problem: Calibration consumes too much RAM or takes too long.

Solution: Limit the number of frames used in Stage 3/4 optimization:

optimization:
  max_calibration_frames: 100  # Reduce from default (null = no limit)

  • Stage 1 (intrinsics) always uses all detected frames (controlled by frame_step)

  • Stage 2 (extrinsic init) uses all frames for pose graph construction

  • Stage 3/4 can be limited to a random subset for faster optimization

Reducing to 100-150 frames typically has minimal impact on calibration quality while significantly reducing memory usage and runtime.

No Detections Found

Problem: aquacal calibrate fails with “No ChArUco board detected in any frame”.

Possible causes:

  • Wrong board configuration (dictionary, dimensions)

  • Board out of focus or too small in frame

  • Poor lighting or motion blur

Solutions:

  1. Verify board config matches your physical board:

    • Check dictionary (e.g., DICT_4X4_50 vs. DICT_6X6_250)

    • Verify squares_x and squares_y match board layout

    • Confirm legacy_pattern setting (check if top-left cell has a marker)

  2. Improve video quality:

    • Ensure board fills at least 30% of frame

    • Check focus (board should be sharp)

    • Increase lighting to avoid motion blur

  3. Lower min_corners threshold:

    detection:
  min_corners: 8  # Lower to 4 or 6 if board is partially visible

See Also