D415 camera: pixel-accurate color and depth rendering with full intrinsics

[siddhss5]

Context

ada_assets provides a MuJoCo camera (camera/d415_color, camera/d415_depth) at the calibrated optical frame on the wrist. MuJoCo's camera model only supports fovy — it assumes centered principal point and square pixels. The real D415 has off-center cx/cy and slightly non-square fx/fy.

D415 specs (from ada_assets camera.xml)

Nominal intrinsics at 1280x720 (aligned_depth_to_color):

• fx=913.16, fy=911.31, cx=635.49, cy=372.79
• Distortion: Brown-Conrady (coefficients from RealSense driver at runtime)
• Depth range: 0.16–10.0 m
• Color FOV: 69.4° x 42.5°, Depth FOV: 65° x 40°

MuJoCo fovy=43° approximates fy=911 but ignores cx/cy offset and fx≠fy.

What needs to happen

Color rendering

1. Render with MuJoCo's camera/d415_color (fovy=43, 1280x720)
2. Optionally reproject with the full intrinsic matrix to correct for off-center principal point — the error is ~5px at 1280 wide (cx=635 vs 640 center), which may be acceptable

Depth rendering

1. Render with camera/d415_depth using renderer.enable_depth_rendering(True)
2. Clip to [0.16, 10.0] m to match D415 operating range
3. Optionally add depth noise model (D415 depth noise increases quadratically with distance)

Camera info publisher (for ROS bridge)

• Publish CameraInfo with the intrinsic matrix K, distortion D, and resolution
• Match the topic structure: /local/camera/color/camera_info, /local/camera/aligned_depth_to_color/camera_info

Intrinsics source

• Nominal values are documented in camera.xml
• On the real robot, intrinsics come from the RealSense driver at runtime
• For sim, use the nominal values as defaults, allow override from calibration file

References

• ada_assets/models/camera.xml — camera component with D415 specs and sites
• ada_ros2/ada_calibrate_camera/ — hand-eye calibration node, subscribes to CameraInfo
• Intel D415 datasheet