Chapter 7: Vision Sensors | EngineAI Open Platform

1 PM01-D435i Depth Camera

This section describes how to launch the PM01 built-in Intel RealSense D435i depth camera node and obtain camera data.

View Depth Camera Device:

Bash

rs-enumerate-devices -s

Example output:

Bash

Device Name                   Serial Number       Firmware Version
Intel RealSense D435I         243322074409        5.15.1.55
Intel RealSense D435I         347622075631        5.15.1.55

Launch Depth Camera Node: EngineAI officially provides camera and LiDAR integration examples based on the ROS2 ecosystem (open-source repository engineai_ros2_workspace); this section includes a built-in example program rs_launch.py for the depth camera.

Bash

ros2 launch realsense2_camera rs_launch.py \
  serial_no:="243322074409" \
  camera_name:=cam_head \
  tf_prefix:=cam_head

Parameter Description:

Parameter	Function	Description
`serial_no`	Bind to a specific camera	Must be specified; wrap with outer double quotes + inner single quotes
`camera_name`	Topic namespace	Avoid topic conflicts among multiple cameras; e.g., `cam_head` generates `/camera/cam_head/xxx`
`tf_prefix`	TF coordinate frame prefix	e.g., `cam_head` generates `cam_head_link`, `cam_head_depth_frame`, etc.

Available Topics:

PLAIN

/camera/cam_head/color/camera_info
/camera/cam_head/color/image_raw
/camera/cam_head/color/metadata
/camera/cam_head/depth/camera_info
/camera/cam_head/depth/image_rect_raw
/camera/cam_head/depth/metadata
/camera/cam_head/extrinsics/depth_to_color

2 T800-Odin2 LiDAR Camera

The T800 series is equipped with the Odin2 LiDAR camera, connected to the Orin mainboard. The Odin2 factory-default IP is 192.168.100.251, and the Orin mainboard IP is 192.168.0.162.

Connect to Orin Mainboard:

Bash

ssh -X username@192.168.0.162

System Requirements:

Ubuntu Version	ROS Version
20.04	ROS1 Noetic, ROS2 Foxy
22.04	ROS2 Humble, ROS2 Iron
24.04	ROS2 Jazzy

Build and Run (ROS2 Humble):

Bash

source /opt/ros/humble/setup.bash
cd your_ros_workspace
./src/odin_ros_driver/script/build_ros2.sh

# Run
source install/setup.bash
ros2 launch odin_ros_driver_rev1 driver.launch.py

# Specify resolution and frame rate
ros2 launch odin_ros_driver_rev1 driver.launch.py image_width:=640 image_height:=544 image_fps:=30

# Do not launch RViz
ros2 launch odin_ros_driver_rev1 driver.launch.py start_rviz:=false

Functional Modules:

Node	Description
`odin_ros_driver_node`	Main driver node: connects to the device, publishes raw data, and provides calibration service
`post_process_node`	Post-processing node: image undistortion, point cloud coloring, stereo rectification
`pointcloud_reprojector_node`	Point cloud reprojection node: sparse point cloud reprojection outputs overlaid visualization image
`pointcloud_to_depth_node`	Point cloud to depth image node: projection + dilation + Sobel edge suppression + nearest-neighbor upsampling

Main Topics Published by the Main Driver (all with per-device prefix /{topic_prefix}/{model}/device{N}/):

Topic Suffix	Type	Description
`cloud/raw`	`sensor_msgs/PointCloud2`	Raw point cloud (256×192)
`cloud/slam`	`sensor_msgs/PointCloud2`	SLAM point cloud
`camera0/compressed`	`sensor_msgs/CompressedImage`	Left camera JPEG compressed image
`camera1/compressed`	`sensor_msgs/CompressedImage`	Right camera JPEG compressed image
`imu`	`sensor_msgs/Imu`	IMU data (acceleration, angular velocity)
`odometry`	`nav_msgs/Odometry`	Odometry data

Common Launch Parameters:

Parameter	Default Value	Description
`start_rviz`	`true`	Whether to launch RViz
`topic_prefix`	`manifold`	Topic prefix root
`image_width`	`0`	Image width, 0 = use device default
`image_height`	`0`	Image height, 0 = use device default
`image_fps`	`0`	Image frame rate, 0 = use device default
`image_format`	`mjpeg`	Image format: mjpeg / yuyv / nv12 / nv21 / rgb24
`sensor_mode`	`0`	dToF sensor mode: 0=HDR / 1=High Peak / 2=Low Peak / 3=Adaptive
`confidence_threshold`	`7`	Point cloud confidence threshold (0~255)