English | 日本語

ROS Hardware-in-the-Loop Simulation System

Affordable Hardware-in-the-Loop Simulation (HILS) for ROS 2 robots, using inexpensive microcontrollers and off-the-shelf USB devices to test the full communication stack including physical interfaces.

UVC camera verification (streaming simulation video from Isaac Sim)

Livox MID360 verification (transmitting ros2 bag data via the native Livox SDK2 protocol)

RC servo PWM capture verification (measuring the PWM signal from an Arduino controller and publishing it as JointState)

Overview

Software simulators (Gazebo / Unity / Isaac Sim) publish directly to ROS topics, which means the actual sensor drivers and physical communication paths (UDP, UART, USB, I2C, etc.) are never tested. This project bridges that gap by converting simulator outputs into device-native protocols, so the real-hardware driver sees what it would see from a real sensor -- all for under $30 in parts.

Repository Structure

ros_hardware_in_the_loop_system/
├── ros2_hils_bridge/                     # ROS 2 packages (git submodule)
├── firmware/                             # RP2040 / ESP32 firmware
│   ├── common/                           #   Shared headers (HILS frame protocol)
│   ├── rp2040_camera_uvc/                #   UVC camera (Pico#2)
│   ├── rp2040_camera_uvc_spi_sender/     #   USB-CDC -> SPI relay (Pico#1)
│   ├── rp2040_actuator_servo_pwm/        #   RC servo PWM capture (controller-side evaluation)
│   ├── rp2040_encoder_quadrature/        #   Quadrature encoder A/B output
│   ├── rp2040_imu_invensense_mpu6050/    #   I2C slave MPU-6050 register map
│   ├── rp2040_ethernet_bridge/           #   Ethernet bridge (reference)
│   └── esp32_can_bridge/                 #   CAN bridge (reference)
├── docs/                                 # Architecture, BOM, verification guides
└── hardware/                             # Schematics, enclosures (planned)

Submodule: ros2_hils_bridge

The ROS 2 packages live in ros2_hils_bridge/ and can be cloned directly into colcon_ws/src. See ros2_hils_bridge/README.md for details.

Naming Convention

To keep the project consistent as new emulators are added, packages and firmware follow a two-level naming pattern:

ros2_hils_bridge/hils_bridge_<sensor_type>/hils_bridge_<sensor_type>_<protocol_or_vendor_series>/
firmware/rp2040_<sensor_type>_<protocol_or_vendor_series>/

• <sensor_type> — physical/functional category: lidar, camera, gps, imu, actuator, encoder, can
• <protocol_or_vendor_series> — chosen by the following rule:
  • Industry-standard protocol (UVC, NMEA0183, PWM, quadrature, …): use the protocol name itself, signaling that any vendor's device with that protocol works (e.g. hils_bridge_camera_uvc, hils_bridge_gps_nmea0183)
  • Vendor-specific protocol: use <vendor>_<series> so that the package's actual scope is unambiguous (e.g. hils_bridge_lidar_livox_mid360, hils_bridge_imu_witmotion_wt901, hils_bridge_imu_invensense_mpu6050). Even within one vendor, generations or series often break compatibility, so always include the series

When adding a new emulator, decide the sensor type, then pick the standard protocol name if the implementation truly works across vendors, otherwise the <vendor>_<series> form. Keep the ROS package name and the corresponding firmware directory in lockstep.

Implementation Status

Track A: PC-based developers (no microcontroller needed)

Device	Method	ROS Package	Firmware	Status
Livox Mid-360	USB-LAN + SW	hils_bridge_lidar_livox_mid360	N/A	Implemented, verified (incl. fault-injection regression vs livox_ros_driver2)
Velodyne VLP-16	USB-LAN + SW	hils_bridge_lidar_velodyne_vlp16	N/A	Implemented, verified
Ouster OS1	USB-LAN + SW	hils_bridge_lidar_ouster_os1	N/A	Implemented, verified1
Hokuyo YVT-35LX	USB-LAN + SW	hils_bridge_lidar_hokuyo_yvt35lx	N/A	Implemented, verified2
GPS (NMEA 0183)	FT234X x 2	hils_bridge_gps_nmea0183	N/A	Implemented, verified
IMU (Witmotion WT901)	FT234X x 2	hils_bridge_imu_witmotion_wt901	N/A	Implemented, verified3

Track B: Microcontroller developers (RP2040 firmware)

Device	Method	ROS Package	Firmware	Status
USB Camera	RP2040 UVC	hils_bridge_camera_uvc	rp2040_camera_uvc + rp2040_camera_uvc_spi_sender	Implemented, verified (incl. fault-injection regression via the Pico pair)
RC Servo (capture)	RP2040 PIO pulse-width measurement	hils_bridge_actuator_servo_pwm	rp2040_actuator_servo_pwm	Implemented, verified (Arduino PWM → JointState)
Quadrature Encoder	RP2040 PIO	hils_bridge_encoder_quadrature	rp2040_encoder_quadrature	Implemented, unverified
I2C IMU (MPU-6050)	RP2040 I2C slave	hils_bridge_imu_invensense_mpu6050	rp2040_imu_invensense_mpu6050 (+ rp2040_mpu6050_reader as bus master surrogate)	Implemented, verified (incl. firmware fault injection: NACK / response delay / register freeze / WHO_AM_I mismatch)

Quick Start

Building the ROS 2 packages

cd ~/colcon_ws/src
git clone --recursive https://github.com/<your-org>/ros_hardware_in_the_loop_system.git
# Or clone only the ROS packages:
# git clone https://github.com/<your-org>/ros2_hils_bridge.git

cd ~/colcon_ws
colcon build
source install/setup.bash

Building firmware (RP2040 only)

cd firmware/rp2040_camera_uvc
mkdir build && cd build
cmake .. -DPICO_SDK_PATH=~/pico-sdk
make -j$(nproc)
# Copy build/rp2040_camera_uvc.uf2 to Pico in BOOTSEL mode

Checking the normal path

Every emulator is checked the same way: feed it sensor data on the simulation side, run the real vendor driver on the robot side, and confirm the driver's own topic. Wiring, IP addresses and container layout are in the Verification Guide; what follows is the check itself.

Device	Feed the emulator	Check on the driver side	Expected
Livox Mid-360	PointCloud2 (rosbag or simulator)	livox_ros_driver2 → /livox/lidar	~10 Hz
Velodyne VLP-16	PointCloud2	velodyne_driver + velodyne_pointcloud → /velodyne_points	~10 Hz
Ouster OS1	PointCloud2 (+ Imu)	ouster_ros → /ouster/points	~10 Hz
Hokuyo YVT-35LX	PointCloud2 on /sim_points	urg3d_node2 → /hokuyo_cloud2, /imu	20 Hz, 2590 points/frame
GPS (NMEA 0183)	NavSatFix on /gps/fix (+ TwistStamped on /gps/vel)	nmea_navsat_driver → /fix, /vel	1 Hz, lat/lon match the input
IMU (WT901)	Imu on /imu/data at 50 Hz	witmotion_ros → /imu	~25 Hz
IMU (MPU-6050, I2C)	Imu on /imu/data	I2C master's [STAT] lines	WHO_AM_I=0x68, ACCEL_Z≈16384 (1 g)
USB camera (UVC)	Image on /image_raw	v4l2-ctl --list-devices, then usb_cam → /image_raw	15 fps, MJPEG only
RC servo PWM	controller drives the pins	/servo_pwm/pulses_us, /servo_pwm/joint_states	1500 us → 0 rad
Quadrature encoder	JointState on /joint_states	logic analyser on GPIO 6-9	1.57 rad at cpr=1000 → ~250 counts

Numeric check

ros2 topic hz /hokuyo_cloud2          # rate
ros2 topic echo --once /hokuyo_cloud2 --no-arr   # header, width, fields

Visual check (LiDAR)

For the YVT-35LX there is a demo script that brings the whole normal path up and leaves it running with RViz — unlike the E2E, which exits as soon as the oracle has judged:

bash ros2_hils_bridge/tools/run_yvt35lx_demo.sh   # Ctrl-C, or close RViz, to stop

It injects no faults; the banner it prints has a copy-paste command if you want to trigger one by hand while it runs. Because its scene is noise-free, it switches the emulator's sensor noise on (20 mm range sigma, 1 % dropout) — that noise is off by default so it cannot double up with a simulation that models its own. For the other sensors, or to set RViz up yourself:

rviz2
# Global Options -> Fixed Frame: the cloud's frame (see below)
# Add -> By topic -> <topic> -> PointCloud2
# PointCloud2 -> Color Transformer: Intensity

The fixed frame must match the cloud, otherwise RViz shows nothing:

ros2 topic echo --once --field header.frame_id /hokuyo_cloud2

urg3d_node2 uses hokuyo3d by default (frame_id parameter) and the documented velodyne_driver command uses velodyne.

Running RViz from a container needs the X socket passed in. docker/launch_docker.sh already does that, so:

bash docker/launch_docker.sh
# inside the container:
source /opt/ros/jazzy/setup.bash && source ~/colcon_ws/install/setup.bash && rviz2

With your own image, note the images here use bash as their entrypoint, so the command goes through it (WSL/WSLg shown):

docker run -it --rm --net=host \
  -e DISPLAY -e WAYLAND_DISPLAY -e XDG_RUNTIME_DIR=/mnt/wslg/runtime-dir \
  -v /tmp/.X11-unix:/tmp/.X11-unix -v /mnt/wslg:/mnt/wslg \
  --entrypoint bash <image> -lc 'source /opt/ros/jazzy/setup.bash && rviz2'

Reading a YVT-35LX cloud. A flat wall appears as a row of U-shaped traces, not a filled surface and not vertical stripes. That is the scan pattern: the mirror oscillates at 1200 Hz while the rangefinder samples at 88800 points/s, so one oscillation is 74 points — one VSSP line. A 20 Hz revolution holds 60 oscillations, of which the 210°/360° inside the field of view are measured, giving 35 lines and 35 x 74 = 2590 points per frame. Within a line the elevation traces one full period of a sine (middle → +35° → middle → −5° → middle) while the azimuth advances 6°, so the traces join into a continuous wave across the field of view. The sensor looks up rather than down: ±105° horizontal, −5°…+35° vertical. Feeding a 10 m wall, a 2.5 m ceiling and a 0.5 m pillar reconstructs across −105.0…+105.0° azimuth and −4.8…35.0° elevation.

Automated pass/fail

For the LiDARs the normal path is also covered by one-command regressions that bring the whole chain up, inject faults and let the test oracle judge — see the Verification Guide and ros2_hils_bridge/tools/:

The YVT-35LX one needs no hardware and no second container, so it is the quickest way to see the whole chain run:

bash docker/build_docker_image.sh    # once; the image carries urg3d_node2
bash docker/launch_docker.sh
# inside the container:
cd ~/colcon_ws && colcon build --packages-up-to hils_bringup \
  hils_bridge_lidar_hokuyo_yvt35lx urg3d_node2 && source install/setup.bash
bash src/ros2_hils_bridge/tools/run_yvt35lx_e2e.sh   # exit 0 = all expectations pass

Documentation

• Architecture Design
• Bill of Materials (BOM)
• Expansion Roadmap
• Verification Guide

License

MIT

Footnotes

1. For Ouster, the emulator exposes HTTP REST API on port 80, so Docker requires sysctls: net.ipv4.ip_unprivileged_port_start=80. See docs/hils_verification_guide.md for details. ↩

2. The YVT-35LX emulator speaks VSSP 2.1 over TCP (control and measurement multiplexed on one socket), so no extra NIC is needed for the loopback E2E: bash ros2_hils_bridge/tools/run_yvt35lx_e2e.sh drives the real urg3d_node2. Like every other driver under test it is not vendored in this repository; the docker image clones it into the workspace (as it already does for livox_ros_driver2 and witmotion_ros), or you can git clone --recursive https://github.com/Hokuyo-aut/urg3d_node2 into your own colcon workspace (needs ros-<distro>-laser-proc and ros-<distro>-diagnostic-updater). That regression also surfaced a driver defect — byte corruption on the VSSP stream segfaults urg3d_node2 — recorded in docs/fault_injection_implementation_policy.md section 22.5. ↩

3. The IMU emulator emits the four standard WT901 packets (0x51 Accel / 0x52 Gyro / 0x53 Euler / 0x59 Quaternion) so it works with witmotion_ros (ElettraSciComp) at default use_native_orientation: true. Driver build needs libqt5serialport5-dev (already in the Dockerfile). See docs/hils_verification_guide.md for setup details. ↩