NVIDIA Holoscan 5: Robot arm gravity compensation, three-operator control loop

1. What Holoscan 5 adds to the control loop

Holoscan 5 provides one development approach for fast robot control loops and high-volume sensor processing. Developers create each stage as an operator, an independently executed processing unit, then connect the data flow as a graph. This reduces the need to learn separate configuration methods for control and image analysis running at different rates.

Placing consecutive tasks in one execution group, called a partition, lets one thread process them in sequence. One worker handles the sequence instead of handing each stage to another thread. The compiler records this arrangement in an execution plan. Developers can inspect task placement and memory preparation before powering on the equipment.

2. Three stages from state reads to motor commands

The tutorial targets the Seeed Studio reBot Arm B601-DM. NVIDIA describes its actuators as suitable for torque control and sensing, unlike inexpensive hobby servos that mainly control position. Servos are motors that control movement in response to commands.

The loop periodically reads joint states from seven motors, calculates gravity torques with Pinocchio, and sends commands to the motors. Pinocchio uses the robot model and current pose to find the torque each joint needs to counter gravity.

The demo runs 400 times per second. It adds a weak restoring force toward a reference pose and damping to calm motion to the calculated gravity torque. This lets the arm support its own weight while moving gently when pushed.

The source places all three operators in one partition and sets queue depth to 1, discarding old data. This configuration can prevent stale joint states from accumulating when computation falls behind.

graph = hsdk.Graph("rebot-gravity", default_clock=hsdk.RealtimeClock)
source = graph.op(JointStateSourceOp, "joint_feedback")
gravity = graph.op(PinocchioGravityOp, "gravity", **controller.parameters.operator_kwargs())
sink = graph.op(MotorCommandSinkOp, "motor_commands")
graph.add_flow(source.state, gravity.state, queue_depth=1, queue_policy=hsdk.QueuePolicy.DROP_OLDEST)
graph.add_flow(gravity.command, sink.command, queue_depth=1, queue_policy=hsdk.QueuePolicy.DROP_OLDEST)
graph.partition("control").add(source).add(gravity).add(sink)

3. Execution planning and safe shutdown

The application compiles the graph into an execution plan, initializes the motors and model, then starts asynchronous execution without waiting for earlier work to finish. The example raises an error with a summary if the plan is empty. NVIDIA says this sequence helps identify graph configuration problems before powering on real equipment.

A watchdog monitors command completion. If no motor command batch completes for 100 milliseconds, it attempts to disable the motors and stops the graph. A shutdown callback, a function invoked automatically at shutdown, also disables the motors.

The example runs on Linux with Python 3.12 or later.

The same repository includes a second Python example that specifies a target position and orientation while responding compliantly to contact. It offers a starting point for extending hand-guided gravity compensation to target-pose control.

4. Latency with and without core pinning

NVIDIA measured operator execution using Nsight Systems, an execution analysis tool. The latencies below span entry into the first operator to entry into the last, excluding motor command transmission. Each run collected more than one million cycles. The first 10 seconds of sequential measurements were excluded, while outliers were retained.

In the table, p50 is the median, with half of measurements at or below it. The p99.99 value has 99.99% of measurements at or below it. The median describes typical response, while high percentiles help reveal rare delays.

Table: Gravity compensation loop latency by implementation

Conditions: Company-reported reBot Arm B601-DM gravity compensation demo; pinned runs use core pinning to bind tasks to specific CPU cores and core isolation to keep other tasks off those cores.

Unit: microseconds

With core pinning and isolation, the company reported observed p99.99 latency reductions of about 5x for C++ and 9.6x for Python. Yet pinned Python's p99.999 remained far above its median at 1,014.0 microseconds. The result highlights the need to address both typical response times and rare delays.

5. Lower latency requires more careful operation

Both implementations use the same C++ computation engine. NVIDIA attributes additional Python overhead to execution and object allocation, including reserving memory for the program. For tight control deadlines, C++ offers greater benefits by controlling memory allocation and system calls, which request operating system services. The company says Python remains practical for composing GPU perception tasks whose CUDA stages take milliseconds or tens of milliseconds.

Low latency brings the operational burden of assigning cores to control and managing interference from other tasks. These measurements reserved physical cores for control and auxiliary work against use by other applications. They did not exclude interrupts, signals that briefly interrupt CPU processing, or some kernel tasks performed by the operating system's core.

Holoscan 5's central contribution is to put robot control in the same execution model as sensor processing, letting developers inspect that execution in advance and measure it.