ModPack Turns Robot Teleoperation Into a Wearable, Modular System

Robot teleoperation rigs tend to grow around one machine and one job. Add a mobile base, a second arm, force feedback, or a moving camera, and the operator interface often needs a redesign.

A Stanford team is testing a more modular answer. ModPack puts the common parts of a teleoperation system into a self-contained wearable backpack, then lets researchers attach the controls needed for a particular robot and task. The system is not a remote-control product ready for general deployment. It is an open research platform for collecting demonstrations and training robot policies, and the first experiments show both where that approach helps and where it remains limited.

The backpack is the common platform

The 3D-printed backpack carries the computer, power, communications, and data storage used during operation. Its internal hardware sits on five removable shelves, while top-mounted slots accept interchangeable leader arms. A lightweight bridge API separates the backpack software from robot-specific code.

That separation is the central idea. Researchers can use a pair of six-degree-of-freedom leader arms with one robot or swap in seven-degree-of-freedom arms for another. The joints map directly to the follower robot's arms, providing more granular control than a joystick or controller that sends only an end-effector pose.

The leader arms include active gravity compensation to reduce operator fatigue. On a robot equipped with force and torque sensors, their motors can also push back against the operator. That haptic feedback gives a physical warning when the robot is pressing too hard or making contact at the wrong angle.

Walking and looking become separate controls

Mobile manipulation adds another problem: an operator may walk, turn, move their arms, and look around at the same time. ModPack uses an iPhone mounted to the backpack to track body movement and map it to the robot's mobile base. An Apple Vision Pro can stream the robot's view and turn the operator's head motion into camera commands.

The software compensates for motion already caused by the mobile base. Without that step, walking and looking could produce duplicate or ambiguous camera movement. Keeping arm, base, and head commands separate lets the operator search around an obstacle while continuing to move the robot.

These pieces are modules, not mandatory equipment. A tabletop task may need only the leader arms. A longer search-and-reach task can add mobility and active perception, while a contact-heavy task can add force feedback.

Two experiments put the modules to work

The researchers tested ModPack on two different bimanual mobile robots and used its demonstrations to train diffusion policies.

In a cloth-placement task, the robot had to pick up a cloth, back away from a counter, find a basket on either side, move to it, and drop the cloth inside. The team collected 125 demonstrations. A policy using only the movable head camera succeeded in 22 of 25 rollouts; a policy using the head and both wrist cameras succeeded in 20 of 25, while wrist cameras alone managed 3 of 25. The result suggests that the wide, controllable head view carried more useful context than close wrist views for this particular task.

For box transfer, the second robot had to carry a box to the empty level of a two-shelf rack. The team collected 102 demonstrations and ran 20 tests per policy. A model using all camera views plus arm-joint torque data succeeded 12 times, compared with 11 for the head-camera model and 6 for the all-camera model without torque. The researchers argue that torque gave the policy a rough signal about alignment during grasping.

Open hardware with honest constraints

ModPack does not solve general robot control. The box policy with the best result still failed eight of 20 trials. The wearable arm motors have limited torque, and supplying a mini PC and two powered leader arms for hours requires batteries that make the backpack heavier.

What the project contributes is a reusable interface for gathering richer training data across different machines. The team says the complete hardware design, software stack, and policy framework will be open-sourced. If that release is practical to reproduce, researchers will be able to build on one shared teleoperation base instead of rebuilding the operator side for every new robot.