Lab C: Safe Agent Actuation

Agent type: Vision-Language-Action (VLA) agent

Goal

Place a safety and authorization layer between a VLA model and a simulated ALOHA bimanual robot. A pi0-FAST policy turns a camera observation and language instruction into action tokens, which are decoded into a continuous robot trajectory before any physical risk can be assessed.

The monitor measures the decoded motion, classifies it against a configurable threshold, and requests context-based authorization through the Secure Swarm Toolkit (SST) when an action is high risk. The actuator independently verifies the protected action and either executes it or holds position.

Safety pipeline

Physical risk is assessed only after FAST decoding. Low-risk actions can follow the direct path, while high-risk actions require context-based SST authorization and protected delivery before independent enforcement at the actuator.

Focus

• VLA action generation and FAST decoding
• Motion-risk analysis in physical joint space
• Context-based authorization for high-risk actions
• Enforcement at the actuator and safe rejection of invalid inputs

What you will do

1. Inspect the relationship between VLA action tokens and decoded actions.
2. Implement and evaluate normalized joint-motion risk analysis.
3. Protect, transfer, verify, and enforce high-risk actions with SST.
4. Sweep risk thresholds to study authorization rate and latency tradeoffs.
5. For the CSE 598 extension, evaluate the security window created by session-key reuse.

Learning outcomes

By the end of this lab, you should be able to:

• explain why physical risk is assessed after action decoding rather than from token identifiers;
• classify robot motion using observed state, physical limits, and a threshold;
• enforce authentication and authorization at the actuator boundary; and
• separate motion classification, policy authorization, and physical safety when interpreting results.

All experiments run in simulation; this project does not control a real robot.

Project repository

Use the public embodied-ai-lab/safe-agent-actuation template to create your group’s private repository. Read its README.md for the ASU Sol workflow and its ASSIGNMENT.md for the current tasks, deliverables, and rubric.