Lab C: Safe Agent Actuation
Agent type: Vision-Language-Action (VLA) agent — recorded response and controlled action experiments
Goal
Study how motion analysis, context-based authorization, and actuator enforcement work together for VLA actions. First, inspect a genuine recorded pi0-FAST DROID response and its FAST decoding evidence. Then use explicitly synthetic ALOHA action fixtures to implement and evaluate the enforcement path.
These are two separate data sources: the recorded DROID response is inspected but never executed or converted into ALOHA commands. The graded motion experiments use 14-dimensional ALOHA fixtures and a controlled current state.
The monitor validates a fixture, measures its normalized motion, and compares
the score with a risk threshold. High-risk actions require context-based
authorization and protected delivery through the
Secure Swarm Toolkit (SST). A recording actuator
harness validates the input and records EXECUTE or HOLD.
Safety pipeline
Part 1 inspects a committed model capture. Parts 2–5 analyze synthetic fixtures and exercise actuator enforcement with real SST. The recording harness measures forwarding and rejection, not robot dynamics or task success. Adapted from the project architecture.
Focus
- VLA response provenance, token availability, and FAST decoding evidence
- Normalized motion analysis on controlled joint and gripper commands
- Context-based authorization and protected delivery for high-risk actions
- Actuator enforcement, invalid-input rejection, and session-key reuse
What you will do
- Inspect the genuine pi0-FAST response, its metadata, checksum, returned shape, and reported token availability. The pinned standard response does not expose raw token IDs.
- Implement motion analysis using synthetic ALOHA fixtures, the current state, physical ranges, and the risk threshold.
- Demonstrate an authorized high-risk action, a denied context, and rejection of an unauthenticated actuator bypass using real SST.
- Compare thresholds on the same fixture/context cases. Report classification, authorization activity, key requests and reuse, and measured Monitor-to-actuator latency.
- For the CSE 598 extension, measure how session-key validity affects reuse and the security window after the context changes.
Learning outcomes
By the end of this lab, you should be able to:
- distinguish a captured model response from synthetic action fixtures;
- explain why motion is assessed in decoded physical space rather than from token identifiers;
- enforce authentication and authorization at the actuator boundary; and
- distinguish motion classification, policy authorization, and observed enforcement from a claim of physical safety.
The graded harness does not simulate robot dynamics or control a real robot. All graded parts run on CPU compute with the committed capture; no GPU, model download, or live model server is required.
Project repository
Use the public embodied-ai-lab/safe-agent-actuation template to create your group’s private repository. Follow its ASU Sol CPU workflow and complete the student TODOs before running the full experiment set. Offline diagnostic authorizers are development aids; graded authorization results require real SST.