Lab A: Agentic Driving Coach
Agent type: Large Language Model (LLM) agent
Goal
Place a small local LLM in the control loop of a simulated car approaching a stop sign. The coach reasons about the situation and produces driving guidance or emergency actuation while the system tracks both logical time and real inference latency.
The central question is not only whether an LLM response is sensible, but also whether it arrives in time to affect the physical outcome. Late, malformed, or unsafe responses trigger a deterministic fallback.
You compare live llama3.2:1b and llama3.2:3b models with deterministic rule
and replay runs. Recorded short-deadline traces let you examine missed deadlines
and late responses separately from your machine’s live inference performance.
System topology
The driver, car, environment, hierarchical coach, inference deadline race, and planner form a closed loop. Driver commands cross 500 ms logical delays, while coach emergency actuation returns over a 200 ms delay. Click the diagram for the full-size topology from the project repository.
Focus
- LLM reasoning inside a cyber-physical system’s control loop
- Xronos reactors, logical time, wall-clock lag, and delayed connections
- Inference deadlines, response validation, and deterministic fallback
- Reproducible comparison of model latency, output quality, and stopping outcome
What you will do
- Work through five Xronos examples, including the difference between an after-the-fact deadline check and a fallback scheduled independently of model completion.
- Run deterministic rule and replay baselines, predict the effects of small reactor edits, and compare the predictions with recorded behavior.
- Compare live local LLMs under identical conditions and examine recorded deadline-stress replays, clearly identifying each source of results.
- Analyze how driver behavior, model timing, and fallback decisions change the simulated car’s physical outcome.
- For the CSE 598 extension, choose a speed-change scenario, generated driver behavior, a deadline sweep, or a live-input reactor and compare it with a controlled baseline.
Learning outcomes
By the end of this lab, you should be able to:
- distinguish logical time from wall-clock inference time, lag, and slack;
- measure latency, deadline misses, and malformed model responses;
- explain why deterministic fallback is needed in an agentic control loop; and
- connect software timing and model behavior to a physical stopping result.
Project repository
Use the public embodied-ai-lab/agentic-driving-coach template to create your group’s private repository. ASU Sol is the primary platform: timing examples and deterministic baselines use CPU compute, while the documented live-model workflow uses a GPU allocation.
