Model Performance
Dive into the performance of each core autonomous vehicle model layer, comparing their current observations against defined target capabilities.
Localization Layer: Precision & Robustness
The localization system is the vehicle's compass. We evaluate its ability to maintain precise position and orientation, even under challenging conditions, by comparing target goals with current performance.
Pose Accuracy: Gaps Identified
Our model currently shows higher error rates than targeted for accurate positioning and heading. Lower values are better for these metrics.
Update & Latency: Room for Improvement
Real-time responsiveness is key. While the update rate is close, latency exceeds the target, impacting system reactivity. Higher update rate is better, lower latency is better.
Perception Layer: Understanding the World
The perception system is the vehicle's eyes and ears, detecting objects, lanes, and semantic elements. We analyze its accuracy and speed against the MVP targets for campus environments.
Object Detection & Tracking Performance
Critical for identifying and following dynamic elements. The model shows areas for improvement in inference speed and tracking stability. Higher values for mAP & MOTA are better, lower for latency & ID Switches.
Lane & Semantic Segmentation Accuracy
Understanding the drivable path and environment is vital. The model shows a gap in both lane detection accuracy and overall semantic segmentation. Higher values are better, lower for offset error.
Trajectory & Behavior: Predicting the Future
Predicting the future movements and intentions of other agents is a complex but essential task for safe autonomous driving. We assess the model's accuracy, speed, and readiness for complex scenarios.
Trajectory Accuracy: Needs Improvement
The model's predicted trajectories show higher displacement errors than targeted, impacting the safety and smoothness of planned paths. Lower values are better.
Behavioral Prediction: Progress Made
While the F1 score is close to target and the prediction horizon is met, continued refinement is needed for robust behavioral understanding. Higher F1 score is better.