Semantic Hierarchy-Guided Adversarial Attack for Autonomous Driving
Gwangbin Kim, SeungJun Kim
AI summary
Problem
Traditional adversarial attacks on semantic segmentation produce arbitrary pixel-wise misclassifications that ignore scene semantics, limiting their realism and effectiveness for autonomous driving safety assessments.
Approach
SHAA integrates a semantic hierarchy affinity matrix with adaptive momentum-based gradient updates to guide perturbations toward semantically meaningful yet highly effective misclassifications.
Key results
- Achieves higher attack success rates and lower mIoU across Cityscapes and NightCity datasets compared to state-of-the-art white-box attacks
- Maintains strong attack effectiveness under small perturbation budgets (ϵ=2) and against input-level defenses like bit-depth reduction and median filtering
- Demonstrates robust cross-architecture transferability to DeepLabV3+, OCRNet, and Mask R-CNN
- Ablation studies confirm semantic consistency and adaptive momentum components are critical for optimal attack performance
Why it matters
Exposes critical, semantically aware vulnerabilities in autonomous driving perception models, informing the design of more resilient defensive strategies and adversarial training pipelines.
Abstract
Autonomous vehicles employ semantic segmentation as a foundational component for perception and scene understand- ing, upon which driving decisions can be informed. Despite their performance, these deep learning models remain susceptible to subtle input perturbations that can cause severe deviation in model output. To enhance algorithmic robustness by examining such vul- nerabilities, researchers have investigated adversarial examples, which are visually imperceptible yet can severely degrade model performance. However, traditional attacks produce arbitrary mis- classificationsthatignoresemanticrelationships,makingtheattack less effective. This letter introduces a semantic hierarchy-guided adversarial attack (SHAA), a white-box adversarial attack against semantic segmentation for autonomous driving. By combining se- mantic hierarchy and adaptive momentum-based updates across the image, SHAA produces semantically nontrivial yet highly effec- tive perturbations. The SHAA method exposes deeper vulnerabili- ties with a higher attack success rate in semantic segmentation than existing methods, aiding the design of a more resilient perception system for autonomous vehicles.