A research team at Xidian University has demonstrated a new type of attack that manipulates multimodal AI systems through altered images. This attack, called CrossMPI, achieved a 66.36% average success rate across tested vision-language models and showed strong transferability between different model architectures.
If your organization uses multimodal AI systems—such as chatbots analyzing images, document processing pipelines, or automated content moderation—you need to understand what failed and what controls could have prevented it.
What Happened
Researchers developed CrossMPI to exploit the image processing pipeline in large vision-language models (LVLMs). Unlike traditional prompt injection attacks that manipulate text, CrossMPI embeds adversarial perturbations directly into images. These alterations are imperceptible to humans but can cause the model to misclassify objects or follow attacker-specified instructions.
The attack succeeded against multiple commercial and open-source models without needing access to model internals—a black-box attack transferable between architectures. When researchers tested the defense mechanism SmoothVLM, it reduced success rates below 5% in several scenarios, but the attack remained effective against unprotected systems.
Which Controls Failed or Were Missing
Input validation at the model boundary failed. Traditional input validation focuses on file type, size, and format, which do not detect adversarial perturbations in pixel values. The models processed malicious images as valid inputs because they passed standard validation checks.
Model output verification was absent. Systems lacked mechanisms to detect when model outputs deviated from expected behavior based on image content. A human would spot the misclassification, but no automated check caught the discrepancy.
Defense-in-depth for AI systems wasn't implemented. Models were deployed with the same trust assumptions as traditional applications. When the primary model failed, no secondary validation layer caught the manipulation.
Monitoring for adversarial inputs was missing. Systems lacked detection mechanisms for images that trigger anomalous model behavior. Standard security monitoring looks for malformed requests or suspicious access patterns—not for valid-looking images that produce invalid outputs.
What the Relevant Standards Require
ISO/IEC 27001:2022 Annex A.8.16 requires security controls for monitoring activities. For AI systems processing untrusted inputs, this means:
- Logging model inputs and outputs for anomaly detection
- Establishing baseline behavior for classification tasks
- Alerting on statistical deviations from expected output distributions
NIST Cybersecurity Framework v2.0 function PR.DS-2 calls for data integrity protection during transmission and at rest. For multimodal AI systems, this extends to:
- Validating that image content matches expected semantic meaning
- Implementing checksums or signatures that detect adversarial perturbations
- Maintaining audit trails linking inputs to outputs
OWASP ASVS v4.0.3 Requirement 13.2.3 addresses RESTful web services and requires validation of content types. For vision-language models, you need to extend this to:
- Semantic validation of image content against expected categories
- Rate limiting on image processing endpoints to slow reconnaissance
- Sandboxing model inference to contain potential exploitation
SOC 2 Type II Common Criteria CC6.1 requires logical and physical access controls. For AI systems, implement:
- Role-based access to model inference endpoints
- Separate processing pipelines for trusted vs. untrusted image sources
- Network segmentation between model serving and critical business systems
None of these standards explicitly address adversarial machine learning attacks. You're responsible for interpreting requirements in the context of your AI deployment risk.
Lessons and Action Items for Your Team
Implement output validation now. Don't rely solely on model confidence scores. Build a secondary classifier or rule-based system to check if model outputs make sense given the input. If your model labels an airplane as a mobile phone, flag it for human review.
Add semantic integrity checks to your image processing pipeline. Use perceptual hashing or feature extraction to verify that uploaded images haven't been adversarially modified. Compare hash values against known-good versions or establish acceptable variance thresholds.
Deploy ensemble defenses. Test SmoothVLM or similar preprocessing techniques that add controlled noise to inputs before model inference. The research showed this reduced attack success below 5% in some scenarios. Measure the accuracy tradeoff in your specific use case.
Separate trust boundaries. Process images from authenticated users differently than public uploads. Apply stricter validation and monitoring to untrusted sources. Route high-risk inputs through additional verification layers before acting on model outputs.
Build monitoring for model behavior drift. Track the distribution of model predictions over time. Sudden shifts in classification patterns may indicate adversarial inputs. Set alerts for statistical anomalies in output distributions.
Update your threat model. If you've assessed AI systems using only traditional application security controls, revisit that analysis. Document image-based prompt injection as a specific threat. Assign it a risk rating based on your deployment context. Identify which systems are vulnerable.
Test your models against adversarial inputs. Generate your own adversarial examples using open-source tools. Measure how your deployed models respond. Use the results to tune detection thresholds and validation rules.
The 66.36% success rate means two-thirds of attacks succeed against unprotected models. Your defense strategy needs to assume some attacks will get through. Plan your containment and recovery procedures accordingly.
