MATE: Policy-Aware Security Auditing for Mobile Agents via Synthesis-Driven Trajectory Learning

**Figure 1: Overview of the MATE process.** 1. Collect functional workflows and security policies from 158 apps. 2. Use a knowledge-grounded synthesis pipeline with three-stage quality checking and repair to construct semantically realistic trajectories. 3. Apply data augmentation and supervised fine-tuning to train MATE and build MATEBench. 4. At deployment, a trajectory adapter and policy retriever convert raw logs into ReAct-style trajectories, retrieve security policies, and feed them to MATE, which receives an instruction, a trajectory, and a customized natural-language policy and outputs a policy-conditioned audit result.

Abstract

LLM-based mobile agents are increasingly deployed to automate complex workflows in mobile applications, but their execution trajectories can violate application- and user-specific security policies. Existing trajectory-level safety approaches either rely on prompting general-purpose LLMs or on static pattern rules, making it difficult to enforce fine-grained natural-language policies and to generalize across heterogeneous apps, policies, and tasks. Moreover, training policy-aware security auditors is hindered by the lack of realistic, policy-conditioned mobile agent trajectories.

In this paper, we present MATE, a lightweight, policy-aware security auditing model for mobile agents. MATE jointly encodes execution trajectories and natural-language security policies to produce policy-conditioned violation judgments with explanations, treating security requirements as editable text rather than fixed model parameters. This design enables MATE to support application-specific and user-defined policies and to adapt to evolving security requirements without retraining. To train MATE, we extract functional descriptions, operation workflows, and security policy from 158 Chinese and English mobile applications and build a knowledge-grounded synthesis pipeline that generates semantically realistic, policy-conditioned agent trajectories with multi-stage quality control, yielding over 140K trajectories for training. Using this pipeline, we construct MATEBench, a trajectory-level security auditing benchmark with two synthetic subsets and one real-world subset of manually collected mobile agent trajectories. Models trained via synthesis-driven trajectory learning achieve over 95% auditing accuracy on MATEBench and over 90% accuracy on two external agent-security benchmarks, exceeding baselines by about 20% on average. Crucially, on the real-world subset of MATEBench, our best model also exceeds 95% accuracy, demonstrating that knowledge-grounded synthesis enables accurate and customizable security auditing for mobile agents under both application-derived and user-specified policies.

Methodology

MATE is a lightweight, policy-conditioned auditor that maps an instruction, a mobile-agent trajectory, and one or more natural-language security policies to a violation decision, a risk category, and a natural-language rationale. Its construction and deployment follow four stages.

App Knowledge

Collect functional descriptions, operation workflows, and security policies from 158 Chinese and English mobile apps.

Data Synthesis

Synthesize executable instructions, ReAct-style trajectories, policy-conditioned labels, risk categories, and explanations.

Robust Training

Apply trajectory-policy mismatch, multi-policy supervision, and multi-app augmentation before supervised fine-tuning.

Deployment

Normalize heterogeneous raw logs with a trajectory adapter and retrieve relevant policies for low-latency local auditing.

Knowledge-Grounded Synthesis and Quality Control

The synthesis pipeline contains four phases: instruction synthesis, trajectory synthesis, annotation synthesis, and quality checking and repair. Each sample is checked for structural completeness, repaired for semantic consistency and policy alignment, and finally verified by human auditors. The resulting training corpus contains more than 140K policy-conditioned trajectories covering single-app and cross-app workflows.

Policy-Conditioned Auditing

Unlike a context-free risk classifier, MATE evaluates the same behavior differently under different policies. Security requirements remain editable natural-language inputs rather than fixed model parameters, allowing app-specific, organization-specific, and user-defined policies to change without retraining the auditor.

MATEBench

MATEBench is a bilingual Chinese-English benchmark for policy-aware, trajectory-level mobile-agent security auditing. It covers 14 risk categories and contains three complementary subsets spanning in-domain synthesis, unseen apps, and real devices.

Subset	Distribution	Apps	Trajectories	Purpose
MATEBench-In	In-domain, synthetic	134 seen apps	2,775	Controlled evaluation on apps represented during training
MATEBench-Out	Out-of-domain, synthetic	24 unseen apps	1,150	Generalization to new app semantics and workflows
MATEBench-Real	Out-of-domain, real device	13 apps / 3 agents	162	Auditing real trajectories with realistic logging noise

Synthetic and real trajectories exhibit closely aligned step-count and GUI-action distributions. MATEBench-In and MATEBench-Out average roughly nine steps, while MATEBench-Real averages roughly eight, supporting the realism of the knowledge-grounded synthesis process.

Evaluation Results

We compare MATE with a static rule engine, general-purpose LLM judges, ShieldAgent, and ShieldLM on MATEBench, R-Judge, and ASSEBench. MATE-3B achieves the strongest overall results across all five evaluation sets, including 95.06% accuracy and 95.60% F1 on real mobile-agent trajectories.

Overall accuracy and F1 results across five policy-aware trajectory auditing benchmarks — **Overall policy-aware auditing performance.** MATE consistently outperforms rule-based and LLM-as-a-judge baselines on R-Judge, ASSEBench, MATEBench-In, MATEBench-Out, and MATEBench-Real.

Model	R-Judge Acc	ASSEBench Acc	MATEBench-In Acc	MATEBench-Out Acc	MATEBench-Real Acc
MATE-0.5B	90.02%	91.41%	94.05%	92.00%	86.42%
MATE-1.5B	91.94%	93.81%	95.42%	92.78%	90.12%
MATE-3B	92.64%	94.98%	96.83%	95.48%	95.06%

Efficient and Deployable Auditing

MATE combines higher auditing quality with substantially lower inference cost. On a single NVIDIA H100, MATE-0.5B audits one MATEBench-Out trajectory in 0.09 seconds, while MATE-3B reaches 95.48% accuracy with a latency of only 0.21 seconds.

Accuracy, F1, latency, and model-size comparison on MATEBench-Out — **Accuracy-efficiency comparison on MATEBench-Out.** MATE models are smaller and faster than strong general-purpose LLM judges while delivering better accuracy and F1.

Ablation Studies

Zero-shot Qwen2.5-Instruct models achieve less than 50% accuracy on average for policy-conditioned trajectory auditing. Fine-tuning on the synthesized MATE corpus raises overall accuracy above 92%, showing that generic instruction-following alone is insufficient for this task.

Comparison between Qwen2.5-Instruct base models and fine-tuned MATE models — **Effect of synthesis-driven trajectory learning.** Task-specific fine-tuning produces large accuracy and F1 gains for 0.5B, 1.5B, and 3B models on all three MATEBench subsets.

External app knowledge and quality repair are both important. On MATEBench-Real, a MATE-0.5B model trained on the full synthesis pipeline reaches 75.93% accuracy in the controlled ablation setting; removing both components reduces accuracy to 46.91%.

External Knowledge	Quality Repair	MATEBench-Real Acc	MATEBench-Real F1
No	No	46.91%	29.51%
No	Yes	51.23%	43.79%
Yes	No	69.75%	66.67%
Yes	Yes	75.93%	79.14%

Real-World Case Studies

MATE-3B accurately audits trajectories collected from deployed mobile agents. The examples below show a malicious coupon link mass-forwarded by Zhipu's AutoGLM and a malware-test page opened by Alibaba's Mobile-Agent. MATE identifies the corresponding risks as Becoming a Fraud Relay and Device Security Compromised, and explains the policy conflict in natural language.

Two real mobile-agent trajectories audited by MATE-3B — **Policy-aware auditing on real mobile agents.** MATE-3B audits trajectories from Zhipu's AutoGLM and Alibaba's Mobile-Agent after raw screenshots are normalized by the trajectory adapter and relevant policies are selected by the policy retriever.

Citation

@misc{jiang2026mate,
  title={MATE: Policy-Aware Security Auditing for Mobile Agents via Synthesis-Driven Trajectory Learning},
  author={Jiang, Changyue and Wang, Jiayi and Wen, Xin and Dai, Jiarun and Hong, Geng and Pan, Xudong},
  year={2026}
}