Single-task results identify what a model can recognize or localize. Matched Agent-Landmark pairs ask whether it succeeds on both sides of the same context. This analysis extends the paper from image and video performance to state-specific capabilities, paired coordination, and diagnostic validation.
Analysis Overview
From individual task performance to grounded Agent-World coordination
We first examine image and video tasks to distinguish semantic decisions from spatial and temporal grounding, then compare performance across media. We next regroup the results by Agent-State and World-State Reasoning and examine their joint success within matched Agent-Landmark contexts. The three pairings target reciprocal coordinate-frame reasoning, action-observation association, and behavior-visibility consistency, capabilities relevant to relative-position navigation, viewpoint adjustment, and persistent target observation.
Dual Semantic Acc measures whether both paired semantic decisions are correct; Dual Evidence Acc independently measures whether both grounding outputs satisfy their task-specific thresholds. Image scores harmonically combine the two image pairings, and cross-media Overall harmonically combines image and video scores. Full task tables, correlation plots, and relation-specific profiles show how these capabilities vary across models. Human validation, failure diagnoses, open-ended questions, and same-model reasoning-mode comparisons then explain the observed patterns; dedicated pages extend the analysis to real-scene reconstructions and structured supervision.
Evaluation snapshot. 2026-09-15; 45 image models, 32 video models, and 32 models in cross-media comparisons. All tables and charts use the shared Leaderboard results and parser icra27-dual-v5-class-aware-f1-selected-view. Invalid outputs count as failures; pair scores use available aligned records. Semantics and evidence are scored independently, so either Dual score can exceed the other. Full metric and parsing protocol.
Image Tasks
Strong single-view decisions do not ensure success on both paired tasks
Gemini 3.8 Flash leads image Dual Semantic and Dual Evidence Acc at 44.4% and 34.3%. It also leads both Agent-State semantic tasks, while GPT-5.5 reaches the highest Self-relative Position accuracy, 65.4%. These results distinguish a strong individual answer from the ability to answer both related questions successfully.
Muse Spark reaches 64.2% and 64.4% on the World-State decisions but only 14.8% and 14.9% BBox Acc@0.5. In contrast, GLM-5.3-Flash reaches 32.7% and 33.0% World-State BBox accuracy, leading the open-weight group. Under the same evaluation protocol, specialized spatial models show no consistent advantage across all four tasks.
#1 within group#2 within group#3 within group
Four image tasks; 45 models and the unranked Human baseline. Dual columns harmonically combine the position and action-observation pair scores.
Model
Agent-State Reasoning
World-State Reasoning
Image Coordination
Landmark-relative Position
Future Observation
Self-relative Position
Action Decision
Two image pairings
Acc
BBox@0.5
mIoU
Acc
BBox@0.5
mIoU
Acc
BBox@0.5
mIoU
Acc
BBox@0.5
mIoU
Dual Sem.
Dual Evi.
Proprietary Models
Claude Opus 4.8T
39.4%
11.4%
23.5%
35.9%
14.3%
25.4%
54.1%
10.4%
23.7%
51.9%
8.0%
21.7%
19.5%
2.5%
Claude Opus 4.7T
39.3%
8.2%
20.1%
37.2%
7.0%
18.1%
49.7%
7.1%
18.6%
39.1%
7.9%
18.0%
16.7%
1.1%
Claude Opus 4.6T
38.1%
2.1%
15.6%
32.4%
2.8%
18.7%
62.0%
8.6%
19.7%
58.9%
9.2%
20.1%
20.5%
0.0%
Claude Sonnet 4.6
37.7%
4.9%
18.2%
23.6%
3.6%
15.1%
48.5%
9.6%
20.2%
61.0%
8.6%
19.9%
15.9%
0.1%
Gemini 3.8 FlashT
60.1%1
66.7%1
57.6%1
74.9%1
73.8%1
62.5%1
61.9%
40.8%1
39.0%1
69.5%1
42.3%1
39.3%1
44.4%1
34.3%1
Gemini 3.1 Pro PreviewT
40.9%3
58.8%2
50.5%2
58.8%2
59.9%2
52.3%2
63.7%3
37.5%2
36.8%2
47.2%
37.1%2
36.4%2
27.1%
30.1%2
Gemini 3.1 Flash Lite
33.7%
48.1%3
45.7%3
30.1%
47.9%3
46.0%3
47.6%
9.7%
21.1%
35.9%
9.3%
20.8%
12.4%
5.5%
Gemini 3 Flash
47.6%2
33.6%
33.8%
45.9%
37.5%
38.5%
56.2%
5.0%
13.5%
51.3%
5.1%
12.2%
24.9%
2.0%
GPT-6 AstraT
35.1%
27.8%
31.2%
43.0%
27.8%
32.7%
57.2%
16.1%
24.3%
55.4%
14.9%
24.3%
21.5%
6.9%3
GPT-5.6 SolT
34.1%
23.9%
28.8%
42.1%
24.6%
30.1%
55.7%
12.9%
23.0%
55.8%
12.5%
22.4%
20.5%
5.6%
GPT-5.6 TerraT
33.7%
24.8%
28.9%
40.5%
27.1%
31.0%
58.9%
14.5%
23.4%
60.2%
12.9%
22.5%
22.3%
6.3%
GPT-5.5
38.0%
19.4%
26.2%
53.4%3
17.5%
26.3%
65.4%1
17.6%
25.1%
53.6%
17.9%
24.7%
27.1%3
5.8%
GPT-5.4
37.9%
12.2%
24.1%
37.6%
13.9%
25.9%
37.8%
12.9%
24.4%
36.0%
12.0%
23.5%
14.0%
3.2%
GPT-5.3
35.2%
11.9%
23.5%
37.3%
13.9%
26.7%
56.5%
14.9%
22.5%
62.3%3
15.4%
22.5%
21.1%
2.9%
MiMo-V2-Omni
28.7%
40.7%
36.5%
21.3%
36.9%
33.4%
36.5%
8.8%
17.6%
38.2%
9.2%
17.3%
9.4%
4.6%
Muse Spark 1.3 ContributorT
38.7%
20.5%
26.7%
48.6%
15.0%
21.2%
64.2%2
14.8%
25.5%
64.4%2
14.9%
24.9%
28.1%2
4.5%
Qwen3.7-Plus
33.3%
18.4%
28.0%
34.6%
21.4%
30.3%
55.4%
18.4%
26.7%
43.4%
18.8%3
27.0%3
15.7%
6.5%
Qwen3.6-Plus
32.8%
16.6%
27.1%
33.5%
14.7%
26.1%
48.4%
7.4%
19.0%
42.5%
6.0%
19.9%
13.7%
1.6%
Qwen3.6-Flash
25.1%
24.6%
32.0%
28.7%
19.2%
29.8%
62.7%
19.4%3
27.1%3
56.7%
18.6%
25.8%
16.0%
6.0%
Qwen3.5-Plus
28.6%
21.4%
27.1%
29.0%
12.3%
24.1%
47.6%
9.7%
19.3%
48.9%
8.3%
20.0%
12.7%
1.6%
Qwen3.5-Flash
27.8%
21.6%
29.8%
28.9%
15.8%
23.1%
52.4%
8.8%
19.4%
54.5%
8.9%
20.1%
15.0%
2.3%
Open-weight Models
GLM-5.3-FlashT
32.4%
38.4%
38.1%
35.2%
45.4%1
43.8%1
63.6%1
32.7%1
33.5%1
57.4%2
33.0%1
34.1%1
19.9%1
20.6%1
GLM-4.6V
30.8%
47.5%1
43.0%1
27.1%
44.9%2
42.3%2
29.2%
9.5%
20.0%
44.2%
10.2%
20.6%
9.9%
5.3%
GLM-4.5V
31.9%
31.6%
32.1%
29.0%
23.4%
26.3%
19.9%
15.9%
22.5%
39.6%
17.8%
24.2%
8.8%
7.7%
Intern-S1-Pro
29.4%
17.4%
26.8%
26.2%
10.5%
22.6%
27.8%
2.7%
13.7%
30.3%
3.2%
14.0%
7.3%
0.4%
InternVL 3.5-241B-A28B
32.9%
31.3%
34.1%
26.1%
17.5%
24.8%
50.1%
5.4%
16.2%
41.8%
5.2%
15.3%
13.0%
1.7%
InternVL 3.5-30B-A3B
27.3%
4.9%
20.0%
24.0%
7.0%
15.3%
35.4%
0.6%
7.6%
41.3%
1.3%
7.3%
9.1%
0.0%
InternVL 3.5-14B
22.8%
7.6%
22.2%
31.9%
7.5%
22.5%
28.5%
4.7%
12.9%
41.3%
5.0%
11.9%
8.7%
0.6%
InternVL 3.5-8B
28.5%
19.7%
34.2%
24.5%
9.5%
29.1%
24.7%
4.2%
12.3%
30.8%
2.5%
11.4%
7.2%
0.6%
InternVL 3.5-4B
25.6%
3.1%
19.8%
26.4%
12.0%
23.7%
22.4%
1.0%
8.1%
36.7%
1.0%
7.5%
7.1%
0.0%
Kimi K2.6
36.1%1
22.1%
28.5%
32.3%
12.2%
20.8%
42.8%
12.3%
21.9%
28.0%
10.8%
20.4%
11.3%
3.7%
Kimi K2.5
34.3%2
27.1%
30.4%
32.7%
18.5%
23.9%
39.4%
5.0%
15.6%
30.1%
4.7%
14.4%
10.7%
1.1%
MiMo-V2.5
33.1%3
43.3%2
38.3%3
27.3%
43.5%3
40.1%3
51.2%
26.6%2
30.7%2
49.7%
27.8%2
31.5%2
15.3%
19.0%2
Qwen3.8-27B
26.2%
34.7%
35.2%
28.8%
17.0%
26.1%
61.9%2
22.8%3
27.6%
49.4%
21.9%3
28.1%3
15.3%
8.5%3
Qwen3.6-27B
30.8%
36.0%
35.2%
36.4%3
13.2%
22.8%
60.4%3
21.7%
27.8%3
53.4%3
19.5%
27.3%
18.4%2
6.2%
Qwen3.5-397B-A17B
27.4%
3.3%
12.6%
35.8%
13.5%
23.5%
47.9%
2.8%
13.0%
46.2%
4.4%
13.9%
14.1%
0.2%
Qwen3.5-122B-A10B
32.6%
29.1%
31.6%
37.8%1
26.0%
31.6%
49.7%
6.1%
19.4%
49.0%
6.8%
18.8%
15.4%
2.1%
Qwen3.5-35B-A3B
29.5%
27.6%
30.5%
27.4%
0.6%
0.8%
53.2%
6.1%
14.5%
57.5%1
7.2%
18.5%
15.5%
0.0%
Qwen3.5-27B
31.6%
42.8%3
39.9%2
37.1%2
22.8%
28.5%
57.8%
7.3%
20.4%
50.6%
7.5%
21.0%
17.8%3
2.9%
Qwen3.5-9B
29.0%
15.9%
25.8%
29.0%
22.9%
32.5%
53.2%
10.3%
20.9%
52.2%
11.5%
22.1%
15.1%
2.8%
Qwen3.5-4B
30.9%
12.9%
23.2%
30.2%
12.4%
23.1%
47.5%
6.8%
17.8%
47.5%
6.3%
18.0%
15.4%
1.2%
Spatially Specialized Models
SpaceOm
24.7%1
2.8%2
6.7%2
24.5%2
4.2%1
11.4%1
35.2%3
2.4%1
8.6%1
32.8%
1.8%2
6.5%2
8.0%3
0.1%2
SpaceR
20.6%3
2.5%3
6.3%3
24.0%3
1.1%
1.8%
42.4%2
2.1%2
6.5%2
48.8%2
2.3%1
6.9%1
9.4%2
0.0%3
SpaceThinker
22.6%2
1.2%
3.0%
25.4%1
1.8%3
5.9%2
32.8%
0.6%
5.1%3
34.3%3
0.7%
4.7%3
7.9%
0.0%3
ViLaSR
18.8%
4.3%1
7.6%1
22.1%
2.0%2
3.5%3
43.1%1
1.9%3
3.5%
49.7%1
1.5%3
3.2%
9.7%1
0.3%1
Human
85.8%
77.9%
57.8%
94.6%
75.4%
59.5%
82.5%
53.8%
47.2%
82.5%
72.5%
57.5%
-
-
Image-task performance. Four image tasks; 45 models and the unranked Human baseline. Dual columns harmonically combine the position and action-observation pair scores. Scores are percentages. Within each group, model families are alphabetical, with newer versions first and larger models first within a version. Colors mark the three highest distinct values per metric, including ties and zero scores. Dashes indicate missing metrics. T: low reasoning effort, or Muse Spark's minimum available setting. Human questionnaire results are unranked; see Human Validation for the protocol.
Image coordination profile
The eight highest image Dual Semantic scores are shown below with their evidence counterparts. Both scores use the same two relation families; a semantic lead need not imply equally strong target localization.
Image Dual Semantic and Evidence Accuracy. Eight-model display subset, selected by unrounded image Dual Semantic Acc. The full table and correlations include all 45 models.
Image: paired semantics versus evidence. Each point represents one of 45 models. Pearson r = 0.650; Spearman rho = 0.520. Dashed lines mark the model means on each axis.
Across all image models, paired semantics and evidence are positively associated (r = 0.650), but their rank agreement is weaker (rho = 0.520). This complements the task-level examples: broad competence helps both outcomes, while the ordering of individual models still depends on whether decisions or localization are assessed.
Video Tasks
Behavior recognition, composition, and temporal grounding remain uneven
Gemini 3.8 Flash leads composite accuracy at 56.4%, whereas Muse Spark leads composite temporal F1@0.5 and mtIoU at 50.0% and 53.1%. MiMo-V2-Omni leads visibility-count accuracy at 59.4%. Qwen3.7-Plus reaches 33.8% atomic label F1 but 6.5% composite exact-set accuracy; the levels probe different demands and use different semantic metrics, so this contrast is not a like-for-like accuracy drop.
Qwen3.6-Plus leads video Dual Semantic Acc at 14.8%, while Muse Spark leads Dual Evidence Acc at 16.2%. In the paired semantic score, behavior success means exact agreement on the full answer set of the joint behavior question, not the Composite Acc column. Evidence success requires sample-level temporal F1@0.5 of at least 0.5 for visibility and every applicable behavior level.
#1 within group#2 within group#3 within group
Behavior and visibility tasks; 32 models and the unranked Human baseline. Dual columns use aligned source trajectories; temporal F1 is class-aware.
Model
Agent-State Reasoning
World-State Reasoning
Video Coordination
Composite Behavior
Atomic Behavior
Landmark Visibility
Behavior-Visibility
Acc
F1@0.5
mtIoU
Label F1
F1@0.5
mtIoU
Count Acc
F1@0.5
mtIoU
Dual Sem.
Dual Evi.
Proprietary Models
Gemini 3.8 FlashT
56.4%1
41.4%3
43.0%3
38.8%2
17.2%
14.6%
46.0%
25.3%
29.0%
6.9%
8.2%
Gemini 3.1 Pro PreviewT
54.8%2
40.2%
37.9%
32.9%
9.0%
8.6%
33.5%
20.4%
30.3%
1.2%
2.1%
Gemini 3.1 Flash Lite
23.7%
23.0%
26.0%
19.7%
11.6%
10.3%
52.8%
44.1%
42.2%
7.3%
5.5%
Gemini 3 Flash
48.3%
47.7%2
52.4%2
35.0%3
27.5%1
22.0%2
55.2%2
45.0%
44.3%
11.4%
12.6%2
MiMo-V2-Omni
26.9%
25.0%
26.9%
30.1%
21.3%
17.4%
59.4%1
52.7%
53.7%1
10.6%
11.5%3
Muse Spark 1.3 ContributorT
49.2%3
50.0%1
53.1%1
39.2%1
27.2%2
22.3%1
47.9%
48.0%
44.2%
12.6%
16.2%1
Qwen3.7-Plus
6.5%
6.2%
6.4%
33.8%
24.2%3
18.3%3
53.6%3
56.8%2
49.2%3
14.0%3
11.5%3
Qwen3.6-Plus
9.8%
9.3%
9.6%
30.1%
15.6%
15.3%
50.8%
52.2%
46.1%
14.8%1
8.5%
Qwen3.6-Flash
32.3%
28.1%
32.0%
24.9%
16.9%
12.9%
52.2%
52.7%
46.7%
11.5%
7.3%
Qwen3.5-Plus
2.4%
2.4%
2.5%
32.1%
17.5%
17.0%
53.2%
54.2%3
46.8%
14.4%2
9.1%
Qwen3.5-Flash
32.6%
29.8%
33.1%
26.9%
17.8%
13.9%
52.3%
58.3%1
51.3%2
9.3%
9.5%
Open-weight Models
GLM-5.3-FlashT
26.0%3
24.5%2
25.8%2
30.3%3
21.2%2
18.0%1
49.1%
43.9%
41.4%
11.8%3
9.4%
GLM-4.6V
31.2%2
1.2%
6.0%
32.2%1
7.5%
6.2%
31.3%
17.4%
33.8%
0.1%
0.5%
InternVL 3.5-30B-A3B
13.3%
1.9%
4.3%
24.5%
5.8%
5.4%
43.9%
8.2%
12.6%
7.2%
0.3%
InternVL 3.5-14B
23.4%
2.3%
7.3%
21.8%
3.9%
3.8%
54.5%2
14.4%
22.7%
3.0%
0.9%
InternVL 3.5-8B
22.4%
6.2%
10.6%
22.9%
4.2%
4.6%
51.9%
16.7%
21.3%
6.4%
0.3%
InternVL 3.5-4B
25.8%
2.3%
5.8%
20.7%
4.3%
3.4%
34.6%
17.1%
32.3%
0.4%
0.5%
Kimi K2.6
11.9%
10.7%
11.1%
31.4%2
20.0%
16.7%
39.9%
48.3%
43.3%
8.5%
7.8%
Kimi K2.5
13.3%
12.4%
13.2%
30.0%
21.2%3
17.6%
50.5%
50.3%3
45.8%
11.5%
10.2%1
MiMo-V2.5
35.6%1
33.4%1
36.9%1
29.5%
21.4%1
18.0%2
58.5%1
54.4%1
50.4%2
11.9%2
10.0%2
Qwen3.8-27B
15.3%
6.8%
7.5%
27.8%
12.4%
8.6%
43.2%
16.3%
23.6%
8.9%
0.5%
Qwen3.6-27B
19.9%
12.8%
14.1%
29.1%
18.9%
15.8%
52.7%
43.3%
42.3%
13.4%1
7.7%
Qwen3.5-397B-A17B
5.2%
1.0%
1.5%
26.6%
10.3%
8.4%
40.4%
21.2%
31.8%
5.7%
1.9%
Qwen3.5-122B-A10B
21.0%
17.8%3
19.9%3
23.4%
13.3%
11.9%
48.4%
46.7%
49.7%3
5.4%
6.5%
Qwen3.5-35B-A3B
21.0%
15.7%
16.7%
27.7%
17.8%
16.0%
54.1%3
50.0%
48.6%
8.9%
9.0%
Qwen3.5-27B
14.6%
13.4%
14.3%
29.7%
20.1%
17.7%3
51.4%
52.2%2
48.7%
11.2%
9.7%3
Qwen3.5-9B
9.5%
7.3%
8.3%
28.2%
17.8%
16.0%
50.7%
33.8%
36.8%
8.0%
6.0%
Qwen3.5-4B
6.4%
1.1%
1.1%
10.0%
0.9%
0.8%
33.9%
45.2%
52.0%1
0.6%
0.4%
Spatially Specialized Models
SpaceOm
26.5%1
0.0%1
2.5%1
25.9%2
4.0%1
3.1%2
32.3%3
17.6%1
34.2%1
0.3%
0.0%1
SpaceR
21.4%2
0.0%1
1.9%2
26.2%1
4.0%2
3.5%1
33.7%2
16.5%2
30.8%2
0.4%3
0.0%1
SpaceThinker
17.2%3
0.0%1
1.3%3
23.1%3
3.7%3
2.6%3
33.7%2
14.3%3
27.1%3
0.9%2
0.0%1
ViLaSR
5.2%
0.0%1
0.4%
9.9%
0.4%
1.1%
35.9%1
10.4%
20.6%
1.6%1
0.0%1
Human
83.9%
62.9%
53.3%
76.3%
83.2%
61.6%
95.0%
96.7%
83.7%
-
-
Video-task performance. Behavior and visibility tasks; 32 models and the unranked Human baseline. Dual columns use aligned source trajectories; temporal F1 is class-aware. Scores are percentages. Within each group, model families are alphabetical, with newer versions first and larger models first within a version. Colors mark the three highest distinct values per metric, including ties and zero scores. Dashes indicate missing metrics. T: low reasoning effort, or Muse Spark's minimum available setting. Human questionnaire results are unranked; see Human Validation for the protocol.
Video coordination profile
The semantic-leading models differ in their ability to localize both processes along the trajectory. Composite and atomic intervals are matched one-to-one within their behavior classes; extra predicted classes count as false positives. A high visible-count score alone does not establish complete visibility intervals.
Video Dual Semantic and Evidence Accuracy. Eight-model display subset, selected by unrounded video Dual Semantic Acc. The full table and correlations include all 32 video models.
Video: paired semantics versus evidence. Each point represents one of 32 models. Pearson r = 0.830; Spearman rho = 0.805. Dashed lines mark the model means on each axis.
The video association is stronger (r = 0.830, rho = 0.805), yet the two metrics still have different leaders. Positive correlation across models is compatible with low absolute paired success and does not make semantic and temporal evaluation redundant.
Cross-media Comparison
Image coordination and video coordination are not interchangeable
Each point below is one of the 32 models with both image and video Dual Semantic Acc. Gemini 3.8 Flash leads image coordination, while Qwen3.6-Plus leads video paired semantics. Comparing the two scores reveals how spatial and temporal coordination vary together across models.
The linear association is modest (r = 0.274), while rank association is higher (rho = 0.493). Some ordering is shared, but image-based coordination is not a reliable substitute for evaluating temporal coordination directly.
Cross-media Dual Semantic Accuracy. Each point represents one of 32 models. Pearson r = 0.274; Spearman rho = 0.493. Dashed lines mark the model means on each axis.
Correlation summary
Pearson correlation measures linear association; Spearman correlation compares ranks using average ranks for ties. Calculations use unrounded model scores, retain zeros, and include every available finite metric pair. These coefficients summarize model-level associations within the evaluated cohort.
Comparison
Models
Pearson r
Spearman rho
Image: Dual Semantic vs Dual Evidence
45
0.650
0.520
Video: Dual Semantic vs Dual Evidence
32
0.830
0.805
Cross-media: Image vs Video Dual Semantic
32
0.274
0.493
Associations within and across observation media. Each model contributes one equally weighted observation. The coefficients compare model-level Dual scores. Download tables and statistics.
Single-task semantic-evidence correlations
These secondary diagnostics retain the task-specific definitions in Tables II and III. Image evidence is BBox Acc@0.5; video evidence is temporal F1@0.5. Atomic semantics use label-set F1. They are not inputs to the cross-media Dual aggregate.
Task or level
Models
Pearson r
Spearman rho
Landmark-relative Position
45
0.482
0.307
Future Observation
45
0.586
0.478
Self-relative Position
45
0.581
0.675
Action Decision
45
0.486
0.509
Composite Behavior
32
0.821
0.630
Atomic Behavior
32
0.739
0.733
Landmark Visibility
32
0.659
0.588
Single-task semantic-evidence associations. Pearson and Spearman coefficients compare each task's semantic metric with its spatial or temporal grounding metric across available models.
Agent-State & World-State
Complementary state estimates must succeed in the same context
Agent-State tasks concern the UAV's position, action consequences, and flight behavior. World-State tasks concern self-relative position, goal-conditioned action, and visibility. The two tables below regroup the same single-task results for the common 32-model cohort. Their shared Overall Dual columns place each state's task performance alongside coordinated Agent-World success.
Agent-State Reasoning
High Future Observation accuracy can coexist with weak temporal behavior recognition. The two image tasks and both behavior levels reveal complementary aspects of agent-state estimation, with each task retaining its own semantic and grounding metrics.
#1 within group#2 within group#3 within group
32 models evaluated in both media. Single-task Agent-State metrics are followed by the shared cross-media Agent-World Dual scores.
Model
Image Agent-State Tasks
Video Agent-State Tasks
Shared Coordination
Landmark-relative Position
Future Observation
Composite Behavior
Atomic Behavior
Cross-media Overall
Acc
BBox@0.5
mIoU
Acc
BBox@0.5
mIoU
Acc
F1@0.5
mtIoU
Label F1
F1@0.5
mtIoU
Dual Sem.
Dual Evi.
Proprietary Models
Gemini 3.8 FlashT
60.1%1
66.7%1
57.6%1
74.9%1
73.8%1
62.5%1
56.4%1
41.4%3
43.0%3
38.8%2
17.2%
14.6%
12.0%
13.2%1
Gemini 3.1 Pro PreviewT
40.9%3
58.8%2
50.5%2
58.8%2
59.9%2
52.3%2
54.8%2
40.2%
37.9%
32.9%
9.0%
8.6%
2.2%
3.8%
Gemini 3.1 Flash Lite
33.7%
48.1%3
45.7%3
30.1%
47.9%3
46.0%3
23.7%
23.0%
26.0%
19.7%
11.6%
10.3%
9.2%
5.5%
Gemini 3 Flash
47.6%2
33.6%
33.8%
45.9%
37.5%
38.5%
48.3%
47.7%2
52.4%2
35.0%3
27.5%1
22.0%2
15.7%2
3.4%
MiMo-V2-Omni
28.7%
40.7%
36.5%
21.3%
36.9%
33.4%
26.9%
25.0%
26.9%
30.1%
21.3%
17.4%
10.0%
6.5%
Muse Spark 1.3 ContributorT
38.7%
20.5%
26.7%
48.6%3
15.0%
21.2%
49.2%3
50.0%1
53.1%1
39.2%1
27.2%2
22.3%1
17.4%1
7.1%3
Qwen3.7-Plus
33.3%
18.4%
28.0%
34.6%
21.4%
30.3%
6.5%
6.2%
6.4%
33.8%
24.2%3
18.3%3
14.8%3
8.3%2
Qwen3.6-Plus
32.8%
16.6%
27.1%
33.5%
14.7%
26.1%
9.8%
9.3%
9.6%
30.1%
15.6%
15.3%
14.3%
2.6%
Qwen3.6-Flash
25.1%
24.6%
32.0%
28.7%
19.2%
29.8%
32.3%
28.1%
32.0%
24.9%
16.9%
12.9%
13.4%
6.6%
Qwen3.5-Plus
28.6%
21.4%
27.1%
29.0%
12.3%
24.1%
2.4%
2.4%
2.5%
32.1%
17.5%
17.0%
13.5%
2.7%
Qwen3.5-Flash
27.8%
21.6%
29.8%
28.9%
15.8%
23.1%
32.6%
29.8%
33.1%
26.9%
17.8%
13.9%
11.5%
3.7%
Open-weight Models
GLM-5.3-FlashT
32.4%
38.4%
38.1%
35.2%
45.4%1
43.8%1
26.0%3
24.5%2
25.8%2
30.3%3
21.2%2
18.0%1
14.8%2
12.9%2
GLM-4.6V
30.8%
47.5%1
43.0%1
27.1%
44.9%2
42.3%2
31.2%2
1.2%
6.0%
32.2%1
7.5%
6.2%
0.2%
0.9%
InternVL 3.5-30B-A3B
27.3%
4.9%
20.0%
24.0%
7.0%
15.3%
13.3%
1.9%
4.3%
24.5%
5.8%
5.4%
8.0%
0.0%
InternVL 3.5-14B
22.8%
7.6%
22.2%
31.9%
7.5%
22.5%
23.4%
2.3%
7.3%
21.8%
3.9%
3.8%
4.5%
0.7%
InternVL 3.5-8B
28.5%
19.7%
34.2%
24.5%
9.5%
29.1%
22.4%
6.2%
10.6%
22.9%
4.2%
4.6%
6.8%
0.4%
InternVL 3.5-4B
25.6%
3.1%
19.8%
26.4%
12.0%
23.7%
25.8%
2.3%
5.8%
20.7%
4.3%
3.4%
0.7%
0.0%
Kimi K2.6
36.1%1
22.1%
28.5%
32.3%
12.2%
20.8%
11.9%
10.7%
11.1%
31.4%2
20.0%
16.7%
9.7%
5.0%
Kimi K2.5
34.3%2
27.1%
30.4%
32.7%
18.5%
23.9%
13.3%
12.4%
13.2%
30.0%
21.2%3
17.6%
11.1%
2.0%
MiMo-V2.5
33.1%3
43.3%2
38.3%3
27.3%
43.5%3
40.1%3
35.6%1
33.4%1
36.9%1
29.5%
21.4%1
18.0%2
13.4%
13.1%1
Qwen3.8-27B
26.2%
34.7%
35.2%
28.8%
17.0%
26.1%
15.3%
6.8%
7.5%
27.8%
12.4%
8.6%
11.2%
0.9%
Qwen3.6-27B
30.8%
36.0%
35.2%
36.4%3
13.2%
22.8%
19.9%
12.8%
14.1%
29.1%
18.9%
15.8%
15.5%1
6.9%3
Qwen3.5-397B-A17B
27.4%
3.3%
12.6%
35.8%
13.5%
23.5%
5.2%
1.0%
1.5%
26.6%
10.3%
8.4%
8.1%
0.3%
Qwen3.5-122B-A10B
32.6%
29.1%
31.6%
37.8%1
26.0%
31.6%
21.0%
17.8%3
19.9%3
23.4%
13.3%
11.9%
8.0%
3.2%
Qwen3.5-35B-A3B
29.5%
27.6%
30.5%
27.4%
0.6%
0.8%
21.0%
15.7%
16.7%
27.7%
17.8%
16.0%
11.3%
0.0%
Qwen3.5-27B
31.6%
42.8%3
39.9%2
37.1%2
22.8%
28.5%
14.6%
13.4%
14.3%
29.7%
20.1%
17.7%3
13.8%3
4.5%
Qwen3.5-9B
29.0%
15.9%
25.8%
29.0%
22.9%
32.5%
9.5%
7.3%
8.3%
28.2%
17.8%
16.0%
10.5%
3.8%
Qwen3.5-4B
30.9%
12.9%
23.2%
30.2%
12.4%
23.1%
6.4%
1.1%
1.1%
10.0%
0.9%
0.8%
1.1%
0.6%
Spatially Specialized Models
SpaceOm
24.7%1
2.8%2
6.7%2
24.5%2
4.2%1
11.4%1
26.5%1
0.0%1
2.5%1
25.9%2
4.0%1
3.1%2
0.6%
0.0%1
SpaceR
20.6%3
2.5%3
6.3%3
24.0%3
1.1%
1.8%
21.4%2
0.0%1
1.9%2
26.2%1
4.0%2
3.5%1
0.8%3
0.0%1
SpaceThinker
22.6%2
1.2%
3.0%
25.4%1
1.8%3
5.9%2
17.2%3
0.0%1
1.3%3
23.1%3
3.7%3
2.6%3
1.6%2
0.0%1
ViLaSR
18.8%
4.3%1
7.6%1
22.1%
2.0%2
3.5%3
5.2%
0.0%1
0.4%
9.9%
0.4%
1.1%
2.7%1
0.0%1
Human
85.8%
77.9%
57.8%
94.6%
75.4%
59.5%
83.9%
62.9%
53.3%
76.3%
83.2%
61.6%
-
-
Agent-State Reasoning performance. 32 models evaluated in both media. Single-task Agent-State metrics are followed by the shared cross-media Agent-World Dual scores. Scores are percentages. Within each group, model families are alphabetical, with newer versions first and larger models first within a version. Colors mark the three highest distinct values per metric, including ties and zero scores. Dashes indicate missing metrics. T: low reasoning effort, or Muse Spark's minimum available setting. Human questionnaire results are unranked; see Human Validation for the protocol.
World-State Reasoning
Self-relative decisions and visibility counts are often stronger than their paired Agent-State decisions, but evidence quality follows a different ordering. The image columns distinguish choosing a direction or action from localizing its target; the video columns distinguish counting appearances from finding the corresponding intervals.
#1 within group#2 within group#3 within group
The same 32-model cross-media cohort. World-State task metrics are followed by the shared cross-media Agent-World Dual scores.
Model
Image World-State Tasks
Video World-State Task
Shared Coordination
Self-relative Position
Action Decision
Landmark Visibility
Cross-media Overall
Acc
BBox@0.5
mIoU
Acc
BBox@0.5
mIoU
Count Acc
F1@0.5
mtIoU
Dual Sem.
Dual Evi.
Proprietary Models
Gemini 3.8 FlashT
61.9%
40.8%1
39.0%1
69.5%1
42.3%1
39.3%1
46.0%
25.3%
29.0%
12.0%
13.2%1
Gemini 3.1 Pro PreviewT
63.7%2
37.5%2
36.8%2
47.2%
37.1%2
36.4%2
33.5%
20.4%
30.3%
2.2%
3.8%
Gemini 3.1 Flash Lite
47.6%
9.7%
21.1%
35.9%
9.3%
20.8%
52.8%
44.1%
42.2%
9.2%
5.5%
Gemini 3 Flash
56.2%
5.0%
13.5%
51.3%
5.1%
12.2%
55.2%2
45.0%
44.3%
15.7%2
3.4%
MiMo-V2-Omni
36.5%
8.8%
17.6%
38.2%
9.2%
17.3%
59.4%1
52.7%
53.7%1
10.0%
6.5%
Muse Spark 1.3 ContributorT
64.2%1
14.8%
25.5%
64.4%2
14.9%
24.9%
47.9%
48.0%
44.2%
17.4%1
7.1%3
Qwen3.7-Plus
55.4%
18.4%
26.7%
43.4%
18.8%3
27.0%3
53.6%3
56.8%2
49.2%3
14.8%3
8.3%2
Qwen3.6-Plus
48.4%
7.4%
19.0%
42.5%
6.0%
19.9%
50.8%
52.2%
46.1%
14.3%
2.6%
Qwen3.6-Flash
62.7%3
19.4%3
27.1%3
56.7%3
18.6%
25.8%
52.2%
52.7%
46.7%
13.4%
6.6%
Qwen3.5-Plus
47.6%
9.7%
19.3%
48.9%
8.3%
20.0%
53.2%
54.2%3
46.8%
13.5%
2.7%
Qwen3.5-Flash
52.4%
8.8%
19.4%
54.5%
8.9%
20.1%
52.3%
58.3%1
51.3%2
11.5%
3.7%
Open-weight Models
GLM-5.3-FlashT
63.6%1
32.7%1
33.5%1
57.4%2
33.0%1
34.1%1
49.1%
43.9%
41.4%
14.8%2
12.9%2
GLM-4.6V
29.2%
9.5%
20.0%
44.2%
10.2%
20.6%
31.3%
17.4%
33.8%
0.2%
0.9%
InternVL 3.5-30B-A3B
35.4%
0.6%
7.6%
41.3%
1.3%
7.3%
43.9%
8.2%
12.6%
8.0%
0.0%
InternVL 3.5-14B
28.5%
4.7%
12.9%
41.3%
5.0%
11.9%
54.5%2
14.4%
22.7%
4.5%
0.7%
InternVL 3.5-8B
24.7%
4.2%
12.3%
30.8%
2.5%
11.4%
51.9%
16.7%
21.3%
6.8%
0.4%
InternVL 3.5-4B
22.4%
1.0%
8.1%
36.7%
1.0%
7.5%
34.6%
17.1%
32.3%
0.7%
0.0%
Kimi K2.6
42.8%
12.3%
21.9%
28.0%
10.8%
20.4%
39.9%
48.3%
43.3%
9.7%
5.0%
Kimi K2.5
39.4%
5.0%
15.6%
30.1%
4.7%
14.4%
50.5%
50.3%3
45.8%
11.1%
2.0%
MiMo-V2.5
51.2%
26.6%2
30.7%2
49.7%
27.8%2
31.5%2
58.5%1
54.4%1
50.4%2
13.4%
13.1%1
Qwen3.8-27B
61.9%2
22.8%3
27.6%
49.4%
21.9%3
28.1%3
43.2%
16.3%
23.6%
11.2%
0.9%
Qwen3.6-27B
60.4%3
21.7%
27.8%3
53.4%3
19.5%
27.3%
52.7%
43.3%
42.3%
15.5%1
6.9%3
Qwen3.5-397B-A17B
47.9%
2.8%
13.0%
46.2%
4.4%
13.9%
40.4%
21.2%
31.8%
8.1%
0.3%
Qwen3.5-122B-A10B
49.7%
6.1%
19.4%
49.0%
6.8%
18.8%
48.4%
46.7%
49.7%3
8.0%
3.2%
Qwen3.5-35B-A3B
53.2%
6.1%
14.5%
57.5%1
7.2%
18.5%
54.1%3
50.0%
48.6%
11.3%
0.0%
Qwen3.5-27B
57.8%
7.3%
20.4%
50.6%
7.5%
21.0%
51.4%
52.2%2
48.7%
13.8%3
4.5%
Qwen3.5-9B
53.2%
10.3%
20.9%
52.2%
11.5%
22.1%
50.7%
33.8%
36.8%
10.5%
3.8%
Qwen3.5-4B
47.5%
6.8%
17.8%
47.5%
6.3%
18.0%
33.9%
45.2%
52.0%1
1.1%
0.6%
Spatially Specialized Models
SpaceOm
35.2%3
2.4%1
8.6%1
32.8%
1.8%2
6.5%2
32.3%3
17.6%1
34.2%1
0.6%
0.0%1
SpaceR
42.4%2
2.1%2
6.5%2
48.8%2
2.3%1
6.9%1
33.7%2
16.5%2
30.8%2
0.8%3
0.0%1
SpaceThinker
32.8%
0.6%
5.1%3
34.3%3
0.7%
4.7%3
33.7%2
14.3%3
27.1%3
1.6%2
0.0%1
ViLaSR
43.1%1
1.9%3
3.5%
49.7%1
1.5%3
3.2%
35.9%1
10.4%
20.6%
2.7%1
0.0%1
Human
82.5%
53.8%
47.2%
82.5%
72.5%
57.5%
95.0%
96.7%
83.7%
-
-
World-State Reasoning performance. The same 32-model cross-media cohort. World-State task metrics are followed by the shared cross-media Agent-World Dual scores. Scores are percentages. Within each group, model families are alphabetical, with newer versions first and larger models first within a version. Colors mark the three highest distinct values per metric, including ties and zero scores. Dashes indicate missing metrics. T: low reasoning effort, or Muse Spark's minimum available setting. Human questionnaire results are unranked; see Human Validation for the protocol.
Matched-domain marginal success and Dual outcomes
For a fair Agent-World comparison, the following diagnostics use the two binary success events from the same aligned records. Values are equally weighted means over the common model cohort, with each relation reported separately. Video behavior semantics here use the full joint answer set, and evidence values are thresholded success rates, not mean F1 or mean IoU.
Pair
Success event
Models
Agent marginal
World marginal
Dual co-success
Independence reference
Position
Semantic
32
31.03%
47.45%
14.54%
15.16%
Position
Evidence
32
24.93%
11.49%
6.13%
4.20%
Action-Observation
Semantic
32
32.95%
46.36%
15.91%
15.82%
Action-Observation
Evidence
32
21.64%
11.44%
5.25%
3.90%
Behavior-Visibility
Semantic
32
12.16%
46.30%
7.50%
5.92%
Behavior-Visibility
Evidence
32
10.40%
41.25%
5.73%
5.20%
State-wise success within matched pair domains. Values are means over the common model cohort. The independence reference averages each model's P(Agent success) x P(World success), calculated on its own aligned records.
In this matched cohort, World-State semantic marginals exceed Agent-State marginals in all three relations. Grounding has a different pattern: Agent-State evidence is stronger in both image relations, but the behavior-side evidence marginal is much lower than visibility evidence in video. The bottleneck therefore depends on both the state being assessed and the observation medium.
Dual success is bounded by the weaker marginal on the same sample domain. Comparing it with both marginals and their independence reference helps interpret shared successes and errors, including dependence associated with sample difficulty.
Position: state-specific and paired semantic success. Each point is one model on the same aligned records; dashed lines mark model means. The marginal contributes directly to Dual success, so their association includes this mathematical dependence.
Future Observation
Action-Observation: marginal versus Dual semantics
Action-Observation: state-specific and paired semantic success. Each point is one model on the same aligned records; dashed lines mark model means. The marginal contributes directly to Dual success, so their association includes this mathematical dependence.
Flight Behavior
Behavior-Visibility: marginal versus Dual semantics
Behavior-Visibility: state-specific and paired semantic success. Each point is one model on the same aligned records; dashed lines mark model means. The marginal contributes directly to Dual success, so their association includes this mathematical dependence.
State-to-Dual correlations
Both semantic and evidence success are included below. The plots show the semantic comparison for the selected relation; evidence correlations use the corresponding grounding-success events. Undefined correlations, such as a constant metric, are reported as dashes.
Pair
Event
Models
Agent / Dual r
Agent / Dual rho
World / Dual r
World / Dual rho
Position
Sem
32
0.877
0.718
0.791
0.918
Position
Evi
32
0.775
0.832
0.970
0.964
Action-Observation
Sem
32
0.931
0.798
0.762
0.812
Action-Observation
Evi
32
0.836
0.863
0.928
0.928
Behavior-Visibility
Sem
32
0.915
0.887
0.811
0.699
Behavior-Visibility
Evi
32
0.976
0.951
0.812
0.768
Agent-State and World-State associations with Dual performance. The same 32 models and matched domains are used for both states. Coefficients use unrounded inputs and average ranks for ties; dashes denote undefined correlations.
Dual Coordination
Three relations, distinct semantic and evidence profiles
The position pair probes UAV-centric and landmark-centric geometry; action-observation links an action with its visual consequence; behavior-visibility links UAV motion with the landmark's appearances along a trajectory. These relations matter for relative-position navigation, viewpoint adjustment, and persistent target observation.
Muse Spark leads cross-media Dual Semantic Acc at 17.40%, while Gemini 3.8 Flash leads Dual Evidence Acc at 13.24%. GLM-5.3-Flash leads both image pairings among open-weight models. The full table below extends the paper's groupwise top-five selection to all eligible models, exposing strengths that a single ranking would obscure.
Image pairs share scene, Agent-Landmark identity, and difficulty, with potentially different camera poses. The resulting scores characterize matched-context coordination across related tasks. Harmonic aggregation emphasizes balanced success across the image relations and the video trajectory.
#1 within group#2 within group#3 within group
Complete Table IV view: all 32 models with paired image and video records. Overall = H(H(Position, Action-Observation), Behavior-Visibility), separately for Sem. and Evi.
Model
Image Coordination
Video Coordination
Cross-media
Position
Action-Observation
Behavior-Visibility
Overall
Dual Sem.
Dual Evi.
Dual Sem.
Dual Evi.
Dual Sem.
Dual Evi.
Dual Sem.
Dual Evi.
Proprietary Models
Gemini 3.8 FlashT
38.3%1
32.5%1
52.8%1
36.2%1
6.9%
8.2%
12.0%
13.2%1
Gemini 3.1 Pro PreviewT
27.0%2
29.5%2
27.1%3
30.8%2
1.2%
2.1%
2.2%
3.8%
Gemini 3.1 Flash Lite
15.5%
6.1%
10.3%
5.0%
7.3%
5.5%
9.2%
5.5%
Gemini 3 Flash
25.6%3
1.9%
24.3%
2.1%
11.4%
12.6%2
15.7%2
3.4%
MiMo-V2-Omni
10.0%
5.3%
9.0%
4.0%
10.6%
11.5%3
10.0%
6.5%
Muse Spark 1.3 ContributorT
24.7%
6.7%
32.6%2
3.4%
12.6%
16.2%1
17.4%1
7.1%3
Qwen3.7-Plus
16.4%
6.5%
15.1%
6.4%3
14.0%3
11.5%3
14.8%3
8.3%2
Qwen3.6-Plus
13.5%
1.7%
14.0%
1.5%
14.8%1
8.5%
14.3%
2.6%
Qwen3.6-Flash
15.9%
7.4%3
16.0%
5.1%
11.5%
7.3%
13.4%
6.6%
Qwen3.5-Plus
11.2%
3.0%
14.6%
1.1%
14.4%2
9.1%
13.5%
2.7%
Qwen3.5-Flash
14.3%
2.6%
15.9%
2.1%
9.3%
9.5%
11.5%
3.7%
Open-weight Models
GLM-5.3-FlashT
19.3%1
20.7%1
20.5%1
20.5%1
11.8%3
9.4%
14.8%2
12.9%2
GLM-4.6V
8.3%
5.2%
12.2%
5.4%
0.1%
0.5%
0.2%
0.9%
InternVL 3.5-30B-A3B
9.3%
0.0%
8.9%
0.1%
7.2%
0.3%
8.0%
0.0%
InternVL 3.5-14B
6.4%
0.7%
13.5%
0.5%
3.0%
0.9%
4.5%
0.7%
InternVL 3.5-8B
6.6%
1.3%
7.8%
0.4%
6.4%
0.3%
6.8%
0.4%
InternVL 3.5-4B
5.5%
0.1%
10.0%
0.0%
0.4%
0.5%
0.7%
0.0%
Kimi K2.6
14.6%
5.3%
9.2%
2.8%
8.5%
7.8%
9.7%
5.0%
Kimi K2.5
11.9%
1.6%
9.7%
0.9%
11.5%
10.2%1
11.1%
2.0%
MiMo-V2.5
16.2%
18.9%2
14.6%
19.1%2
11.9%2
10.0%2
13.4%
13.1%1
Qwen3.8-27B
15.8%
13.1%3
14.7%
6.3%3
8.9%
0.5%
11.2%
0.9%
Qwen3.6-27B
17.5%2
13.0%
19.3%2
4.1%
13.4%1
7.7%
15.5%1
6.9%3
Qwen3.5-397B-A17B
12.0%
0.1%
17.0%
1.3%
5.7%
1.9%
8.1%
0.3%
Qwen3.5-122B-A10B
13.5%
2.5%
18.0%
1.9%
5.4%
6.5%
8.0%
3.2%
Qwen3.5-35B-A3B
15.3%
2.4%
15.7%
0.0%
8.9%
9.0%
11.3%
0.0%
Qwen3.5-27B
17.4%3
3.8%
18.2%3
2.3%
11.2%
9.7%3
13.8%3
4.5%
Qwen3.5-9B
14.4%
2.4%
15.8%
3.2%
8.0%
6.0%
10.5%
3.8%
Qwen3.5-4B
15.4%
1.1%
15.4%
1.3%
0.6%
0.4%
1.1%
0.6%
Spatially Specialized Models
SpaceOm
8.4%2
0.2%2
7.6%
0.1%2
0.3%
0.0%1
0.6%
0.0%1
SpaceR
8.1%3
0.2%2
11.2%1
0.0%3
0.4%3
0.0%1
0.8%3
0.0%1
SpaceThinker
7.5%
0.0%3
8.3%3
0.0%3
0.9%2
0.0%1
1.6%2
0.0%1
ViLaSR
9.6%1
0.5%1
9.8%2
0.2%1
1.6%1
0.0%1
2.7%1
0.0%1
Complete dual-cognition coordination results. Complete Table IV view: all 32 models with paired image and video records. Overall = H(H(Position, Action-Observation), Behavior-Visibility), separately for Sem. and Evi. Scores are percentages. Within each group, model families are alphabetical, with newer versions first and larger models first within a version. Colors mark the three highest distinct values per metric, including ties and zero scores. Dashes indicate missing metrics. T: low reasoning effort, or Muse Spark's minimum available setting.
Aggregation and coverage
For either Sem. or Evi., let P, A, and B be the position, action-observation, and behavior-visibility pair scores. With H(x,y) = 2xy / (x+y), and H(0,0) = 0, the image score is H(P,A), and Overall is H(H(P,A),B). The nested formula gives equal weight to the two media after combining the image relations.
Pair domain
Models
Minimum aligned pairs
Maximum aligned pairs
Position
45
1024
1024
Action-Observation
45
1024
1024
Behavior-Visibility
32
1024
1024
Coverage of the three paired evaluations. Counts refer to aligned records available for each model; invalid answers remain in the denominator as failures. Human results are reported at the individual-task level.
Capability profiles
The radar resolves both outcome types for each relation. Its six axes use a shared percentage scale and the paired definitions in the complete table, making each model's relative strengths directly comparable.
Dual-Cognition Coordination
Paired semantic and evidence profiles across three coordination demands
Six axes report paired co-success for reciprocal position reasoning, action–observation association,
and behavior–visibility consistency in matched Agent–Landmark contexts.
Coordination remains profile-dependent
The six axes organize three paired demands under the same Agent–Landmark context. Relative-position
coordination tests reciprocal UAV- and landmark-centric judgments; action–observation coordination
links a commanded motion to its expected view and the action required for a target view; and
behavior–visibility coordination couples UAV motion with landmark appearance over time.
Semantic and evidence co-success are reported independently for each pair. A broad, balanced polygon
indicates higher co-success across the measured relations, whereas a narrow spike exposes
task-specific strength. Gemini 3.8 Flash is strongest on the image pairings, while the compact video
axes are led by different model families.
Exact values, aligned-record counts, and the complete model table are reported in the paired audit.
Dual-cognition capability profiles. Six model series compare semantic and evidence co-success across position, action-observation, and behavior-visibility pairs. All axes use the same percentage scale; the table above provides the exact values.
Model-level matched-domain and conditional audit
Counts, both marginals, observed co-success, independence references, and binary phi coefficients are retained per model and relation. Conditional columns report semantic P(Agent correct | World correct) and its reverse. They are undefined when the conditioning side has no successes. Phi is a within-model, across-pair association, unlike the across-model correlations above.
Position
Model
Pairs
Agent Sem.
World Sem.
Dual Sem.
Indep. Sem.
Sem. phi
A given W
W given A
Agent Evi.
World Evi.
Dual Evi.
Indep. Evi.
Evi. phi
Claude Opus 4.8
1024
39.36%
54.10%
19.34%
21.29%
-0.080
35.74%
49.13%
11.43%
10.45%
2.44%
1.19%
0.128
Claude Opus 4.7
1024
39.26%
49.71%
19.24%
19.51%
-0.011
38.70%
49.00%
8.20%
7.13%
1.17%
0.58%
0.083
Claude Opus 4.6
1024
38.09%
62.01%
22.56%
23.62%
-0.045
36.38%
59.23%
2.15%
8.59%
0.00%
0.18%
-0.045
Claude Sonnet 4.6
1024
37.70%
48.54%
16.11%
18.30%
-0.090
33.20%
42.75%
4.88%
9.57%
0.29%
0.47%
-0.027
Gemini 3.8 Flash
1024
60.06%
61.91%
38.28%
37.18%
0.046
61.83%
63.74%
66.70%
40.82%
32.52%
27.23%
0.228
Gemini 3.1 Pro Preview
1024
40.92%
63.67%
26.95%
26.05%
0.038
42.33%
65.87%
58.79%
37.50%
29.49%
22.05%
0.312
Gemini 3.1 Flash Lite
1024
33.69%
47.56%
15.53%
16.02%
-0.021
32.65%
46.09%
48.14%
9.67%
6.05%
4.65%
0.095
Gemini 3 Flash
1024
47.56%
56.15%
25.59%
26.71%
-0.045
45.57%
53.80%
33.59%
4.98%
1.86%
1.67%
0.018
GPT-6 Astra
1024
35.06%
57.23%
18.55%
20.06%
-0.064
32.42%
52.92%
27.83%
16.11%
7.32%
4.48%
0.172
GPT-5.6 Sol
1024
34.08%
55.66%
18.36%
18.97%
-0.026
32.98%
53.87%
23.93%
12.89%
5.57%
3.08%
0.174
GPT-5.6 Terra
1024
33.69%
58.89%
19.53%
19.84%
-0.013
33.17%
57.97%
24.80%
14.45%
7.13%
3.59%
0.233
GPT-5.5
1024
37.99%
65.43%
25.20%
24.86%
0.015
38.51%
66.32%
19.43%
17.58%
6.64%
3.42%
0.214
GPT-5.4
1024
37.89%
37.79%
14.36%
14.32%
0.002
37.98%
37.89%
12.21%
12.89%
3.91%
1.57%
0.213
GPT-5.3
1024
35.16%
56.54%
18.95%
19.88%
-0.039
33.51%
53.89%
11.91%
14.94%
2.93%
1.78%
0.100
MiMo-V2-Omni
1024
28.71%
36.52%
9.96%
10.49%
-0.024
27.27%
34.69%
40.72%
8.79%
5.27%
3.58%
0.122
Muse Spark 1.3 Contributor
1024
38.67%
64.16%
24.71%
24.81%
-0.004
38.51%
63.89%
20.51%
14.84%
6.74%
3.04%
0.257
Qwen3.7-Plus
1024
33.30%
55.37%
16.41%
18.44%
-0.087
29.63%
49.27%
18.36%
18.36%
6.54%
3.37%
0.212
Qwen3.6-Plus
1024
32.81%
48.44%
13.48%
15.89%
-0.103
27.82%
41.07%
16.60%
7.42%
1.66%
1.23%
0.044
Qwen3.6-Flash
1024
25.10%
62.70%
15.92%
15.74%
0.009
25.39%
63.42%
24.61%
19.43%
7.42%
4.78%
0.155
Qwen3.5-Plus
1024
28.61%
47.56%
11.23%
13.61%
-0.105
23.61%
39.25%
21.39%
9.67%
3.03%
2.07%
0.079
Qwen3.5-Flash
1024
27.83%
52.44%
14.26%
14.60%
-0.015
27.19%
51.23%
21.58%
8.79%
2.64%
1.90%
0.064
GLM-5.3-Flash
1024
32.42%
63.57%
19.34%
20.61%
-0.057
30.41%
59.64%
38.38%
32.71%
20.70%
12.56%
0.357
GLM-4.6V
1024
30.76%
29.20%
8.30%
8.98%
-0.032
28.43%
26.98%
47.46%
9.47%
5.18%
4.50%
0.047
GLM-4.5V
1024
31.93%
19.92%
7.13%
6.36%
0.041
35.78%
22.32%
31.64%
15.92%
9.18%
5.04%
0.244
Intern-S1-Pro
1024
29.39%
27.83%
7.62%
8.18%
-0.028
27.37%
25.91%
17.38%
2.73%
0.49%
0.48%
0.002
InternVL 3.5-241B-A28B
1024
32.91%
50.10%
15.72%
16.49%
-0.033
31.38%
47.77%
31.35%
5.37%
3.13%
1.68%
0.138
InternVL 3.5-30B-A3B
1024
27.34%
35.35%
9.28%
9.67%
-0.018
26.24%
33.93%
4.88%
0.59%
0.00%
0.03%
-0.017
InternVL 3.5-14B
1024
22.75%
28.52%
6.45%
6.49%
-0.002
22.60%
28.33%
7.62%
4.69%
0.68%
0.36%
0.058
InternVL 3.5-8B
1024
28.52%
24.71%
6.64%
7.05%
-0.021
26.88%
23.29%
19.73%
4.20%
1.27%
0.83%
0.055
InternVL 3.5-4B
1024
25.59%
22.36%
5.47%
5.72%
-0.014
24.45%
21.37%
3.13%
0.98%
0.10%
0.03%
0.039
Kimi K2.5
1024
34.28%
39.36%
11.91%
13.49%
-0.068
30.27%
34.76%
27.05%
4.98%
1.56%
1.35%
0.022
Kimi K2.6
1024
36.13%
42.77%
14.65%
15.46%
-0.034
34.25%
40.54%
22.07%
12.30%
5.27%
2.72%
0.188
MiMo-V2.5
1024
33.11%
51.17%
16.21%
16.94%
-0.031
31.68%
48.97%
43.26%
26.56%
18.95%
11.49%
0.341
Qwen3.8-27B
1024
26.17%
61.91%
15.82%
16.20%
-0.018
25.55%
60.45%
34.67%
22.75%
13.09%
7.89%
0.261
Qwen3.6-27B
1024
30.76%
60.35%
17.48%
18.57%
-0.048
28.96%
56.83%
36.04%
21.68%
12.99%
7.81%
0.262
Qwen3.5-397B-A17B
1024
27.44%
47.95%
12.01%
13.16%
-0.051
25.05%
43.77%
3.32%
2.83%
0.10%
0.09%
0.001
Qwen3.5-122B-A10B
1024
32.62%
49.71%
13.48%
16.21%
-0.117
27.11%
41.32%
29.10%
6.05%
2.54%
1.76%
0.072
Qwen3.5-35B-A3B
1024
29.49%
53.22%
15.33%
15.70%
-0.016
28.81%
51.99%
27.64%
6.05%
2.44%
1.67%
0.072
Qwen3.5-27B
1024
31.64%
57.81%
17.38%
18.29%
-0.040
30.07%
54.94%
42.77%
7.32%
3.81%
3.13%
0.052
Qwen3.5-9B
1024
29.00%
53.22%
14.36%
15.44%
-0.048
26.97%
49.49%
15.92%
10.25%
2.44%
1.63%
0.073
Qwen3.5-4B
1024
30.86%
47.46%
15.43%
14.65%
0.034
32.51%
50.00%
12.89%
6.84%
1.07%
0.88%
0.023
SpaceOm
1024
24.71%
35.16%
8.40%
8.69%
-0.014
23.89%
33.99%
2.83%
2.44%
0.20%
0.07%
0.049
SpaceR
1024
20.61%
42.38%
8.11%
8.73%
-0.031
19.12%
39.34%
2.54%
2.15%
0.20%
0.05%
0.062
SpaceThinker
1024
22.56%
32.81%
7.52%
7.40%
0.006
22.92%
33.33%
1.17%
0.59%
0.00%
0.01%
-0.008
ViLaSR
1024
18.85%
43.07%
9.57%
8.12%
0.075
22.22%
50.78%
4.30%
1.86%
0.49%
0.08%
0.149
Action-Observation
Model
Pairs
Agent Sem.
World Sem.
Dual Sem.
Indep. Sem.
Sem. phi
A given W
W given A
Agent Evi.
World Evi.
Dual Evi.
Indep. Evi.
Evi. phi
Claude Opus 4.8
1024
35.94%
51.86%
19.73%
18.64%
0.046
38.04%
54.89%
14.26%
8.01%
2.54%
1.14%
0.147
Claude Opus 4.7
1024
37.21%
39.06%
14.75%
14.53%
0.009
37.75%
39.63%
7.03%
7.91%
0.98%
0.56%
0.061
Claude Opus 4.6
1024
32.42%
58.89%
18.85%
19.09%
-0.011
32.01%
58.13%
2.83%
9.18%
0.29%
0.26%
0.007
Claude Sonnet 4.6
1024
23.63%
61.04%
15.72%
14.42%
0.063
25.76%
66.53%
3.61%
8.59%
0.10%
0.31%
-0.041
Gemini 3.8 Flash
1024
74.90%
69.53%
52.83%
52.08%
0.038
75.98%
70.53%
73.83%
42.29%
36.23%
31.22%
0.231
Gemini 3.1 Pro Preview
1024
58.79%
47.17%
27.15%
27.73%
-0.024
57.56%
46.18%
59.86%
37.11%
30.76%
22.21%
0.361
Gemini 3.1 Flash Lite
1024
30.08%
35.94%
10.25%
10.81%
-0.025
28.53%
34.09%
47.95%
9.28%
4.98%
4.45%
0.037
Gemini 3 Flash
1024
45.90%
51.27%
24.32%
23.53%
0.031
47.43%
52.98%
37.50%
5.08%
2.15%
1.90%
0.023
GPT-6 Astra
1024
42.97%
55.37%
25.68%
23.79%
0.077
46.38%
59.77%
27.83%
14.94%
6.54%
4.16%
0.149
GPT-5.6 Sol
1024
42.09%
55.76%
23.14%
23.47%
-0.013
41.51%
54.99%
24.61%
12.50%
5.57%
3.08%
0.175
GPT-5.6 Terra
1024
40.53%
60.16%
26.07%
24.38%
0.070
43.34%
64.34%
27.05%
12.89%
5.66%
3.49%
0.146
GPT-5.5
1024
53.42%
53.61%
29.30%
28.64%
0.026
54.64%
54.84%
17.48%
17.87%
5.18%
3.12%
0.141
GPT-5.4
1024
37.60%
36.04%
13.57%
13.55%
0.001
37.67%
36.10%
13.87%
12.01%
2.73%
1.67%
0.095
GPT-5.3
1024
37.30%
62.30%
23.83%
23.24%
0.025
38.24%
63.87%
13.87%
15.43%
2.93%
2.14%
0.063
MiMo-V2-Omni
1024
21.29%
38.18%
8.98%
8.13%
0.043
23.53%
42.20%
36.91%
9.18%
4.00%
3.39%
0.044
Muse Spark 1.3 Contributor
1024
48.63%
64.36%
32.62%
31.30%
0.055
50.68%
67.07%
15.04%
14.94%
3.42%
2.25%
0.092
Qwen3.7-Plus
1024
34.57%
43.36%
15.14%
14.99%
0.006
34.91%
43.79%
21.39%
18.75%
6.45%
4.01%
0.152
Qwen3.6-Plus
1024
33.50%
42.48%
13.96%
14.23%
-0.011
32.87%
41.69%
14.75%
5.96%
1.46%
0.88%
0.070
Qwen3.6-Flash
1024
28.71%
56.74%
16.02%
16.29%
-0.012
28.23%
55.78%
19.24%
18.55%
5.08%
3.57%
0.098
Qwen3.5-Plus
1024
29.00%
48.93%
14.65%
14.19%
0.020
29.94%
50.51%
12.30%
8.30%
1.07%
1.02%
0.006
Qwen3.5-Flash
1024
28.91%
54.49%
15.92%
15.75%
0.007
29.21%
55.07%
15.82%
8.89%
2.05%
1.41%
0.062
GLM-5.3-Flash
1024
35.16%
57.42%
20.51%
20.19%
0.014
35.71%
58.33%
45.41%
33.01%
20.51%
14.99%
0.236
GLM-4.6V
1024
27.05%
44.24%
12.21%
11.97%
0.011
27.59%
45.13%
44.92%
10.16%
5.37%
4.56%
0.054
GLM-4.5V
1024
29.00%
39.65%
11.33%
11.50%
-0.008
28.57%
39.06%
23.44%
17.77%
6.64%
4.17%
0.153
Intern-S1-Pro
1024
26.17%
30.27%
7.03%
7.92%
-0.044
23.23%
26.87%
10.55%
3.22%
0.39%
0.34%
0.009
InternVL 3.5-241B-A28B
1024
26.07%
41.80%
11.04%
10.90%
0.006
26.40%
42.32%
17.48%
5.18%
1.17%
0.90%
0.032
InternVL 3.5-30B-A3B
1024
24.02%
41.31%
8.89%
9.92%
-0.049
21.51%
36.99%
7.03%
1.27%
0.10%
0.09%
0.003
InternVL 3.5-14B
1024
31.93%
41.31%
13.48%
13.19%
0.012
32.62%
42.20%
7.52%
4.98%
0.49%
0.37%
0.020
InternVL 3.5-8B
1024
24.51%
30.76%
7.81%
7.54%
0.014
25.40%
31.87%
9.47%
2.54%
0.39%
0.24%
0.033
InternVL 3.5-4B
1024
26.37%
36.72%
9.96%
9.68%
0.013
27.13%
37.78%
12.01%
0.98%
0.00%
0.12%
-0.037
Kimi K2.5
1024
32.71%
30.08%
9.67%
9.84%
-0.008
32.14%
29.55%
18.46%
4.69%
0.88%
0.87%
0.002
Kimi K2.6
1024
32.32%
28.03%
9.18%
9.06%
0.006
32.75%
28.40%
12.21%
10.84%
2.83%
1.32%
0.148
MiMo-V2.5
1024
27.34%
49.71%
14.55%
13.59%
0.043
29.27%
53.21%
43.46%
27.83%
19.14%
12.09%
0.317
Qwen3.8-27B
1024
28.81%
49.41%
14.75%
14.24%
0.023
29.84%
51.19%
16.99%
21.88%
6.25%
3.72%
0.163
Qwen3.6-27B
1024
36.43%
53.42%
19.34%
19.46%
-0.005
36.20%
53.08%
13.18%
19.53%
4.10%
2.57%
0.114
Qwen3.5-397B-A17B
1024
35.84%
46.19%
16.99%
16.55%
0.018
36.79%
47.41%
13.48%
4.39%
1.27%
0.59%
0.097
Qwen3.5-122B-A10B
1024
37.79%
49.02%
17.97%
18.53%
-0.023
36.65%
47.55%
25.98%
6.84%
1.86%
1.78%
0.007
Qwen3.5-35B-A3B
1024
27.44%
57.52%
15.72%
15.78%
-0.003
27.33%
57.30%
0.59%
7.23%
0.00%
0.04%
-0.021
Qwen3.5-27B
1024
37.11%
50.59%
18.16%
18.77%
-0.025
35.91%
48.95%
22.75%
7.52%
2.34%
1.71%
0.057
Qwen3.5-9B
1024
29.00%
52.25%
15.82%
15.15%
0.029
30.28%
54.55%
22.95%
11.52%
3.22%
2.64%
0.043
Qwen3.5-4B
1024
30.18%
47.46%
15.43%
14.32%
0.048
32.51%
51.13%
12.40%
6.35%
1.27%
0.79%
0.060
SpaceOm
1024
24.51%
32.81%
7.62%
8.04%
-0.021
23.21%
31.08%
4.20%
1.76%
0.10%
0.07%
0.009
SpaceR
1024
24.02%
48.83%
11.23%
11.73%
-0.023
23.00%
46.75%
1.07%
2.34%
0.00%
0.03%
-0.016
SpaceThinker
1024
25.39%
34.28%
8.30%
8.70%
-0.019
24.22%
32.69%
1.76%
0.68%
0.00%
0.01%
-0.011
ViLaSR
1024
22.07%
49.71%
9.77%
10.97%
-0.058
19.65%
44.25%
1.95%
1.46%
0.20%
0.03%
0.100
Behavior-Visibility
Model
Pairs
Agent Sem.
World Sem.
Dual Sem.
Indep. Sem.
Sem. phi
A given W
W given A
Agent Evi.
World Evi.
Dual Evi.
Indep. Evi.
Evi. phi
Gemini 3.8 Flash
1024
11.72%
46.00%
6.93%
5.39%
0.096
15.07%
59.17%
15.63%
30.47%
8.20%
4.76%
0.206
Gemini 3.1 Pro Preview
1024
6.74%
33.50%
1.17%
2.26%
-0.092
3.50%
17.39%
6.54%
26.76%
2.05%
1.75%
0.027
Gemini 3.1 Flash Lite
1024
9.38%
52.83%
7.32%
4.95%
0.163
13.86%
78.13%
8.50%
47.95%
5.47%
4.07%
0.100
Gemini 3 Flash
1024
19.53%
55.18%
11.43%
10.78%
0.033
20.71%
58.50%
24.22%
48.93%
12.60%
11.85%
0.035
MiMo-V2-Omni
1024
17.48%
59.38%
10.64%
10.38%
0.014
17.93%
60.89%
16.70%
58.89%
11.52%
9.83%
0.092
Muse Spark 1.3 Contributor
1024
18.07%
47.85%
12.60%
8.65%
0.206
26.33%
69.73%
25.10%
51.95%
16.21%
13.04%
0.146
Qwen3.7-Plus
1024
19.04%
53.61%
13.96%
10.21%
0.192
26.05%
73.33%
18.95%
62.30%
11.52%
11.80%
-0.015
Qwen3.6-Plus
1024
16.99%
50.78%
14.84%
8.63%
0.331
29.23%
87.36%
12.30%
57.13%
8.50%
7.03%
0.090
Qwen3.6-Flash
1024
13.87%
52.25%
11.52%
7.25%
0.248
22.06%
83.10%
12.50%
58.59%
7.32%
7.32%
0.000
Qwen3.5-Plus
1024
19.04%
53.22%
14.36%
10.14%
0.215
26.97%
75.38%
13.96%
59.57%
9.08%
8.32%
0.045
Qwen3.5-Flash
1024
13.09%
52.34%
9.28%
6.85%
0.144
17.72%
70.90%
12.89%
65.04%
9.47%
8.38%
0.068
GLM-5.3-Flash
1024
15.63%
49.12%
11.82%
7.68%
0.228
24.06%
75.63%
16.80%
48.63%
9.38%
8.17%
0.065
GLM-4.6V
1024
4.39%
31.25%
0.10%
1.37%
-0.134
0.31%
2.22%
2.15%
24.90%
0.49%
0.54%
-0.007
InternVL 3.5-30B-A3B
1024
9.57%
43.95%
7.23%
4.21%
0.207
16.44%
75.51%
2.93%
9.86%
0.29%
0.29%
0.001
InternVL 3.5-14B
1024
5.47%
54.49%
3.03%
2.98%
0.004
5.56%
55.36%
2.64%
17.29%
0.88%
0.46%
0.070
InternVL 3.5-8B
1024
7.03%
51.86%
6.45%
3.65%
0.219
12.43%
91.67%
1.95%
17.09%
0.29%
0.33%
-0.008
InternVL 3.5-4B
1024
3.42%
34.57%
0.39%
1.18%
-0.092
1.13%
11.43%
2.05%
24.41%
0.49%
0.50%
-0.002
Kimi K2.5
1024
17.68%
50.49%
11.52%
8.92%
0.136
22.82%
65.19%
17.58%
53.81%
10.16%
9.46%
0.037
Kimi K2.6
1024
16.80%
39.94%
8.50%
6.71%
0.098
21.27%
50.58%
16.41%
53.91%
7.81%
8.84%
-0.056
MiMo-V2.5
1024
16.21%
58.50%
11.91%
9.48%
0.134
20.37%
73.49%
17.09%
60.16%
9.96%
10.28%
-0.017
Qwen3.8-27B
1024
10.16%
43.16%
8.89%
4.38%
0.301
20.59%
87.50%
4.79%
17.09%
0.49%
0.82%
-0.041
Qwen3.6-27B
1024
15.33%
52.73%
13.38%
8.09%
0.294
25.37%
87.26%
13.48%
47.95%
7.71%
6.46%
0.073
Qwen3.5-397B-A17B
1024
11.91%
40.43%
5.66%
4.82%
0.053
14.01%
47.54%
5.47%
26.27%
1.86%
1.44%
0.042
Qwen3.5-122B-A10B
1024
11.62%
48.44%
5.37%
5.63%
-0.016
11.09%
46.22%
11.43%
54.30%
6.54%
6.20%
0.021
Qwen3.5-35B-A3B
1024
14.36%
54.10%
8.89%
7.77%
0.064
16.43%
61.90%
13.96%
56.93%
8.98%
7.95%
0.060
Qwen3.5-27B
1024
15.82%
51.37%
11.23%
8.13%
0.170
21.86%
70.99%
15.72%
58.01%
9.67%
9.12%
0.030
Qwen3.5-9B
1024
14.16%
50.68%
8.01%
7.18%
0.048
15.80%
56.55%
13.87%
40.82%
5.96%
5.66%
0.017
Qwen3.5-4B
1024
7.13%
33.89%
0.59%
2.42%
-0.150
1.73%
8.22%
0.78%
57.32%
0.39%
0.45%
-0.013
SpaceOm
1024
7.13%
32.32%
0.29%
2.30%
-0.167
0.91%
4.11%
1.86%
25.29%
0.00%
0.47%
-0.080
SpaceR
1024
5.37%
33.69%
0.39%
1.81%
-0.133
1.16%
7.27%
2.05%
23.05%
0.00%
0.47%
-0.079
SpaceThinker
1024
7.91%
33.69%
0.88%
2.67%
-0.140
2.61%
11.11%
2.05%
20.21%
0.00%
0.41%
-0.073
ViLaSR
1024
7.03%
35.94%
1.56%
2.53%
-0.079
4.35%
22.22%
0.39%
15.04%
0.00%
0.06%
-0.026
Detailed matched-pair statistics. Each relation includes all available main-table models, including image-only models for the two image pairings. Conditional success and phi are computed from the stored binary outcomes.
Human Validation
Human Baseline Protocol
Human annotators reach 86.4% image and 85.1% video accuracy on stratified questionnaire pools comprising 30 completed questionnaires, supporting human solvability of the benchmark tasks. Human Dual scores require matched records from the same annotator and are therefore left unreported.
The image interface presents reference, query, and candidate views as appropriate. Annotators choose an answer and adjust the normalized landmark box. The video interface supports replay of the first-person sequence, behavior or visibility selection, and editable temporal intervals. Answers are saved before navigation. The galleries show the four image-task interfaces and the two video-task interfaces.
Image Task Interface
Four image tasks use answer selection with landmark bounding boxes
Annotators inspect a landmark reference view and a query or candidate view, choose one semantic answer, and adjust the normalized bounding box around the target landmark.
Landmark-relative Position Reasoning. The annotator infers the UAV position in the landmark-centric frame and localizes the landmark in the query view. Action-conditioned Future Observation Prediction. The annotator selects the future view produced by the specified UAV action and grounds the target landmark. Self-relative Position Reasoning. The annotator judges the landmark direction in the UAV-centric view and supplies its bounding box. Goal-conditioned Action Decision. The annotator selects the action needed to reach the target relation and localizes the referenced landmark.
Video Task Interface
Two video tasks use semantic decisions with editable time intervals
Annotators watch the complete first-person flight video, select behavior labels or report landmark visibility, and mark every supporting interval on an editable timeline.
Flight Behavior Recognition and Temporal Localization. Multiple behavior options can be selected, with one temporal interval assigned to each selected action. Landmark Visibility Counting and Interval Reasoning. The annotator marks all visible intervals; the interface derives and checks the corresponding appearance count.
Semantic solvability and evidence localization
The task-level results below complement the model evaluation with human semantic and localization performance. Their different strengths, particularly in self-relative spatial localization, highlight the value of assessing the answer and its grounding separately.
Task or level
Semantic metric
Evidence metric at 0.5
Mean overlap
Landmark-relative Position
85.8%
77.9%
57.8%
Future Observation
94.6%
75.4%
59.5%
Self-relative Position
82.5%
53.8%
47.2%
Action Decision
82.5%
72.5%
57.5%
Composite Behavior
83.9%
62.9%
53.3%
Atomic Behavior
76.3%
83.2%
61.6%
Landmark Visibility
95.0%
96.7%
83.7%
Human task-level performance. Image columns report option Acc, BBox Acc@0.5, and mIoU; video columns retain the questionnaire semantic scores, temporal F1@0.5, and mtIoU. Values are unranked, with task denominators determined by the stratified questionnaire assignments.
Diagnostic Failures
Spatial reference frames, temporal tracking, and motion semantics
A manual audit of 700 Instant failures and 600 Thinking failures identifies recurring sources of error. Within the Instant failure sample, dual-coordinate reasoning accounts for 33.7%, temporal tracking for 23.6%, and motion semantics for 21.9%. These categories connect the aggregate scores to specific spatial and temporal demands.
Error category
Instant: 700 failures
Thinking: 600 failures
Visual perception and landmark identity
0.00%
13.51%
Spatial grounding
15.86%
17.84%
Dual-coordinate spatial reasoning
33.71%
16.22%
Motion semantics and hierarchical action reasoning
21.86%
17.12%
Temporal tracking and localization
23.57%
16.04%
Language, logic, or protocol
5.00%
19.28%
Error composition in manually audited failures. Percentages use the audited failures in each mode as their denominator. The two mode samples are independently collected, so the comparison describes failure composition rather than a paired estimate of mode effects.
A QA-level diagnosis distinguishes correct semantics with incorrect evidence, correct evidence with incorrect semantics, success on both, and failure on both. The distribution covers 6,144 outputs of Qwen3.5-4B Thinking, complementing the two-task Dual metrics with a closer view of individual answer-grounding outcomes.
Image evidence is thresholded spatial overlap under the task's candidate-domain rule. Video evidence uses the recorded task-level temporal success criterion. The point cloud displays categorical counts, while the selected cases expose the actual boxes and intervals.
QA-level Distribution
Semantic correctness versus evidence grounding
Each point represents one QA result from Qwen3.5-4B Thinking. The dashed thresholds divide
semantic and evidence success into four diagnostic quadrants.
Semantic-evidence distribution of QA outcomes. Each point represents one result from Qwen3.5-4B Thinking.
Quadrant membership encodes semantic and evidence correctness; deterministic jitter separates
overlapping points within each quadrant. The distribution uses the same
semantic and evidence thresholds as the qualitative cases below.
Quadrant I · Both Correct · Semantic ✓ · Evidence ✓
env_21_1_12_self_shared_4way_000029_where
Image 1 shows the Right facade of stack of white and blue shipping containers with 'super cargo' text in the landmark-centric coordinate frame. Based on that reference, what is your position relative to the landmark in image 2? Select one option and return the normalized bounding box of the landmark in image 2.
Options:
A. Back
B. Left
C. Front
D. Right
Original Question
-
Open-ended Question
-
Diagnostic focus: Coordinated spatial perception and frame transformation. Both high-level direction and physical bounding box align precisely with ground truth.
BBox:[0.350, 0.300, 0.650, 0.700]✓IoU = 0.547Valid bounding box pass (IoU 0.547) and correct direction choice.
Prediction Summary. The model successfully solves both the cognitive reasoning task and grounds its decision in high-fidelity spatial/temporal evidence. Demonstrates coordinated dual cognition. Coordinated spatial perception and frame transformation. Both high-level direction and physical bounding box align precisely with ground truth.
The medium-specific breakdown summarizes the same distribution. Semantic-only cases isolate correct decisions with insufficient localization; evidence-only cases isolate successful localization with an incorrect decision. The linked examples show how these outcomes arise in particular views and temporal intervals.
Medium
Both correct
Semantic only
Evidence only
Neither
QA results
Image
6.86%
34.72%
8.37%
50.05%
4,096
Video
18.21%
5.47%
5.96%
70.36%
2,048
Semantic-evidence outcomes by medium. Each row partitions the outputs of Qwen3.5-4B Thinking into four mutually exclusive categories, with percentages calculated over the indicated QA count.
Open-ended & Qualitative Validation
The bottlenecks are not limited to answer-choice selection
Open-ended validation covers 6 models and all 12 scenes, targeting 4,096 image questions and 2,048 video questions. Removing answer choices leaves the visual inputs and grounding requirements in place but changes the semantic response protocol. Qwen3.5-9B reaches 11.73% landmark-relative and 29.88% self-relative accuracy; atomic behavior reaches 62.55% semantic accuracy but 26.10% temporal F1@0.5.
The supplementary protocol uses completed-response denominators: Qwen3.5-9B contributes 3,940 returned image responses and 2,048 video responses; unreturned responses are excluded. Comparisons with multiple-choice results should account for this coverage and the changed semantic response format.
The examples place the original and open-ended prompts alongside representative generated answers. They reproduce the semantic-grounding separation observed in the main evaluation: correct directional language can accompany an imprecise box, while a correct visible count can omit part of an interval.
Task 1 · Landmark-relative Position Reasoning
Multiple-choice to open-ended comparison
Image 1 shows the Front facade of light gray mid-rise building with red tiled roof in the landmark-centric coordinate frame. Based on that reference, where are you relative to the landmark in image 2? Answer with one or a few words and return the normalized bounding box of the landmark in image 2.
Original Question
Image 1 shows the Front facade of light gray mid-rise building with red tiled roof... Based on that reference, what is your position relative to the landmark in image 2? Select one option and return the normalized bounding box.
Options:
A. Right
B. Back
C. Left
D. Front
Open-ended Question
Image 1 shows the Front facade of light gray mid-rise building with red tiled roof in the landmark-centric coordinate frame. Based on that reference, where are you relative to the landmark in image 2? Answer with one or a few words and return the normalized bounding box of the landmark in image 2.
Replaces multiple-choice letters [A, B, C, D] with open natural language direction fill-in ('Right', 'Front-Left', etc.), paired with strict normalized bounding box prediction.
Semantic:Incorrect direction ('Left' vs GT 'Right')✕
Evidence:IoU = 0.681 (Acc@0.5 Passed)✓
Qwen3.5-9B attains only 11.73% overall semantic accuracy in open-ended testing on this task. Notice that removing option letters reveals the true weakness in ego-motion frame translation, while spatial bounding box capability remains intact.
Proprietary frontier model successfully infers open direction while maintaining accurate landmark grounding.
Prediction Summary. The same visual evidence is evaluated without multiple-choice options. Replaces multiple-choice letters [A, B, C, D] with open natural language direction fill-in ('Right', 'Front-Left', etc.), paired with strict normalized bounding box prediction.
Matched multiple-choice and open-ended examples. The same visual context supports both question formats. Complete results and coverage retain the original validation metrics and identify unreported values separately from zeros.
Thinking vs Instant
Additional deliberation has model- and modality-dependent effects
In the same-model comparisons reported in the paper, Qwen3.7-Plus improves image/video accuracy from 41.7%/31.4% to 55.0%/39.9%. MiMo-V2.5 improves image accuracy from 40.3% to 44.7%, but its image mIoU drops from 33.8% to 24.5% and video accuracy from 41.6% to 34.9%. More reasoning therefore does not uniformly improve semantic decisions and physical localization together.
The same-model comparison reports aggregate validation Acc and mean spatial or temporal overlap. These task-level measures reveal how reasoning mode affects decisions and localization; Dual scores use the separate pairwise protocol defined above.
Model / mode
Image
Video
Validation Acc
mIoU
Validation Acc
mtIoU
GPT-5.5-I
52.6%
25.2%
-
-
GPT-5.5-T
53.5%
27.7%
-
-
GPT-5.4-I
37.3%
24.1%
-
-
GPT-5.4-T
42.5%
24.2%
-
-
Qwen3.7-Plus-I
41.7%
27.6%
31.4%
37.0%
Qwen3.7-Plus-T
55.0%
26.2%
39.9%
43.5%
MiMo-V2.5-I
40.3%
33.8%
41.6%
48.4%
MiMo-V2.5-T
44.7%
24.5%
34.9%
41.7%
Qwen3.5-9B-I
40.9%
24.7%
29.8%
33.1%
Qwen3.5-9B-T
40.2%
19.3%
27.2%
29.5%
Same-model Instant and Thinking comparisons. I = Instant; T = low-effort Thinking. Scores follow the paper's task-level reasoning-mode ablation, with dashes for unreported modalities.
Beyond the Main Benchmark
Real-scene reconstruction and structured supervision
The three-scene pilot extends construction and evaluation to rendered reconstructions from real aerial imagery and metric geometry, with 30 reviewed landmarks, 400 image questions, and 16 reported models. It reproduces the broad semantic-grounding trends seen in simulation. The training probe uses 11,232 images from six disjoint scenes and demonstrates improved image performance and overall transfer to video through SFT and GRPO.
From paired perception and reasoning to dependable UAV behavior
UAV-DualCog reveals complementary demands in Agent-State Reasoning, World-State Reasoning, and explicit spatial-temporal grounding. Model rankings differ between semantic and evidence success, while the three paired relations expose distinct strengths in coordinate-frame interpretation, action-observation association, and behavior-visibility consistency. Human, open-ended, and diagnostic analyses help explain these patterns; real-scene reconstructions extend their reach, and the optimization probe demonstrates the value of structured supervision.
Future development can build on these findings to improve models that coordinate state estimates and grounding during embodied action. Extending evaluation to physical UAV platforms and more diverse real-world scenes will connect these capabilities to relative-position navigation, subgoal completion, stopping decisions, and persistent target observation, advancing reliable aerial autonomy.
