FineCog-Nav: Integrating Fine-grained Cognitive Modules for Zero-shot Multimodal UAV Navigation

CVPR 2026 Findings

Dian Shao^1†, Zhengzheng Xu¹, Peiyang Wang², Like Liu¹, Yule Wang¹, Jieqi Shi^2†, Jing Huo^2
1Northwestern Polytechnical University    ²Nanjing University
^†Corresponding Authors

📄 Paper 📝 arXiv 💻 Code 💾 Dataset

Abstract

Poster

Teaser

Vision-Language Navigation (VLN) for UAVs requires understanding ambiguous multi-step instructions and executing long-horizon planning in complex 3D environments from an egocentric view. Existing zero-shot methods often rely on very large models, generic prompts, and loosely organized modules, which limits robustness and interpretability. We propose FineCog-Nav, a top-down framework that simulates human cognition through fine-grained collaborative modules, including perception, attention, memory, imagination, reasoning, planning, and decision-making. Each module is driven by a moderate-sized foundation model with carefully designed cognitive-role prompts and structured input-output protocols. To support detailed evaluation, we construct AerialVLN-Fine, a curated benchmark with 300 trajectories, sentence-level instruction-trajectory alignment, and refined landmark-grounded instructions. Experiments show that FineCog-Nav consistently improves instruction adherence, long-horizon planning quality, and generalization to unseen environments in zero-shot UAV navigation.

Method

FineCog-Nav Framework Pipeline

FineCog-Nav organizes zero-shot UAV navigation into a closed cognitive loop: (1) instruction parsing and subgoal extraction, (2) attention-guided perception, (3) imagination-assisted subgoal judgment, (4) multi-level memory management at step/subgoal/instruction granularity, and (5) explainable decision-making with collision-aware action selection. This modular interdependence improves interpretability and task progression over monolithic prompting pipelines.

Task objective: let the final UAV position $p_{stop}$ approach destination $p_{dest}$ under threshold $\delta$, i.e. $\|p_{stop} - p_{dest}\| \leq \delta$. The action space includes movement, turning, altitude changes, and task completion.

Experimental Results

Table 1. Comparison with Single-module BaseModel

We evaluate several base LLMs with basic prompts, forming single-module baselines referred to as BaseModels. For a fair and meaningful comparison, all experiments are conducted on AerialVLN-Fine, which provides a cleaner and more discriminative benchmark. FineCog-Nav consistently outperforms BaseModel across metrics.

Table 2. Comparison with Framework Baselines

We first evaluate on AerialVLN-Fine, followed by scalable experiments on the larger and noisier AerialVLN-S-Val subset. Across both datasets, FineCog-Nav consistently outperforms all baselines across all metrics, validating the efficacy of the cognitive collaborative framework.

Table 3. Ablation Analysis

Single-module ablations show that removing any module leads to performance drops, with the largest decline occurring when hierarchical memory is replaced by plain history. Joint ablations also degrade performance. The best results are achieved when all modules are enabled.

Table 4. Sentence-level Analysis

Recall that AerialVLN-Fine provides sentence-level annotations for fine-grained evaluation. As the base LLM size increases, performance steadily improves. Small sentence-level differences can accumulate over long trajectories, leading to larger gaps in full-trajectory metrics.

Across all four tables, the results consistently support the same conclusion: a cognition-inspired, collaborative modular framework enables more reliable and interpretable zero-shot UAV navigation than both single-module prompting and existing multi-agent baselines.

Qualitative Analysis

Qualitative Navigation Trajectories

Representative trajectories show that the agent grounds language cues to visible landmarks and executes coherent subgoal transitions. Even in difficult cases, FineCog-Nav tends to keep instruction-consistent behavior rather than drifting early.

Human Study Results

Human study results further support the quantitative findings: participants generally prefer FineCog-Nav for instruction-following quality and perceived navigation intelligence.

AerialVLN-Fine Dataset

Overview of AerialVLN-Fine

AerialVLN-Fine is a curated benchmark built from AerialVLN for more reliable zero-shot UAV VLN evaluation. It provides sentence-level alignment between instruction segments and trajectory segments, and refines ambiguous expressions with explicit visual endpoints and landmark references.

Key Statistics

• Total trajectories: 300 high-quality instruction-trajectory pairs
• Source split: Val-Seen and Val-Unseen scenes from AerialVLN
• Average per trajectory: 189 meters and 76 actions
• Fine-grained semantics: 4.6 aligned instruction sentences per trajectory
• Total scale: 56,050 meters and 22,835 actions
• Instruction refinement: total words 17,572 to 30,762; average length 59 to 103
• Sentence-level total: 1,383 aligned sentences (average 41 meters, 17 actions)

The dataset is designed to support fine-grained capability diagnosis and sentence-level evaluation, while keeping annotation quality high through repeated manual verification.

💾 Dataset Link 💻 Code Link

Citation

Coming soon.

Contact

For questions about this work, please contact:
shaodian@nwpu.edu.cn  |  isjieqi@nju.edu.cn