Training & Optimization

UAV-DualCog-Train and Two-Stage Optimization

UAV-DualCog also serves as a structured supervision source. This page describes the disjoint training set, multimodal supervised fine-tuning, group-relative policy optimization, and the resulting direct image gains and cross-modal video transfer.

Training & Optimization

Two-Stage SFT & GRPO Optimization Probe: Hyperparameters, Rewards, and Curves

Beyond evaluation, we examine whether UAV-DualCog can serve as effective supervision to bridge the semantic–evidence and Agent–World dual-cognition gaps. We construct UAV-DualCog-Train from six scenes strictly disjoint from the evaluation benchmark (11,232 samples, 234 landmarks) and train an edge-deployable base model (Qwen3.5-4B) using a two-stage alignment pipeline: Multimodal Supervised Fine-Tuning (SFT) followed by Joint Group Relative Policy Optimization (GRPO).

Stage 1 · Multimodal SFT

Stabilizing Output Structure & Spatial Coordination

Stage 1 applies Parameter-Efficient Fine-Tuning (PEFT / LoRA) to train the model to output strict JSON conforming to the dual-cognition task contracts while jointly predicting multiple-choice answers and landmark bounding boxes.

Hyperparameter	Value	Configuration Detail
Base Architecture	Qwen3.5-4B	Native multimodal agent backbone
PEFT Strategy	LoRA (r=16, α=32)	Attention projections (q, k, v, o) and MLP projections (gate, up, down)
Learning Rate	5 × 10^-5	Reported SFT run setting; scheduler and warmup follow SFTConfig defaults
Effective Batch Size	32	Per-device batch size 1 with 32 gradient accumulation steps
Epochs	3.0	Full pass over 11,232 disjoint training samples
Optimizer & Precision	SFTConfig defaults / BF16	Optimizer parameters follow SFTConfig defaults; BF16 is enabled for CUDA training

Stage 2 · Multi-Reward GRPO

Joint Optimization via Reinforcement Learning

Stage 2 employs Group Relative Policy Optimization (GRPO) to optimize the model directly against task rewards without requiring an external critic network, sampling two candidate completions per prompt.

Hyperparameter	Value	Configuration Detail
Learning Rate	1 × 10^-6	Reported joint-GRPO run setting
Optimization Steps	600 steps	Checkpoints saved every 10 steps
Group Rollout Size (G)	2 completions	Two responses compared for each prompt
Effective Batch Size	16	Per-device batch size 1 with 16 gradient accumulation steps
Maximum Image Side	720 px	Shared by the reported SFT and GRPO runs
Reward Weights	w_f=1.0, w_a=1.0, w_b=1.0	Balanced format, answer, and GIoU objectives

Reward Formulation

Multi-Objective Composite Reward Contract

The four training tasks span two difficulty levels and retain the benchmark's answer-and-BBox output contract. After SFT stabilizes structured responses, GRPO scores each sampled completion with three equally weighted rewards for answer-and-box parseability, semantic correctness, and spatial grounding:

R = R_{\mathrm{format}} + R_{\mathrm{answer}} + R_{\mathrm{bbox}}

R_format ∈ {0, 1}: Returns 1 when both an answer and a four-coordinate box can be extracted, and 0 otherwise. The parser accepts JSON fields answer_option_id (A–H) and bbox_xyxy_norm, with labeled-text fallbacks Answer: or Option: and BBox: or Bounding Box:. Box corners are reordered and clipped to [0, 1]; when the largest absolute coordinate exceeds 1.5, coordinates are first divided by 1,000. This extraction-based reward supports the canonical JSON output contract without requiring strict JSON serialization.
R_answer ∈ {0, 1}: Discrete 0/1 indicator testing whether the predicted option matches ground truth a^*.
R_bbox = clip((GIoU(B̂, B^*) + 1) / 2, 0, 1) ∈ [0, 1]: Generalized Intersection over Union supplies a continuous localization signal, rescaled to the same reward range as the other two terms. GIoU(B̂, B^*) = IoU(B̂, B^*) - (|C| - |B̂ ∪ B^*|) / |C|, where C is the smallest enclosing bounding box. Missing or unparseable boxes receive zero reward; zero-area boxes also receive zero BBox reward.

Metric	Base (Qwen3.5-4B)	Optimized	Gain
Image Option Acc	38.99%	59.77%	+20.78 pp
Image BBox@0.5	9.27%	61.04%	+51.76 pp
Image mIoU	20.12%	52.78%	+32.66 pp
Video Global Acc	16.75%	26.67%	+9.91 pp
Video F1@0.5	16.51%	33.40%	+16.89 pp
Video mtIoU	18.74%	34.58%	+15.84 pp

Table 12. Two-stage optimization results on Qwen3.5-4B. Beyond boosting in-domain image metrics (BBox@0.5 rises from 9.27% to 61.04%), the optimized model also exhibits cross-modal transfer to video behavior localization (+16.89 pp F1@0.5) despite zero video training samples.

Overall Media Performance Gain across Image and Video metrics — **Overall Media Performance Gain.** Overall Media Gain. Comparison between base Qwen3.5-4B and the optimized model across image semantic, spatial grounding, and video temporal metrics.

Image Task Performance Gain across 4 tasks — **Image Task Performance Gain.** Image Task Gains. Task-by-task accuracy and BBox@0.5 gains across Landmark-relative Position, Future Observation, Self-relative Position, and Action Decision.

Reproduction

Training scripts and reported-run commands

The released code package includes dataset preparation, multimodal SFT, joint GRPO, checkpoint auditing, and transfer evaluation under scripts/uav_dualcog/uav_dualcog_optimize/. The complete command sequence and reported-run arguments are provided in the Training Probe section of Usage.