AeroScene: Progressive Scene Synthesis for Aerial Robotics

Drones are increasingly applied in delivery, inspection, and surveillance, requiring them to navigate and operate within complex 3D environments. Synthesizing realistic scenes for such applications necessitates hierarchical layout generation, where coarse-scale structures (e.g., rooms, terrains, building layouts) establish navigable flight corridors, while fine-scale details (e.g., obstacles, landing areas) ensure task-specific feasibility. However, existing scene creation methods for drone simulators are designed primarily by humans, making them challenging to scale, and often fail to accommodate both physical and fidelity requirements such as unobstructed aerial navigation, accessible interaction areas (e.g., landing pads, inspection points), and coherent indoor-outdoor transitions. In this paper, we propose AeroScene, a hierarchical diffusion-based framework for 3D scene generation tailored to drone tasks. Our approach operates across scales: coarse-scale synthesis generates high-level structures that preserve airspace and navigability, while fine-scale synthesis refines object placement and specifies drone-interaction areas for task execution. AeroScene includes Cross-scale Progressive Attention, which explicitly models dependencies across scales, ensuring that fine-scale details remain consistent with coarse-scale spatial structures. In addition, we design task-aware guidance functions that encourage collision-free plausibility, maintain semantic correlations in hierarchical orders, and handle relationships between indoor and outdoor objects, thereby aligning generated layouts with real-world aerial operation requirements. To support downstream tasks, scenes created by our method are directly embedded into NVIDIA Isaac Sim for physics-ready simulation.

We present AeroScene, a hierarchical diffusion model for generating drone-navigable 3D indoor-outdoor scenes. The framework operates in two complementary scales: coarse-grained convolutional blocks extract high-level layout features (e.g., building footprints, terrain, major functional zones), while fine-grained blocks capture detailed object geometry and placement. These multi-scale features are fused via a novel Cross-scale Progressive Attention module that performs bidirectional top-down and bottom-up interactions, ensuring fine details remain semantically and spatially consistent with coarse structures. During denoising, the model incorporates task-aware guidance objectives — including collision avoidance (preventing object overlap and preserving free flight corridors), coarse-to-fine consistency, and semantic constraint alignment — to produce layouts that are both visually plausible and physically feasible for aerial navigation. The final denoised hierarchical layout is converted into a tokenized scene representation and rendered into realistic 3D environments compatible with physics simulation.

The table below shows that our proposed AeroScene method consistently outperforms existing scene synthesis approaches, including both autoregressive and diffusion-based baselines, across our custom drone-oriented dataset and the standard 3D-FRONT benchmark. It achieves the best scores on all five metrics—FID and KID for distribution fidelity, and CR, CFC, and SP for collision avoidance and navigability—demonstrating superior generation of realistic, collision-free, and drone-friendly indoor-outdoor layouts thanks to the hierarchical diffusion design, cross-scale progressive attention, and task-aware guidance.