ACM Multimedia 2026 Rio de Janeiro

Geometry-Aware Camera Localization for Bronchoscopy

Fusing preoperative airway geometry with intraoperative video for accurate, stable, and real-time 6-DoF bronchoscope localization.

Lumin ChenQingyao Tian Jinpeng LiHaoyu Jiang Huai LiaoXinyan Huang Hongbin LiuDong Yi

Chinese Academy of Sciences · Sun Yat-sen University

Paper Code DOI

GABL overview and quantitative comparison with prior bronchoscopy localization methods

Research trajectory

Four generations, one pursuit: reliable localization.

Our bronchoscope localization research has progressed from probabilistic tracking to foundation models, vision-language reasoning, and finally dense geometry-aware localization. Each generation addresses a different barrier to accurate, robust, and real-time navigation.

v1 · MICCAI 2024

PANS

Particle filtering combines motion, anatomy, and geometry for robust real-time tracking.

Read paper v2 · MICCAI 2025

PANSv2

Endoscopic foundation models and automatic re-initialization improve generalization and recovery.

Read paper v3 · 2026

BREATH-VL

Vision-language semantics initialize geometric registration in the correct airway branch.

Read paper v4 · ACM MM 2026

GABL

Dense multi-scale geometry enables accurate, stable, and real-time 6-DoF localization.

Read paper

Why geometry matters

One framework, three complementary scales.

Camera localization in bronchoscopy is challenged by repetitive anatomy, limited texture, and strict latency requirements. GABL injects explicit geometric supervision at the structural, temporal, and appearance levels, combining preoperative CT priors with intraoperative observations in a unified localization framework.

Structure

An anchor graph derived from preoperative CT provides dense anatomical priors for coarse-to-fine localization.

Motion

A causal Transformer models temporal dynamics to stabilize predictions and reduce trajectory drift.

Appearance

RGB-depth matching aligns intraoperative observations with rendered geometry in a shared representation space.

Framework

Geometry-aware supervision from CT to trajectory.

Detailed GABL architecture showing prior modeling, encoders, matching, temporal tracking, and anchor localization — GABL jointly learns anchor localization, temporal tracking, and appearance-geometry matching.

Inside GABL

Three components connect video, geometry, and pose.

GABL encodes intraoperative temporal context, organizes patient-specific airway geometry, and dynamically combines global detection with local tracking to produce a robust 6-DoF trajectory.

Video temporal modeling

A causal Transformer fuses the current frame with historical RGB, pose, and depth embeddings. Random history dropout during training improves robustness while the lower-triangular mask preserves online inference.

3D geometric prior modeling

Preoperative CT is converted into an airway mesh, centerline, and anchor tree. A graph encoder captures global topology, while pose and depth encoders preserve fine local geometry.

Geometric prior modeling from CT and airway mesh to global topology and local anchor geometry

6-DoF pose decoding

Coarse-to-fine anchor localization supplies an absolute pose, RGB-depth matching verifies geometric consistency, and temporal tracking provides smooth incremental updates between frames.

Pose decoding pipeline with pose detector, RGB-depth matcher, and pose tracker

Ablation studies

Every component earns its place.

We isolate the contribution of each localization component, compare temporal data augmentation strategies, and identify the history dropout rate that best balances accuracy and robustness.

Localization components

Removing any module degrades the complete system.

Localization component ablation results
Setting	ATE_trans ↓	ATE_rot ↓	SR-5 ↑	SR-10 ↑
w/o GCN	10.67	31.48	57.59	78.55
w/o Pose Regressor	9.31	107.54	41.09	77.09
w/o Pose Tracker	16.48	38.43	53.57	82.60
w/o Matcher	7.63	30.86	58.09	82.66
Full Model	7.01	29.56	61.04	83.66

Data augmentation

Skip and reverse augmentation are complementary.

Data augmentation setting ablation results
Skip	Reverse	ATE_trans ↓	ATE_rot ↓	SR-5 ↑	SR-10 ↑
–	–	7.97	31.71	54.75	80.72
✓	–	7.84	32.24	58.02	82.08
–	✓	7.71	33.56	59.65	81.52
✓	✓	7.01	29.56	61.04	83.66

History dropout

A 0.75 dropout rate gives the strongest overall result.

Causal Transformer dropout rate ablation results
Dropout rate	ATE_trans ↓	ATE_rot ↓	SR-5 ↑	SR-10 ↑
0.25	8.04	31.51	57.94	83.12
0.50	7.59	30.76	58.27	82.74
0.75	7.01	29.56	61.04	83.66
1.0	7.94	31.25	58.57	82.48

Results

Accurate enough to navigate. Fast enough to guide.

7.01mm

Translation ATE

8.37% lower

29.56deg

Rotation ATE

31.76% lower

61.04%

SR-5

Best reported

33.6FPS

Inference speed

Real-time

Localization in action

From the live RGB view to a global airway pose.

The rendered view follows the intraoperative video while the global panel tracks the predicted bronchoscope position inside the airway.

Animated GABL localization result showing RGB input, rendered airway view, and global camera position — RGB observation · geometry-rendered view · global localization

Qualitative localization

Predicted trajectories closely follow the airway geometry.

Ground-truth and predicted bronchoscope trajectories overlaid on three airway models

Citation

Build on GABL.

@inproceedings{chen2026gabl,
  title     = {Geometry-Aware Camera Localization for Bronchoscopy},
  author    = {Chen, Lumin and Tian, Qingyao and Li, Jinpeng and
               Jiang, Haoyu and Liao, Huai and Huang, Xinyan and
               Liu, Hongbin and Yi, Dong},
  booktitle = {Proceedings of the 34th ACM International Conference
               on Multimedia},
  year      = {2026},
  doi       = {10.1145/3767308.3835259}
}