Video geometry

LoFT

Looped Foundation Transformer for Video Geometry

Reconstructing 3D scenes from RGB video through temporal attention and iterative camera refinement.

Fig. 1 Street reconstruction
LoFT point-cloud reconstruction of a street lined with brick buildings
Full teaser
PaperComing soon
CodeComing soon
Hugging FaceComing soon

Overview

From video
to 3D.

Adjacent video frames often see the same surface. Views farther apart reveal how that surface sits in 3D.

LoFT uses both. One loop gathers visual evidence across an ordered clip. A second loop refines each camera using its previous estimate. The resulting cameras place per-frame depth maps in a shared coordinate system.

Training datasets
80
Sequences
2.97M
Images
961.8M

Method

Two loops for video geometry.

LoFT architecture with a visual aggregation loop followed by a camera refinement loop and per-frame depth decoding.
Fig. 2 Architecture of LoFTEnlarge figure

Visual aggregation

Local attention expands from 3 to 5, 9, and 17 frames. Between these local steps, every frame attends to the full clip, including distant views of the same place.

Camera refinement

Each iteration predicts pose and field of view, then feeds the updated camera state into the next iteration. A dense head decodes depth and confidence once per frame.

Results

Reconstruction
benchmarks.

Full LoFT teaser showing benchmark comparisons alongside a reconstructed street. View full teaser

LoFT achieves the lowest depth AbsRel and highest pose AUC at 30° among the ranked methods on each of the four evaluation splits.

LoFT depth, pose, and trajectory scores. Arrows indicate the better direction.
BenchmarkAbsRel ↓δ1.03 ↑AUC@3° ↑AUC@30° ↑ATE ↓
SpatialBench Full0.06370.69590.47670.83410.5041
SpatialBench Continuous0.07340.67650.47570.83420.2131
ClassicBench0.13160.54640.35060.76230.5015
VRBench0.08520.57510.38790.82710.2973

SpatialBench uses the Medium set. AbsRel and δ measure depth accuracy, AUC measures relative camera pose, and ATE measures trajectory error.

Demo

Interactive point clouds.

Figure