Research Highlights

Real spaces are shaped by the people who use them. Objects are moved, stacked, and left out of place, and that clutter is a large part of what makes real scenes hard to understand and interesting to model.

Real-world data

Datasets and benchmarks

Indoor scenes captured in 3D and annotated with object labels.

ScanNet++

ICCV 2023 (Oral)
C. Yeshwanth, Y. Liu, M. Nießner, A. Dai

ScanNet++ couples high-end and commodity capture of the same indoor scenes: sub-millimeter laser scans, 33-megapixel DSLR images and iPhone RGB-D video, registered to each other. Objects are annotated with free-form labels, label-ambiguous cases are marked explicitly, and the labels are mapped to fixed class sets for the benchmarks.

1,000+scenes
1.1MDSLR images
10MiPhone RGB-D frames
111kannotated instances
  • Benchmarks: novel view synthesis (DSLR, iPhone), 3D semantic and instance segmentation
  • CVPR challenges 2024, 2025, 2026
Dataset Benchmarks CVPR 2026 workshop Paper Code
ScanNet capture and annotation overview with annotated scenes

ScanNet

CVPR 2017 (Spotlight) · ScanNet200: ECCV 2022
A. Dai, A. Chang, M. Savva, M. Halber, T. Funkhouser, M. Nießner
ScanNet200: D. Rozenberszki, O. Litany, A. Dai

ScanNet is an RGB-D video dataset of indoor scenes with 3D camera poses, surface reconstructions and instance-level semantic segmentations, collected with a scalable capture approach on commodity sensors and annotated through crowdsourcing. ScanNet200 extends its benchmark to the long tail of object categories found in real indoor scenes.

1,500+scans
2.5MRGB-D views
200classes in ScanNet200
  • CVPR challenges 2019–2023
  • SGP Dataset Award 2020
Dataset Benchmark ScanNet200 Paper Code
Interaction

Perceiving interaction

Synthesizing human interaction with objects and 3D scenes, often zero-shot, and recovering how articulated objects move.

HOI-PAGE
L. Li, A. Dai
Zero-shot generation of 4D human-object interactions from text, guided by part-level affordance reasoning with LLMs and motion cues from generated videos.
ICML 2026
UnfoldArt
M. Boudjoghra, I. Laptev, A. Dai
Reconstruction of complete articulated objects from a single image or text, combining multi-agent reasoning about part motion with a video prior that reveals occluded geometry.
NeurIPS 2026
GenZI
L. Li, A. Dai
Zero-shot synthesis of 3D humans interacting with scenes, lifting 2D interaction hypotheses from vision-language models into 3D.
CVPR 2024
CG-HOI
C. Diller, A. Dai
Text-conditioned generation of dynamic human-object interactions, with human motion, object motion and contact modeled jointly in a diffusion model.
CVPR 2024
Generation

Mesh generation

Generating meshes directly as triangles, rather than extracting dense surfaces from neural fields, gives compact meshes that work with standard graphics tools.

MeshGPT
Y. Siddiqui, A. Alliegro, A. Artemov, T. Tommasi, D. Sirigatti, V. Rosov, A. Dai, M. Nießner
Autoregressive generation of triangle meshes with a decoder-only transformer over learned triangle tokens, producing compact meshes with sharp edges.
CVPR 2024 (Highlight)
MeshArt
D. Gao, Y. Siddiqui, L. Li, A. Dai
Articulated mesh generation hierarchically: first, articulation-aware part structure, followed by mesh faces of each part.
CVPR 2025
MeshPad
H. Li, Z. Erkoç, L. Li, D. Sirigatti, V. Rosov, A. Dai, M. Nießner
Interactive sketch-based mesh creation and editing, with each sketch edit mapped to triangle deletion and addition in a few seconds.
ICCV 2025
TriFlow
H. Li, Z. Erkoç, D. Sirigatti, V. Rosov, L. Li, A. Dai, M. Nießner
Artist-like mesh topology for input geometry, from a nearest-vertex vector field generated with flow matching.
ECCV 2026 (Oral)

All publications →