TopoOR

A Unified Topological Scene Representation
for the Operating Room

Tony Danjun Wang  ·  Ka Young Kim  ·  Tolga Birdal  ·  Nassir Navab  ·  Lennart Bastian

_{Technical University of Munich  ·  Munich Center for Machine Learning  ·  Imperial College London  ·  Kyung Hee University}

---

With TopoOR we propose a unified higher-order topological representation that captures group interactions among people, tools, and robots and integrates information from 3D geometry, audio, and robot sensors.

Installation

Run all commands from the repository root. Install uv, then run:

uv sync --locked

Data and weights

Request MM-OR through its dataset download instructions, then copy or symlink the downloaded MM-OR_data/MM-OR_processed/ directory to data/mm_or/.

Download the packed boxes, poses, embeddings, and released checkpoints:

bash scripts/download.sh

The packed modalities cover both training and evaluation; no preprocessing step is needed. The script uses uv and gdown and skips existing nonempty files. For manual downloads, copy checkpoints/ and data/ from the TopoOR/ directory in the Google Drive folder into the repository root.

Expected directory layout

data/
├── mm_or/
│   ├── take_timestamp_to_next_action/<take>.json
│   ├── take_timestamp_to_robot_phase/<take>.json
│   ├── take_timestamp_to_sterility_breach/<take>.json
│   └── <take>/
│       ├── timestamp_to_pcd_and_frames_list.json   # ..._azure.json for 010_PKA
│       ├── camera{01..05}.json
│       ├── colorimage/camera{01..05}_colorimage-<frame>.jpg
│       ├── simstation/camera01_<frame>.jpg
│       └── relation_labels/<frame>.json
└── perception/
    ├── bounding_boxes.pkl
    ├── human_poses.pkl
    ├── audio_embeddings.pkl
    ├── text_embeddings.pkl
    └── json_embeddings.pkl

checkpoints/
├── workflow_recognition/
│   ├── hparams.yaml
│   └── checkpoints/latest_step_012000.ckpt
└── relation_prediction/
    ├── hparams.yaml
    └── checkpoints/latest_step_024000.ckpt

Training

WANDB_MODE=offline .venv/bin/python src/train.py experiment=workflow_recognition
WANDB_MODE=offline .venv/bin/python src/train.py experiment=relation_prediction

Runs write to logs/<experiment_name>/.

Warning

Due to the extremely limited dataset size, scores can vary with the environment.

Evaluation

For workflow recognition (Next Action and Robot Phase), run:

.venv/bin/python src/eval.py --run checkpoints/workflow_recognition --output output/workflow_results.json

Note

Robot Phase excludes 004_PKA as that take contains annotation artifacts.

For sterility breach detection, run:

.venv/bin/python scripts/sterility_breach_eval.py

For relation prediction, run:

.venv/bin/python scripts/relation_prediction_eval.py --run checkpoints/relation_prediction --out output/relation_logits.npz

To evaluate a trained model, replace --run checkpoints/... with --run logs/<experiment_name>. The evaluators load that run's configuration and latest checkpoint.

Citation

@inproceedings{wang2026topoor,
  author    = {Wang, Tony Danjun and Kim, Ka Young and Birdal, Tolga
               and Navab, Nassir and Bastian, Lennart},
  title     = {{TopoOR}: A Unified Topological Scene Representation
               for the Operating Room},
  booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
  year      = {2026},
  publisher = {Springer Nature Switzerland},
  series    = {Lecture Notes in Computer Science},
  volume    = {16892},
  month     = sep,
  url       = {https://papers.miccai.org/miccai-2026/1068-Paper2412.html}
}

License

MIT.