D4D Dataset

Experimental Setup

Data were acquired from porcine cadavers using a da Vinci Xi stereo endoscope and a Zivid structured-light camera, registered via optical tracking. The dataset enables quantitative geometric evaluation of 3D reconstruction in both visible and occluded regions.

Sample Sessions

The dataset follows a three-level hierarchy: each Specimen (one porcine cadaver) contains one or more Sessions (continuous recordings, named by date and time), and each Session contains one or more Clips (the individual tissue-manipulation segments). Each slide below is one Session, and every Session holds one or more Clips with paired endoscopic video and structured-light geometry.

Specimen 1 - Session 2025_03_06-16_49_40

← Previous Sample Session 1 / 4 Next →

Summary images display (left to right, for each clip row):

1. Left endoscopic image (rectified stereo view)
2. SLC RGB rendering (point cloud from curated pose)
3. Stereo depth map (from stereo matching)
4. SLC depth rendering (structured-light depth)

Combined 2x2 View

Endoscope Video

Zivid Start/End

Combined 2x2 View

Endoscope Video

Zivid Start/End

Combined 2x2 View

Endoscope Video

Zivid Start/End

Moved-camera session. The endoscope is repositioned partway through, giving two viewpoints within one continuous recording. The Zivid camera does not move, so only the endoscopic view changes.

Combined 2x2 View

Zivid Start/End

Installation

Requires Python 3.10

pip install -e .

Usage

from d4d.loader import D4D
from visualize import visualize_clip_state

# Load dataset
d4d = D4D("/path/to/preprocessed_restructured")

# Iterate: Dataset → Specimen → Session → Clip
for specimen in d4d:
    for session in specimen:
        for clip in session:
            print(f"{clip.name}: {len(clip.left_img_paths)} images, {clip.duration:.1f}s")

# Access specific clip
specimen = next(iter(d4d))
session = next(iter(specimen))
clip = next(iter(session))

# Clip properties
clip.left_img_paths          # List of left image paths
clip.right_img_paths         # List of right image paths
clip.stereo_depth_paths      # List of depth map paths
clip.pointclouds             # Dict with 'start'/'end' Zivid PLY paths
clip.endoscope_params        # Endoscope camera parameters
clip.zivid_params            # Zivid camera parameters
clip.poses                   # Dict with 'start'/'end' curated camera poses

# Visualize with curated poses
if clip.pointclouds.get('start') and clip.left_img_paths and clip.stereo_depth_paths:
    visualize_clip_state(
        clip.pointclouds['start'],
        clip.left_img_paths[0],
        clip.stereo_depth_paths[0],
        clip.endoscope_params,
        clip.poses['start']
    )

Dataset Structure

The dataset is organised by the Specimen / Session / Clip hierarchy. Session-level folders hold the full unrectified images, while clip-level folders hold the rectified, per-clip data.

Dependencies