Generalization acrosspeople and pairings.

Benchmark tasks

Not part of frozen Baseline V1

These appeared in earlier planning but are not published as frozen Baseline V1 benchmark results.

Evaluation protocols

A generalization ladder: each step moves the test set further from what the model saw in training.

1. 01

   Duplicate-safe fixed

   Reference benchmark

   Locked split of 444 train / 97 validation / 96 test records. Exact-duplicate audiovisual groups stay within one partition, removing duplicate leakage.

   444 train97 validation96 test
2. 02

   Pair-held-out

   Primary benchmark

   Four folds. Test participant pairs do not appear as pairs in training for that fold; individual participants may still be seen with different partners. The primary benchmark for unseen pair combinations.

3. 03

   Participant-held-out diagnostic

   Diagnostic

   One participant is absent from training and validation and appears only in diagnostic test interactions. Six participants are eligible for four-class evaluation and eight for binary evaluation. A complementary check on unseen-participant transfer.

Conceptual comparison of evaluation protocols

Reference benchmark · Unseen pair · Unseen participant

The protocols distinguish fitting familiar players from learning movement structure that transfers to unseen pairings and participants.

Baseline V1 results

Locked models

Macro-F1, all protocols

Highest value per row in bold.

Interpretation

• RGB and pose both perform strongly on the duplicate-safe fixed reference split.
• Under pair-held-out evaluation, pose retains substantially more performance than frozen RGB.
• Under participant-held-out diagnostics, pose again transfers better than RGB.
• Fusion is not consistently better than pose alone, especially on the binary task.

The headline V1 finding is about generalization across people and pairings — not high fixed-split accuracy.

Raw video refinement

Source gameplay recordings are refined into short canonical clips that preserve the complete Ampe interaction, with consistent record identifiers for annotation, pose extraction, skeleton rendering and benchmarking.

1. 01Screen

   Check source recordings for usable two-player visibility and stable capture.

2. 02Segment

   Cut continuous play into short clips around the jump–clap–leg reveal cycle.

3. 03Label

   Assign the canonical movement token.

4. 04Trim

   Retain enough temporal context around the interaction.

5. 05Verify

   Review visually; keep the C### identifier across all downstream representations.

Publication principle

Refinement preserves the natural movement dynamics of the source interaction. This documentation describes the published V1 pipeline, not earlier exploratory segmentation claims that are not reflected in the frozen release.

Pose & skeleton extraction

V1 uses 33-landmark MediaPipe pose outputs for each represented player and stores the canonical normalized representation as long-form CSV rows.

Processing

1. 01
   Process canonical RGB clips frame by frame.
2. 02
   Maintain two-person track identity through the clip using the extraction/tracking pipeline and release mappings.
3. 03
   Store normalized x/y coordinates and visibility values in the canonical pose CSV.
4. 04
   Export corresponding pixel-coordinate skeleton CSV and JSON for rendering.
5. 05
   Generate rendered skeleton MP4 videos for qualitative inspection.

Missing detections

Pose estimation is imperfect and some frame-player groups are incomplete. Baseline V1 model preparation includes presence indicators rather than assuming all 66 player-landmark positions are observed in every frame.

Quality control & validation

Release-level QC goals

• 1:1 identifier coverage across RGB, normalized pose, skeleton CSV, skeleton JSON and skeleton video.
• Readable RGB media with synchronized embedded audio.
• Pose/skeleton frame, player and landmark schema consistency.
• Participant/pair mappings consistent with canonical clip IDs.
• Exact-duplicate groups documented and kept within fixed benchmark partitions.
• Split definitions and label files validated against the canonical manifest.

Skeleton review

Rendered skeleton motion is compared with the original RGB footage. Structural checks validate frame counts and CSV/JSON consistency, while manual review identifies identity swapping, limb misalignment, tracking drift or localized anatomy errors. Repairs are localized and revalidated.

Benchmark freeze

Ampe-DB Baseline V1 passed 35 freeze checks with 0 failures, using fixed random seeds 17, 29 and 43.

Known limitations

Documented openly so results can be interpreted within the scope of the release.

Scale

637 canonical clips from 15 participants and 11 unordered pairs is focused rather than large-scale.

Exact duplicates

557 unique complete-file RGB/audio hashes across 637 records. Provenance is transparent and controlled in the fixed split, but duplicate content remains a source-corpus limitation.

2D front view

Pose provides no depth and should not be treated as 3D biomechanics or marker-based motion capture.

Pose estimation noise

Fast jumps, overlap and low-confidence detections can produce missing or imperfect landmarks.

Non-uniform media

Most clips are 30 FPS, with smaller 24 FPS and 60 FPS subsets; H.264 and HEVC both occur.

Participant-held-out coverage

Only participants with sufficient class coverage are eligible: 6 for the four-class task and 8 for the binary diagnostic.

Fusion stability

Simple late fusion is not uniformly superior to pose, and some RGB-heavy alpha selections are unstable in diagnostics.

Cultural scope

The corpus does not represent all Ampe variants or African movement broadly.

Skeleton visualizations

Derivative QC artifacts; they do not certify perfect anatomical reconstruction in every frame.

Related publications

Citation source

Use the release CITATION.cff file for authoritative citation metadata, including any repository or DOI fields once assigned.

Contextual literature

The research paper situates Ampe-DB alongside established human action, human–human interaction, dance and motion, and pose-estimation datasets. Refer to the paper’s bibliography for the full reference list.