Egocentric Video Annotation for Physical AI & Robotics.
High-precision spatiotemporal supervision engineered for imitation learning, human-to-robot kinematic retargeting, and Vision-Language-Action (VLA) foundation models. Built to rigorous Ego4D, Ego-Exo4D, and HOT3D benchmark standards.
A standard bounding box tells a model what an object is. It tells an embodied policy nothing about how to grasp it.
Standard data labeling companies slice video into isolated frame contact sheets and query generic VLMs. In physical manipulation, this causes disastrous policy jitter, temporal aliasing, and missed contact mechanics. High-quality egocentric annotation requires sub-10ms boundary precision, 3D hand mesh recovery, 6-DoF object poses, and explicit contact phase demarcation.
What High Quality Means in First-Person Video
Pre-Condition
Captures stable initial spatial configuration before hand intent initiates. Amodal object bounding boxes and environmental context established.
Contact Initiation
Exact millisecond the finger pad touches object surface. Per-finger contact flags (thumb, index, palm) and normal vectors logged without temporal jitter.
Point of No Return
The irreversible state transition milestone: valve breached, tool locked, screw threaded, or surface deformed. The anchor for temporal reinforcement learning.
Post-Condition
Grasp release or terminal placement verification. Object rest stability confirmed, verifying episode success or failure classification.
21-Keypoint 3D MANO Meshes
Dual-hand persistent tracking through heavy tool occlusion. We run HaMeR-based pre-annotation backed by expert multi-pass human correction, yielding continuous joint angles, wrist 6-DoF trajectories, and MPJPE < 8mm.
Dynamic Object State & BBox
Tracking rigid and articulated object poses relative to camera and wrist frames. Amodal bounding boxes preserve continuous 3D bounding geometry even when 80% occluded by human hands or robotic grippers.
Gaze Grounding & Audio Transients
Eye tracking gaze fixations mapped directly to interaction target objects with angular error ≤ 2.0°. Paired with acoustic transient segmentation (impact clicks, latch snaps) for multimodal attention conditioning.
Hierarchical Annotation Stack
Episode & Scenario Metadata
High-level intent descriptions in canonical natural language, environment lighting classification, camera lens distortion parameters, and binary/graded success definitions.
Temporal Phase Segmentation
Division of long-horizon tasks into atomic verbs and sub-goals (reach, align, grasp, transport, seat, actuate, release). Ego4D-compliant grammar with actor (#C) tags.
Contact Dynamics & Milestone Keyframes
Identification of PRE, CONTACT, Point-of-No-Return (PNR), and POST condition keyframes. Per-finger contact topology, grasp class, and state transformation triggers.
Dense 3D Pose, Mesh & Gaze Stream
Continuous 21-joint MANO hand mesh parameters, 6-DoF object poses, eye gaze fixation points, and amodal 3D bounding geometry synchronized at 30–60Hz.
Order a benchmark pilot batch of 5 to 10 hours with full QA reports.
Send us your raw egocentric footage or custom sensor logs. We return annotated episodes conforming to your exact taxonomy, complete with inter-annotator agreement metrics and MANO tensors.