TrueTrace / Blog

Buying egocentric data for robot learning: 8 checks before you sign

By Danqian Lou, Founder · October 20, 2026

If you are buying first-person human video to train manipulation policies or world models, most of what decides whether the data is usable is invisible in a demo reel. Below are the checks we run on our own recordings and the questions we think a buyer should ask any vendor, including us. Each one comes with what a good answer looks like.

1. Ask for the sync distribution, not a "synchronized" checkbox

Multi-camera and IMU streams drift, drop frames and restart. "Hardware-synchronized" is a design claim; what you want is a measurement.

Ask for the per-pair timestamp difference between cameras across a whole recording, and look at the 99th percentile, not the mean. A short burst of misaligned frames disappears in an average. On our reference stereo recording, 3,750 frame pairs have a median difference of 0.008 ms and a p99 of 0.014 ms; the gate we apply is p99 ≤ 1 ms.

Ask the same of frame intervals (any interval much longer than the median is a dropped frame) and of IMU samples (gaps).

2. Check the calibration against the footage itself

A stereo rig's calibration is measured once, at the factory or on a bench. Headsets get knocked. If the extrinsics no longer match the hardware, every depth map and 3D keypoint built on them is quietly wrong.

There is a simple test: rectify sampled frame pairs with the delivered calibration and match features between the two views. With a correct calibration, matching points sit on the same image row. Ask the vendor for the median vertical offset.

We run this on every recording, and it does fail. On our own evaluation sample the factory extrinsics gave a median vertical offset of 1.49 px against our 1 px gate. We refined the stereo rotation from the scene (a 0.21° correction), which brought it to 0.75 px, and we deliver both versions. A vendor who has never seen this check fail is probably not running it.

3. Understand what a 3D hand number means

"3D hand pose" can mean very different things. Questions worth asking:

• How is it produced? Monocular estimation, stereo triangulation and motion capture are different classes of data.
• What does the error figure measure? Reprojection error, the usual figure for stereo hands, measures how well the two views agree with each other. It is not accuracy against ground truth. Ours is a median of 3.16 px on the evaluation clip; that tells you the views are consistent, nothing more.
• What happens when detection fails? Frames should be dropped and marked, not filled with guesses.
• Do you get the raw points? If the vendor fits a hand model, ask for the raw triangulated points as well, so you can apply your own filtering.

4. Run a sample through your own loader before talking about volume

Formats look compatible on a slide and break in a dataloader. Before negotiating hours, ask for one episode in your target format (HDF5, RLDS, LeRobot or your own schema) and load it end to end. Check the coordinate frame, the units, which clock the timestamps are on, and whether episode boundaries and per-frame labels line up with the video.

A vendor should be able to document the layout to file level. Ours is public at /data-format/.

5. Agree how hours are counted before collection starts

This is where we learned the most. In our first delivery, 98.0% of the segments the buyer scored were rated valid, and some additional footage was still not counted because it exceeded the buyer's platform quotas: limits on how much the same task at the same site could count.

The lesson is that hours are not the product; hours that count under your rules are. Give the vendor your per-site, per-task and per-operator caps up front and ask them to return a scene-by-scene plan of billable hours under those rules before anything is committed. If they cannot do that arithmetic, you will end up paying for footage you cannot use.

6. Look at where the footage comes from

Ask who is on camera and why. Real workers doing their own jobs during normal operations produce different behaviour from people performing a task for the camera: pace, shortcuts, interruptions, error recovery. Neither is wrong, but you should know which one you are buying. Ask for the environment and task list per batch, not a list of the vendor's best sites.

7. Check consent and provenance on paper

For footage of people at work, ask:

• Does every person on camera sign written consent, and does it cover delivery to you and your designated third parties?
• Is there a written agreement with each site owner?
• How are faces and other personal information handled before delivery, and does the vendor describe it accurately (de-identification is not anonymisation)?
• Is there a capture ledger and a SHA256 manifest, so you can tie each file to a session?
• What license does the sample come under, and what will the commercial license allow?

8. Ask what the vendor does not do

A useful last question: "What are the known limitations of this data?" Every dataset has them: label boundaries with a tolerance of seconds, left/right hand labels that swap in first-person views, depth that is a preview rather than delivery grade. A vendor who writes these down in the package README is easier to work with than one who only learns about them from your QA team.

How we apply this: every check above is part of our standard QC, and the method behind each gate is written up at /quality/. If you want to run these checks yourself, our evaluation package includes the stereo video, IMU, calibration, 3D hands, action segments and the raw QC metrics: contact@truetracedata.com.