I Audited 26 Episodes of a 59,000-Episode Open Robot Dataset. The Data Is Fine — the Metadata Isn't.

A few numbers first, because they set up everything else.

There is an organisation called OpenLET publishing real-robot manipulation datasets on AtomGit (a Chinese Git hosting platform). Their headline claim is "over 100,000 minutes of open-source real-robot data". The main repo, LET-Base-Dataset, advertises 60,976 rosbag files and 926.3 hours.

I wanted to use this data. Before writing a single line of training code, I wanted to know where it would break. So I audited it.

Scope, stated up front: 26 episodes from the Kuavo4Pro platform, spanning 25 task directories — car sheet metal loading, SPS parts picking and sorting, iron block tooling, conveyor sorting, grocery loading, microwave placement, folder handover, bottle sorting, shampoo application. Real industrial and retail scenes, not toy tasks. That is still a small sample — 0.05% of the dataset. I therefore split every finding below into two buckets:

• Exact verification — computed from the full file tree, independent of sampling. Reproducible to the byte.
• Sample observation — describes these 26 episodes only. Not extrapolated to population proportions.

The tool is RDA (robot-data-audit), which I wrote. Treat that as a conflict of interest and check my numbers — every figure below traces back to a per-episode report I still have on disk.

Layer 1: Counting files is cheaper than you think

The first thing worth auditing is never the model. It's the file list.

You can pull the full directory tree without downloading a single binary:

git clone --no-checkout --depth 1 https://atomgit.com/OpenLET/LET-Base-Dataset.git cd LET-Base-Dataset git ls-tree -r HEAD --name-only | grep -c '\.bag$' # 60976 ← matches the README exactly 

Good. Now filter out the noise:

git ls-tree -r HEAD --name-only | grep '\.bag$' | grep -vc '/\._' # 59045 

1,931 files begin with ._.

These are macOS AppleDouble resource forks. If you have ever copied a directory to an external drive from a Mac, you have seen them: hidden sidecar files holding Finder metadata, tens to hundreds of bytes each, not rosbag data.

They are committed into the repository and counted as episodes.

Three consequences:

1. The real episode count is 59,045, not 60,976 — 3.2% lower.
2. Any script that walks the file list and feeds each path to a rosbag reader will crash on them. Mine did, first run.
3. Any duration estimate built on file count inherits the 3.2% inflation.

One nuance I only caught on a second pass: those 59,045 include 7,920 simulation bags (four directories under rosbag/sim/). Pure real-robot count is 51,125 (Kuavo4Pro 50,622 + Kuavo5W 503). The volume math later in this post uses the 59,045 denominator; treat it as "all bags, sim included".

The annotation files are clean, for what it's worth:

git ls-tree -r HEAD --name-only | grep -c '\.json$' # 30746 ← matches the README 

Task directory distribution is highly uneven: 82 directories, ranging from 11 to 3,015 episodes, median 316. If you sample uniformly by directory you will massively over-weight the small tasks.

And two more dimensions of composition, also counted exactly from paths:

Both matter more than they look:

1. 99% of the data comes from a single robot model. "Open real-robot data" sounds like a multi-embodiment collection; it's one platform at volume. Cross-embodiment transfer can't be validated on this.
2. 41% carries no annotation. That's not a defect — unlabelled data is useful for self-supervised pretraining. But it means every annotation complaint below applies to only half the dataset.

Layer 2: 97.3% of the stated duration is fitted, not measured

The README's 926.3 hours decomposes like this:

• 2,049 trajectories have durations measured from real timestamps in their annotation JSON.
• 59,348 have durations inferred from a fitted bag-size ↔ duration relationship.

So 97.3% of the headline duration is an estimate, not a measurement. That's not inherently wrong — bag size and duration are strongly correlated. The problem is that the number gets quoted downstream as if it were measured.

There's also a self-consistency issue:

2,049 + 59,348 = 61,397 

But there are only 60,976 bags. 421 too many. Two different counting bases were used and never reconciled.

Then there's the gap to the headline claim:

Publicly verifiable: about 57k minutes. Headline claim: 100k minutes. The missing 43k could be contributed data, unpublished subsets, or the Body repo's undisclosed duration — I have no evidence it doesn't exist. But the verifiable half is half.

I recomputed duration by a completely independent route, trusting no stated number:

Measured bitrate (bytes ÷ real duration across 26 bags): 21.9 MB/s 59,045 episodes × 1.19 GB/episode (mean of sampling pool) ≈ 70.2 TB 70.2 TB ÷ 21.9 MB/s ≈ 53,485 minutes ≈ 891 hours 

Within 4% of the README's 926.3 hours. So the 926-hour figure is basically honest. The 100k-minute figure is a different, broader scope. The mistake is mixing them.

(That 70 TB also explains why nobody audits this dataset in full. At my measured 5–13 MB/s, a complete download is 1,500+ hours. Hence 26 episodes.)

Sample durations: median 26.6 s, mean 27.4 s, p10 3.8 s, p90 48.0 s, range 3.1–70.6 s.

One methodological note, because it changes the arithmetic: I sampled smallest episodes first to save bandwidth, so my sample skews short. That's why the extrapolation uses the sampling pool's mean bag size, not the sample's mean duration. Mixing those two up would halve the estimate.

Layer 3: The timestamp problem — and it's not where you'd expect

This is the finding I didn't anticipate, and the most actionable one.

While computing per-topic publish rates, /sensors_data_raw (raw joint stream, nominally 500 Hz) came out anywhere between 466 Hz and 999 Hz across episodes. A 2× spread.

My first assumption was "the nominal rate is wrong". The real answer is more interesting. I compared two timestamps on the same camera stream — the bag record time (when the message hit disk) and the message header stamp (when the camera exposed the frame):

from rosbags.rosbag1 import Reader from rosbags.typesys import Stores, get_typestore store = get_typestore(Stores.ROS1_NOETIC) rec, stamp = [], [] with Reader(path) as reader: for conn, t, raw in reader.messages(): if conn.topic != "/cam_h/color/image_raw/compressed": continue msg = store.deserialize_ros1(raw, conn.msgtype) rec.append(t) stamp.append(msg.header.stamp.sec * 10**9 + msg.header.stamp.nanosec) 

Result:

record interval p1 3.87 ms p50 33.26 ms p99 170.63 ms stamp interval p1 32.09 ms p50 33.37 ms p99 34.65 ms record − stamp lag p50 18.41 ms p99 397.66 ms max 511.9 ms 

The header stamps are immaculate. p1 to p99 spans 32.1–34.7 ms — a rock-steady 30 Hz, drifting by less than a millisecond across 22 seconds.

The record times are a mess — intervals jumping from 3.87 ms to 170 ms, lag ranging 18 ms to 512 ms.

The conclusion is unambiguous:

The camera is sampling cleanly. The rosbag writer is what's jittery. Messages get buffered and flushed in batches, so record time reflects disk scheduling, not physical sampling.

Which means:

Any downstream pipeline that aligns modalities on bag record time is wrong on this dataset, by up to half a second.

Half a second of vision-action misalignment is fatal for imitation learning. Your policy learns "see A, act half a second later" instead of "see A, act".

Now the joint stream. Counting messages per wall-clock second:

Episode A: [1001, 998, 999, 997, 995, 1004, 596, 28, 79] ← 1000 Hz, then collapse Episode B: [502, 499, 500, 498, 500, 500, 497, 484, 501, ...] ← 500 Hz, stable for 24 s Episode C: [1002, 799, 1184, 1000, 963, 41, 672, 704, 1130, 943, 818, 671, 937, 1089, 993, 953, 604, 887, 1193, 1729, 1003, 662, 68] ← 41 to 1729 

Across 26 episodes:

• 20 episodes ≈ 500 Hz (466–500)
• 4 episodes ≈ 1000 Hz (991–999)
• 2 episodes in between (718, 858) — mixed within the episode

Same platform, same nominal spec, 2× difference in actual rate, plus intra-episode switching. That's a capture-side configuration inconsistency, not a measurement artifact.

And all three episodes show the same tail behaviour: the last 0.5–1.5 seconds collapse across every stream simultaneously — counts dropping to single digits per second, like a camera being stopped mid-recording. If that tail is consumed as normal data, multi-modal alignment in the final second is garbage.

Practical takeaway: don't assume a fixed rate. Resample on header.stamp, and drop the last 1.5 s of every episode.

Half the action vector is dead

/kuavo_arm_traj is 14-dimensional (two 7-DoF arms). I measured per-dimension variance:

10 of 26 episodes have only 7 dimensions varying. The other 7 are identically zero for the entire episode.

Cause: these are single-arm tasks. One arm's 7 slots are left empty, which is semantically reasonable. The directory names back this up — car_sheet_metal_square_right_hand, car_sheet_metal_loading_square_QR.

For training, it's a trap:

1. The model outputs 14 dims, 7 of which are always zero — half the output capacity wasted.
2. Worse, the model happily learns "emit a constant on half the channels", and validation loss still looks fine.
3. Which half is dead is not fixed. Checking per episode: 9 of the 10 dead-dim episodes are zeros at indices 0–6, one is zeros at indices 7–13. So this dataset contains both "only arm A moves" and "only arm B moves" variants. A dataloader must never hardcode "the first 7 dims are one arm" — detect per episode by variance.

(An earlier draft of this section claimed the layout also varies within a task category. I checked, and I can't support that: only one of the 25 directories has two samples, and both are 7-dim. Retracted. All I can say is that dimensionality and dead-dim position vary across tasks; within-task consistency needs a larger sample.)

Suggested handling: check per-episode variance at load time, build separate dataloaders by effective dimensionality, or mask the dead dims.

A place where I initially got the causation wrong, worth spelling out. I tried to auto-derive the action↔state mapping by cross-correlation (which action dim corresponds to which of the 28 state dims). 15 of 26 episodes resolved unambiguously; 11 did not.

I assumed the failures were the single-arm episodes — seven all-zero dims give cross-correlation nothing to lock onto. Checking: only 3 of the 11 are 7-dim. Eight are fully 14-dim, and they fail because a few individual dims have near-zero-lag correlation as low as 0.16–0.55 (median correlation is still 0.61–0.97).

Probable cause is tracking lag: action is the commanded target angle, state is the measured angle, and the measured value takes time to catch up. On high-inertia shoulder joints with gravity compensation, that lag is largest, which drags zero-lag correlation down. This is a hypothesis — I did not run a lag sweep to confirm it.

Either way, the downstream implication is the same: don't assume action ≈ next-frame state. On those joints in those 8 episodes, that assumption collapses command and response into one quantity — which is exactly the mapping the policy is supposed to learn.

Annotations carry less signal than the file count suggests

One JSON per episode, ~844 bytes. I parsed 25 of 26 (one episode has no JSON):

• Schema is consistent: 12 keys (deviceSn, initSceneText, primaryScene, marks, …)
• Median field fill rate: 37.5% — six of ten fields empty on average
• 1 episode has an empty marks array — no action annotation at all
• Only 3 episodes are fully populated

marks is the interesting loss. It should carry the temporal "what is happening now" annotation — the task semantics. Empty means you have video plus trajectory: the model can learn what happens next, not why.

Usable semantics reduce to two levels: the directory name and taskName.

What actually passed

To keep the balance honest: the data itself is clean.

RDA runs 23 checks. All 26 episodes passed.

No dirty data. No corrupt frames. No NaNs. The format is sound. The problems are in the metadata and the time axis, not in the recordings. Both are fixable without re-collecting anything.

Two checks did fail:

• Idle ratio: median 16.2%, but 8 episodes exceed 50% and 3 exceed 80%. A lot of standing-still footage. For imitation learning, long static stretches dilute the signal — consider dropping idle segments.
• Action discontinuity: 5 episodes show spikes. I have not verified case-by-case whether these are legitimate gripper open/close transitions or genuine capture glitches.

Also worth noting: workspace coverage median 5.7%. That measures how much 3D volume the end effector traverses. 5.7% means motion is highly concentrated. Fine for single-task data; not enough if you want manipulation diversity.

What I could not audit

This matters as much as what I found.

1. I audited 1 of 6 visual streams. Each bag contains six: head / left wrist / right wrist, each with colour and depth. I converted only head colour, to control download volume. Wrist views and all depth data were not audited.
2. Joint limit checking never ran. It needs a Kuavo URDF limit table, which the repo doesn't ship. So whether the 14 arm dims exceed physical limits is unknown.
3. Sensor synchronisation was not checked. rosbag preserves per-topic timestamps; after conversion to LeRobot there is one timeline and the original inter-stream sync is gone. That check can't run on converted data.
4. Kuavo4Pro only, and Labelled only. All 26 episodes sit under the Labelled split (28,821 episodes total). The 24,304 Unlabelled episodes — 41% of the dataset — were not audited at all. Neither were the 503 Kuavo5W episodes.
5. 26 episodes cannot be extrapolated. I sampled the smallest episodes first (bandwidth). Durations skew short. Do not read population proportions off this.

Six fixes I'd make if this were my dataset

All low-cost, high-leverage:

1. Delete the ._* files, or gitignore them. One command. Saves every user hours of debugging.
2. Separate measured from inferred durations in the README. Say which of the 926.3 hours is measured and which is fitted from file size. That's not a mark against the project — it's professionalism.
3. Ship a stats.json. Real duration, message counts, per-stream rates for every episode. Costs one pass over bag headers to generate; saves each consumer a day of work. I spent most of this audit building exactly that.
4. Fix the timestamps, or document them. Either write record time from header.stamp at capture, or state plainly in the README: use header.stamp, record time is unreliable. The second option is free.
5. Fill or mark inactive action dimensions. For single-arm tasks, either write NaN so downstream knows it's missing, or add active_dims: [7..13] to the metadata. Constant zero is the worst option — it's indistinguishable from "this joint genuinely shouldn't move".
6. Ship the URDF limit table. That single file turns joint-limit checking from a no-op into a real test, and it's the check most worth running.

Reproducing this

pip install robot-data-audit rda audit /path/to/lerobot_dataset --full 

Pulling the data needs no login — the LFS batch API hands out CDN links directly:

curl -s -X POST https://atomgit.com/OpenLET/LET-Base-Dataset.git/info/lfs/objects/batch \ -H "Accept: application/vnd.git-lfs+json" \ -H "Content-Type: application/vnd.git-lfs+json" \ -d '{"operation":"download","transfers":["basic"], "objects":[{"oid":"<lfs-oid>","size":<bytes>}]}' 

The returned href supports HTTP Range requests, and the file's sha256 equals its LFS oid, so transfer integrity is verifiable per episode.

Raw data is rosbag1; RDA reads LeRobot format only, so a conversion step is required (rosbags + av for H.264 encoding, 10 Hz resampling). Per-episode reports — check results, conversion details, raw stream statistics — are kept locally; ask if you want them for line-by-line verification.

Three caveats, repeated because they matter: the sample is 26 episodes, not a census; only Kuavo4Pro was touched; and RDA is my own tool, so treat its verdicts with appropriate suspicion and verify against the raw numbers.

The data is real, and the capture work is decent. **What this dataset lacks isn't data — it's an honest metadata layer.