Robot Vision Data Splits: Keep Related Sessions Together Before Testing

A robot vision model can score well on a test set even when the test examples are closely related to the training examples. Consecutive video frames, repeated trials with the same object or recordings from one workcell can share information. A useful evaluation begins by asking what kind of new situation the model is expected to handle and whether the split actually tests that situation.

Choose the unit of independence for the question

If the goal is to evaluate a model on a new recording session, frames from one session should not be casually scattered across training and testing. If the intended question concerns new physical objects or sites, a different grouping may be needed. There is no universally correct split: the grouping should reflect the deployment question and the available data, and its limitations should be documented.

Scikit-learn's grouped cross-validation documentation describes keeping groups separate between training and test folds. It is a general method, not a robotics-specific guarantee. The dataset owner must choose meaningful groups and confirm that the resulting test covers the situations relevant to the application.

Find related examples before splitting

Record identifiers for recording session, robot, camera, location, object and other relevant sources. Near-duplicate images may arise from adjacent frames or repeated exports of the same recording. A random frame-level split can put nearly identical views on both sides, making the test less informative about performance on a new session. If provenance is incomplete, state that limitation and investigate before claiming independence.

As a hypothetical example, a robot records a box from many angles during one visit. Placing some frames from that visit in training and others in testing answers a different question from holding out the whole visit. Neither split automatically proves performance on an unseen box or a new factory; those require a test designed around that question.

Keep preprocessing inside the training process

Data leakage can also occur when information from the test set influences preprocessing or model selection. Scikit-learn's common pitfalls guidance explains why transformations learned from data should be fitted on the training subset and then applied consistently to later data. The exact pipeline depends on the model, but the principle is to prevent information unavailable at deployment from influencing training or tuning.

Document how normalization, feature selection, augmentation and model choices were handled. Keep a final evaluation set separate from repeated development decisions when the study design calls for one. If results were used to choose among models, describe that use; the numbers should not be presented as an untouched final assessment without qualification.

Check the distribution as well as the boundary

Grouping can reduce overlap while creating a test set with few examples of a rare condition. Review the number of groups, examples and relevant categories in each partition. A small number of workcells or objects can make results unstable, even when there are many frames. Report the composition and avoid treating repeated frames as independent evidence of performance across many environments.

For safety-relevant robotics, a dataset score alone does not establish that a system is safe in the real world. Evaluation should be designed by qualified people and supplemented by the testing required for the intended application. If a condition is absent from the test data, say that it has not been assessed rather than reporting a score for it.

A useful report can present results by group or scenario as well as an overall score, provided that each group has enough data for the intended interpretation. A strong aggregate result can conceal poor performance in one lighting condition or at one site. Conversely, a very small subgroup can yield an unstable estimate. Include the sample counts and the uncertainty or limitations appropriate to the analysis, and avoid generalizing beyond the recorded environments. Where data collection continues, agree in advance how later samples will be assigned so that the evaluation boundary remains meaningful.

Preserve the split for reproducibility

Save the dataset and annotation versions, group definitions, split method, random seed where applicable and the identifiers assigned to each partition. Record any exclusions or corrections and retain the code or procedure used. Protect sensitive identifiers according to the dataset owner's rules. When new data arrives, create a documented revision instead of silently changing the earlier test set.

For related data quality questions, see our guide to robotics dataset labels and the robotics data collection. A clear split makes the result easier to interpret; it still needs to be matched to the intended deployment and independently validated where required.