What the tests measure
A detector can find the mouse and still place its nose on a cup. We test finding the animal, placing landmarks and keeping the track on the mouse separately. None of those tests tells us by itself whether RONNIE scores a behaviour as an observer would.
The research record
We keep the question, test data, code and result for each experiment, including changes that didn’t help. The figures below come from the 24 October 2026 snapshot of that record.
Tracking benchmarks
Each panel describes a different test. Open its table for the comparison.
Finding the animal
96%
of the test frames had a correct detection (88 of 92, up from 22). The detector trained on 392 separate taps marking the mouse.
Test figures
| Measure | Before | After |
|---|---|---|
| Animal found on held-back tapped frames | 24% | 96% |
| Landmarks placed, stranger phase | 48% | 83% |
Steady tracking
0.2%
of frames in one test clip had a track on the apparatus after the tracking change, down from 28.1%.
Test figures
| Measure | Before | After |
|---|---|---|
| Frames drifting onto the apparatus | 28.1% | 0.22% |
| Flicker between frames | 3.73% | 0.09% |
Without labels
+7.3 points
in landmark accuracy on 2,724 test frames after adapting to new scenes without using their labels.
Test figures
| Measure | Before | After |
|---|---|---|
| Landmark accuracy, change | — | +7.3 points (5.6 to 9.1) |
| Frames with the animal found | 30.2% | 55.6% |
Our research
1,000+
dated entries in the experiment log between July and the October snapshot.
Entries by month
| Month | Entries |
|---|---|
| July 2026 | 165 |
| August 2026 | 295 |
| September 2026 | 529 |
| October 2026 | 35 |
| Dated entries | 1,024 |
What changed in the tests
Finding the mouse beside a cup
During the stranger phase, the detector sometimes framed the cup. Centring the same pose model on the mouse increased the share of frames with landmark predictions from 48% to 83%. In a separate test, we retrained the detector using 392 taps marking the animal.
Keeping the track off the apparatus
We added a tracking pass without retraining the model. The drift and flicker figures above come from one clip. We also tested the tracking change on five clips across three rigs.
Adapting to unfamiliar recordings
We split a public dataset of 5,460 human-annotated frames by scene. The model adapted to the held-back recordings without their labels; we used the labels to score its predictions. The gain in landmark accuracy was 7.3 points (95% interval: 5.6 to 9.1).
Questions we’re still testing
Can a mouse silhouette screen out misplaced paw and tail predictions before a model learns from them? On our own rig, filtering these predictions improved landmark accuracy by about 25 points. In a preliminary analysis, about 80% of the remaining error followed a consistent offset along the spine. We don’t yet know whether either finding holds on another rig.
Observer study with Sahlgrenska
We’re preparing a protocol with researchers at Sahlgrenska Academy. It asks whether RONNIE’s behavioural scores agree with trained observers as closely as those observers agree with each other.
- Define chamber entry, exit and contact, and agree on the analysis and success criteria before scoring.
- Have observers score the recordings without seeing the model output. Two observers will independently score a subset.
- Compare the model’s agreement with observers against their agreement with each other.
An earlier comparison covering six animals exposed a difference in where the middle chamber ended. The planned study needs a common boundary. It has not yet produced an observer-agreement or time-saving result.
Beyond the three-chamber test
Long home-cage recordings or changes in gait raise different questions. Each needs its own footage and human assessment. With a single overhead camera, grooming and close investigation can be hard to distinguish.
Background on movement patterns: Wiltschko et al. (2015), Mapping sub-second structure in mouse behavior, Neuron 88, 1121–1135; Weinreb et al. (2024), Keypoint-MoSeq: parsing behavior by linking point tracking to pose dynamics, Nature Methods 21, 1329–1339.
Interpreting a result
If two trackers disagree, review the frame. If they agree, they may still both be wrong. We use independent human references to assess accuracy; comparisons with a model’s own earlier predictions are development checks. The 2.59-pixel error on a 409-frame development set is one such check.
Keep the camera view and chamber-entry rule with any occupancy result. A different boundary can change the time in a chamber even when the track stays the same.