When a Fish Loses Its Balance, AI Marks the Moment — and Reveals Hidden Thermal Differences in Medaka

One of the clearest signs that a fish has reached its thermal limit is surprisingly simple: it can no longer stay upright. Researchers at Nagoya University and the University of Fukui have built an AI-based system to identify that moment automatically. By combining DeepLabCut markerless pose tracking with an image-classification model, the system detects loss of equilibrium, or LOE, from recorded video instead of requiring a researcher to watch fish continuously for hours. In validation, its disagreement with an experienced investigator was comparable to the disagreement between two trained human observers. The study was published in Scientific Reports on September 10.^{[1] [2] [3] [4]}

Once the scoring problem was automated, the biology became visible at scale. Six medaka strains did not respond identically to cold and heat stress. In a second comparison across seven Oryzias species plus zebrafish, Japanese medaka, Oryzias latipes, retained equilibrium the longest under the cold-stress protocol. A newly described northern Taiwanese species, Oryzias cabaranensis, also performed better than its latitude alone might predict. The broader lesson is not that one fish is simply “stronger” than another. Thermal tolerance emerges from genetic background, life stage, evolutionary history and local environment.^{[1] [10]}

A thermal endpoint that used to require hours of watching

Fish physiologists have several ways to measure thermal tolerance. Incipient lethal methods use mortality after exposure to fixed temperatures. Chronic methods change temperature slowly over days. Critical thermal methods, or CTM, instead shift temperature during a shorter experiment until the animal reaches a sublethal endpoint such as loss of equilibrium. If the fish is returned promptly to suitable conditions, LOE can provide a useful endpoint without requiring death.^[1]

The weakness has always been the observer. A fish rarely flips like a switch from “normal” to “failed.” Its body angle changes, swimming becomes unstable, and the precise moment of LOE can depend on how the researcher interprets what is happening. Long sessions introduce fatigue. Large experiments multiply the labor. If the biological difference between two strains is subtle, observer variation can compete with the effect being measured.

For this study, the team defined LOE operationally as the point when one of two previously visible pectoral fins disappears from the overhead view as the fish starts to roll to one side. The AI is therefore not making a vague judgment about distress. It is reproducing a defined behavioral scoring rule from video.^[1]

Seven body points, plus the whole image

The automated pipeline has two main layers. First, DeepLabCut tracks seven anatomical landmarks: the nose, left and right pectoral fins, the center of the body, two additional body points and the tail. DeepLabCut itself was introduced in 2018 as a deep-learning method for markerless tracking of user-defined animal body parts and has since become widely used in computational animal-behavior research.^[7]

The new experiment used a custom acrylic tank divided into 48 individual 40-by-40-millimeter chambers, placed inside an incubator and filmed from above. To make tracking more reliable, the researchers first segmented the video into individual chambers and applied optional preprocessing for shadows, color and contrast. Their keypoint detector used a ResNet50 backbone; 1,560 processed training images and 174 test images were prepared for that stage.^[1]

But keypoints alone were not enough. A second classifier, built on ResNet34, examined the image frame while also receiving the coordinates and confidence scores of the seven tracked points. The combined representation produced a probability that each frame showed normal behavior or LOE. The predictions were then converted into a time series, smoothed with a 30-second moving average and post-processed to identify the transition to the LOE state.^[1]

The AI did not “beat” the experts — it entered the range of expert disagreement

The most revealing part of the study may be its validation. Fifty fish videos were scored by the automated system and by an experienced investigator who had conducted the behavioral assays. Twelve people without prior experience also scored the same material.

Against the reference investigator, the automated system had a mean absolute error of 9.1 minutes, a root mean square error of 16.4 minutes and a Spearman correlation of 0.85. It also had a systematic tendency to call LOE earlier: its mean bias was −6.6 minutes.^[1]

The median inexperienced observer differed from the reference by 11.3 minutes MAE and 18.4 minutes RMSE. More importantly, a second experienced investigator differed from the reference by 9.5 minutes MAE and 18.0 minutes RMSE — essentially the same scale of disagreement as the machine. The authors therefore describe the automated error as falling within normal inter-observer variability among trained researchers.^[1]

Six medaka strains, six different trajectories

The researchers first tested six laboratory strains: three inbred strains derived from the southern Japanese lineage of O. latipes — HdrR-II1, HO5 and HB11A — one HNI-II strain derived from O. sakaizumii, and the d-rR/TOKYO and OK-Cab laboratory strains. Adult fish were acclimated at 26°C before the thermal assays.^[1]

In the cold assay, mean LOE times after the incubator setpoint was lowered were 86.8 minutes for HNI-II, 89.4 for HB11A, 92.3 for d-rR/TOKYO, 101.6 for OK-Cab, 106.7 for HO5 and 112.3 for HdrR-II1. HdrR-II1 had the longest mean time, but its statistical grouping overlapped with several other strains; it is more accurate to call it among the most cold-tolerant strains tested than uniquely superior.^[1]

In the heat assay, HNI-II again had the shortest mean LOE time at 89.1 minutes, while HdrR-II1 was longest at 105.8. Yet after multiple-comparison testing, only the HdrR-II1–HNI-II difference was statistically significant. The other rankings should not be inflated into strong claims of strain superiority.^[1]

The northern-derived line produced the most surprising result

HNI-II creates an apparent contradiction. Earlier work cited by the authors found that cell lines from northern-derived medaka showed stronger proliferation at low temperatures than southern-derived lines, and that HNI-II embryos maintained more stable heart rhythms under cold conditions than HdrR-II1 embryos. In the adult LOE assay, however, HNI-II had the shortest mean tolerance time under both cold and heat stress.^[1]

The authors argue that these measurements are not equivalent. Temperature response in cells and embryos can be dominated by properties such as membrane fluidity, protein stability and cardiac physiology. Adult LOE is an integrated whole-animal outcome involving the nervous system, muscle and cardiovascular system. A lineage can therefore perform well on one cold-related phenotype and poorly on another.

There is also a sampling limitation: HNI-II was the only O. sakaizumii-derived strain in the panel, while several tested O. latipes strains are relatively closely related. The authors explicitly caution that the six-strain dataset cannot represent the full diversity of northern and southern medaka lineages.^[1]

Across species, Japanese medaka stayed upright the longest

The researchers then widened the comparison. They tested cold tolerance in seven Oryzias species — O. latipes, O. sinensis, O. cabaranensis, O. curvinotus, O. luzonensis, O. celebensis and O. javanicus — plus zebrafish, Danio rerio. For O. latipes, they deliberately used an outbred wild-type population rather than a single laboratory strain because the first experiment had already shown how much tolerance could differ among strains.^[1]

O. latipes had the longest mean LOE time at 145.0 minutes and was significantly more cold-tolerant than all the other species tested. O. cabaranensis averaged 106.0 minutes and O. sinensis 99.2; those two did not differ significantly from each other. The remaining means were 72.8 for O. curvinotus, 66.9 for zebrafish, 58.8 for O. luzonensis, 53.5 for O. javanicus and 47.7 for O. celebensis.^[1]

Broadly, cold tolerance tracked native latitude: species from higher latitudes tended to remain upright longer. But O. cabaranensis and O. sinensis were relatively cold tolerant despite their lower-latitude distributions. The authors therefore point to phylogenetic history and local environmental conditions, not latitude alone, as likely contributors.^[1]

A Taiwanese species that science had only just named

Oryzias cabaranensis had been formally described as a new species only at the end of 2025. Collected in northern Taiwan, it had previously been treated as Japanese ricefish or Chinese ricefish. The taxonomic study distinguished it morphologically as a separate endemic species. Its name refers to Cabaran, an older name associated with Yilan; Taiwanese researchers use the Chinese name “Kavalan ricefish.”^{[10] [11]}

Its appearance near the top of the new cold-tolerance ranking is intriguing, but it must be read cautiously: only three O. cabaranensis individuals were included. The experiment does not establish a unique cold-adaptation gene or define the thermal limits of the wild Taiwanese population. It identifies a promising biological exception that can now be investigated with larger samples and genetic tools.

Why medaka make this more than an AI demonstration

Japan has spent decades turning medaka into a genetic model rather than simply a convenient small fish. In 2007, Japanese researchers published a draft medaka genome in Nature. DDBJ reported roughly 700 million bases and 20,141 identified genes, with more than 80% of genes showing similarity to human genes.^[8]

The National BioResource Project has since maintained inbred strains, mutants, transgenic lines and related species. Most of the laboratory strains and related Oryzias species in the new study came from NBRP Medaka. Researchers can cross strains, use dense genomic resources and move from a measurable trait to the genes responsible for it. That is what makes automated LOE scoring potentially powerful: it creates a high-throughput phenotype that can be paired with a mature genetic system.^[9]

This study did not predict climate-change losses

The university releases say the system could eventually contribute to predicting the effects of climate change on fish. “Eventually” is important. This study was conducted under controlled laboratory conditions with a limited set of strains and species. Wild populations experience daily and seasonal temperature cycles together with oxygen changes, salinity, food availability, flow, disease and ecological competition.

Some interspecies sample sizes were small: three fish each for O. celebensis and O. cabaranensis, four each for zebrafish and O. luzonensis. Those data are enough to expose interesting differences in this experiment, but not to produce population-level extinction forecasts.^[1]

The climate value comes from scalability. If the same objective endpoint can be measured across many strains, generations, acclimation histories and environmental treatments, researchers can begin separating inherited tolerance from plastic responses and local adaptation. AI is useful here not because it forecasts the future by itself, but because it makes a much larger and more reproducible biological dataset possible.

From one wobble to a gene

Specially Appointed Lecturer Tomoya Nakayama of Nagoya University lists temperature adaptation, seasonal adaptation, animal physiology and molecular biology among his research fields. Associate Professor Tatsuhito Hasegawa at the University of Fukui works in behavioral recognition and deep learning. The project sits directly at that intersection: biologists define a meaningful phenotype; computer scientists make it measurable without a human watching every frame.^{[5] [6]}

The next questions are biological. Which genes account for the HdrR-II1–HNI-II difference? Are cold tolerance and heat tolerance controlled by overlapping pathways? Why can embryo and adult measurements point in opposite directions? What gives northern Taiwan’s O. cabaranensis its relatively strong performance in the cold assay?

The machine has not answered those questions. It has made them easier to ask properly. A fish rolls a few degrees to one side; one pectoral fin disappears from a camera’s view; an algorithm puts a reproducible timestamp on the transition. That small number can now be connected to the unusually rich genetics of medaka. The moment a fish loses its balance may become a doorway into the molecular history of how vertebrates adapt to temperature.