MIT unveils computer vision system to count fish in citizen science projects

MIT Sea Grant, together with the Woodwell Climate Research Center, CSAIL and other partners, has unveiled a deep learning system that counts fish in underwater video. The idea is not to replace volunteers, but to close the blind spots of manual monitoring and obtain more accurate data on migration.

Why manual counting is limited

Every spring, river herring return from Massachusetts coastal waters to rivers and streams, where they spawn in fresh water. Their populations have fallen sharply over recent decades, so conservation agencies and fisheries managers need to know how many fish actually pass through the rivers and when. The problem is that classic monitoring usually relies on visual counts from the bank and on volunteer help, and such observations capture only short fragments of the overall picture.

The manual approach has hard limits. Volunteers mostly work during the day and cannot watch the fish flow continuously, so nighttime movements and short migration surges easily drop out of the data. Sometimes hundreds of fish pass through a stretch of river within a few minutes, and such peaks are hard to estimate by eye. More advanced methods such as acoustic monitoring and sonar do not help everywhere and cost more, while manually reviewing underwater video remains too labor-intensive. That is why the team bet on computer vision as a more scalable option.

How the model was built

The researchers built a full pipeline, from installing underwater cameras in the field to labeling clips and training the model. Video was collected on three Massachusetts rivers at once: the Coonamessett in Falmouth, the Ipswich in Ipswich and the Santuit in Mashpee. This design was meant not as a showcase but as a check on whether the system can work in real conditions, where lighting, water clarity, fish density, season and time of day constantly change.

• Underwater cameras recorded migration on three different rivers
• Training used clips with varied lighting, water quality, species and fish density
• The team manually labeled 1,435 video clips and 59,850 frames with bounding boxes
• The algorithm's results were checked against manual video review, bank counts and PIT tag data

The key finding turned out to be quite practical: the best-performing models were those trained on diverse data from several locations and several years. This approach produced detailed seasonal estimates that matched the results of traditional monitoring. In other words, this is not a lab prototype but a system capable of producing counts comparable to a human's, at a higher temporal resolution.

What the data showed

The most interesting part is not only the automated counting but also the new observations of fish behavior. In video of the 2024 migration in the Coonamessett River, the system counted 42,510 river herring. The analysis showed that upstream movement peaked at dawn, while downstream movement happened mainly at night. The researchers link this to fish using darker, calmer periods to reduce the risk of encountering predators. For ecologists, this is no longer just a number but a richer picture of migration.

"This work will improve fisheries monitoring and population assessment capabilities for managers and conservation groups," says

Robert Vincent of MIT Sea Grant.

The authors state plainly that automated counting should not displace traditional methods all at once. Long observation series matter in their own right, and agencies need to keep data comparable until the new systems are fully deployed. Citizen science, moreover, does not disappear here but changes role: volunteers are still needed to maintain cameras, label video and validate models. Combined with computer vision, this gives a more complete ecological monitoring system than either approach alone.

What this means

For AI, this is a good example of computer vision moving from demos into field tasks with measurable benefits. For ecology, it is a way to strengthen volunteer programs, get near-continuous observation and make conservation decisions based on denser, more accurate data. If such systems become widespread, monitoring of rare and migratory species could be done more often, more cheaply and with smaller losses in quality.