MIT demonstrates computer vision system for counting fish in citizen science projects

MIT Sea Grant along with Woodwell Climate Research Center, CSAIL, and other partners demonstrated a deep learning-based system that counts fish from underwater video. The idea is not to replace volunteers, but to fill blind spots in manual monitoring and get more accurate data on migration.

Why Manual Counting Is Limited

Every spring, river herring return from the coastal waters of Massachusetts to rivers and streams, where they spawn in fresh water. Over recent decades, their populations have declined sharply, so it's important for conservation agencies and fisheries managers to understand how many fish actually pass through the rivers and when this happens. The problem is that classical monitoring typically relies on visual counts from the shore and volunteer help, and such observations only provide short fragments of the full picture.

Manual approaches have strict limitations. Volunteers usually work during the day and cannot continuously monitor fish flow, so nighttime movements and brief migration surges easily drop out of the data. Sometimes hundreds of individuals pass through a river section in minutes, and such peaks are hard to assess by eye. More advanced methods like acoustic monitoring and sonar don't work everywhere and cost more, while manual review of underwater video remains too labor-intensive. This is why the team bet on computer vision as a more scalable option.

How They Built the Model

Researchers built a complete pipeline: from installing underwater cameras in the field to labeling videos and training the model. Video was collected across three Massachusetts rivers — Coonamessett in Falmouth, Ipswich in Ipswich, and Santuit in Mashpee. This design was needed not for a nice demo, but to test whether the system could work in real conditions, where lighting, water clarity, fish density, season, and time of day constantly change.

• Underwater cameras captured migration across three different rivers
• Training used footage with varying light, water quality, fish species, and density
• The team manually labeled 1,435 video clips and 59,850 frames with bounding boxes
• Algorithm results were cross-checked against manual video review, shore counts, and PIT-tag data

A key finding turned out to be quite practical: models trained on diverse data from multiple locations over several years worked best. This approach delivered detailed seasonal estimates that matched traditional monitoring results. In other words, this is not a laboratory prototype but a system capable of delivering human-comparable counts while doing so at higher temporal resolution.

What the Data Showed

The most interesting part is not just automating the count, but new observations about fish behavior. On 2024 migration video from the Coonamessett River, the system counted 42,510 river herring. Analysis showed that upstream movement peaked at dawn, while downstream movement occurred mostly at night. Researchers attribute this to fish using darker and calmer periods to reduce the risk of predator encounters. For ecologists, this is no longer just a number, but a richer picture of migration.

"This work will improve fisheries monitoring capabilities and population assessments for managers and conservation groups," says

Robert Vincent from MIT Sea Grant.

At the same time, the authors directly state that automatic counting should not immediately displace traditional methods. Long observation records are important in themselves, and agencies need to maintain data comparability while new systems are being fully deployed. Moreover, citizen science doesn't disappear here but changes its role: volunteers are still needed for camera maintenance, video labeling, and model validation. Combined with computer vision, this creates a more complete ecological monitoring system than either approach alone.

What This Means

For AI, this is a good example of computer vision moving from demos to field tasks with measurable benefit. For ecology, it's a way to strengthen volunteer programs, get near-continuous observation, and make conservation decisions based on denser and more accurate data. If such systems become widespread, monitoring of rare and migratory species could be done more frequently, more cheaply, and with fewer quality losses.