Species signatures and a sampling doctrine for fisheries AI
In brief
- Random frame splits reward vessel/camera shortcuts; hold out vessels, trips and time.
- Species recognition should be hierarchical and calibrated when sister species are visually similar.
- Sampling is a sequential decision problem: combine unbiased audit with uncertainty, rarity, shift and disagreement.
Fisheries imagery contains many shortcuts: deck colour, camera view, crew workflow, gear, season and where fish appear in the frame. A deep network can exploit those correlations and still look excellent on a random image split.
I use species signature as a practical target: visual evidence that should remain useful across vessels—shape, fin configuration, colour pattern, proportions and texture—while reducing reliance on vessel-specific background.
Representation learning
The first evaluation is not just Top-1 accuracy. I want to know what structures the embedding:
- nearest-neighbour retrieval across vessels;
- species vs vessel separability in latent space;
- calibrated confusion between related species;
- performance when the camera/vessel has never been seen;
- counterfactual sensitivity to background and crop.
Object-centric and metric-learning ideas are useful here because they push the representation toward the animal rather than the scene.
Hierarchy beats forced specificity
For visually similar taxa, the model should be allowed to stop at a broader label when evidence is insufficient. A hierarchy can convert one brittle classifier into a selective system:
fish → tuna-like → genus/family → species, with confidence thresholds at each level.
The evaluation then includes selective risk: how accurate is the system on the fraction of cases for which it chooses to make a fine-grained claim?
Sampling is a policy
Review capacity is limited, so deciding what a human should inspect next is a sequential decision problem. I propose to combine:
- random audit — preserves unbiased coverage;
- uncertainty — improves decision boundaries;
- rare/priority taxa — protects low-frequency classes;
- domain shift — new vessel/camera/season;
- disagreement — model/model or model/reviewer conflict.
This resembles active learning, but the long-term framing is closer to reinforcement learning: choose review actions that maximise future evidence quality under a review budget. The reward must include coverage and rare-event recall, not just immediate model improvement, or the policy will oversample easy-to-improve regions.
Evaluation
I propose to track:
- vessel-held-out species recall;
- macro-F1 / balanced accuracy;
- calibration and selective risk;
- rare-class recall;
- review minutes per useful new label;
- performance gain per 1,000 reviewed events;
- unbiased audit error after active sampling.
A random audit stream remains necessary because actively selected review data are policy-dependent and therefore cannot be assumed to represent fleet prevalence.
Impact
This approach connects deep representation learning with a practical governance question: how do we spend scarce human review where it improves both the model and our knowledge of its failures?