The Hidden Cost of Farmer Identity Collisions in Nigerian Agricultural Programmes

Key Findings (Executive Summary)

• The standard SHA-256 hash-based farmer identifier produces a 10.12% collision rate on real programme data. Adding phone number and gender to the hash reduces it to 6.44%, but collisions remain — this is a structural naming problem, not a data entry problem.
• The estimated financial exposure from uncorrected identity collisions across the study datasets is NGN 246 billion (approximately USD 1.54 million).
• A three-model machine learning pipeline (Logistic Regression, Artificial Neural Network, and Multivariate Adaptive Regression Splines in a weighted ensemble) identifies 1,739 high-confidence duplicate pairs from a validation pool of 106,583 candidate pairs, with a mean confidence of 87.9%.
• GPS coordinates fail as a deduplication signal because of GPS fixation: enumerators record the same coordinate for every farmer in a survey session. Name similarity is a more reliable standalone signal for detecting spelling-variant duplicates.
• The machine learning pipeline yields an estimated 2,871-fold return on investment relative to its implementation cost.