CRM Deduplication Engine for a Sales-Led Growth Company
35% of accounts in Salesforce were duplicates. Reps were calling the same companies, territories were miscounted, and forecast accuracy was below 55%. A custom matching and merge pipeline cut duplicates to under 3% in six weeks.
Cost of problem
$200k/year in duplicated prospecting effort
Savings achieved
$200k/year in duplicated prospecting effort eliminated
Time saved
40 hours/month of manual data cleanup
ROI
8x ROI in year one
Build time
6 weeks
The Problem
After two acquisitions, the Salesforce org had accumulated 35,000 accounts with an estimated 35% duplication rate. Sales reps were unknowingly prospecting into accounts already owned by colleagues, territory carve-outs were based on inflated account counts, and the CRM was treated as unreliable by leadership. Forecast calls were supplemented with spreadsheet overrides because 'Salesforce just can't be trusted.'
What Was Built
A Python deduplication engine using fuzzy name matching (RapidFuzz), domain normalization, and phone number clustering to identify likely duplicates. A confidence scoring system surfaces high-confidence merges for auto-processing and low-confidence matches for human review via a Streamlit queue. Salesforce updates are applied via the Bulk API to avoid hitting governor limits.
How It Was Implemented
The pipeline runs in three phases: (1) candidate generation using blocking keys (domain, zip+name prefix) to avoid O(n²) comparisons, (2) pairwise scoring across 8 signals weighted by field reliability, (3) cluster resolution using union-find to handle transitive duplicates. All merges are logged to a separate object in Salesforce for auditability.
The Outcome
Duplicate rate dropped from 35% to under 3% within 6 weeks. Territory plan was recalculated on clean data, shifting headcount allocation across 4 regions. Forecast accuracy improved from 55% to 74% within two quarters as pipeline visibility improved. CRM trust scores in the quarterly RevOps survey went from 2.1/5 to 4.3/5.