Merge duplicates

Duplicates are the #1 threat to data quality. Every duplicate erodes trust the moment you reach out — sending the same message twice to the same account is the fastest way to look like a spammer. The whole product is built around zero duplicates: prevent at the moment of creation, detect on demand, resolve by merge.

LeadHunter never auto-deletes a duplicate. Merge is always the operation, because deleting loses information you might still want.

Detection levels

LeadHunter compares a new account with the accounts already in the project using six signals. Each has a fixed confidence:

Level	Match	Confidence
1	Google Place ID — the same Google Maps place	100%
2	Imported ID — the same `imported_id` (the source system’s row id)	100%
3	Name + city — identical after normalisation	95%
4	Phone — the same digits	90%
5	Website domain — the same domain	85%
6	Similar name in the same city — ≥85% similarity	85–94%

How the comparison normalises:

Name ignores case, accents, punctuation, word order, legal suffixes (Inc, LLC, GmbH, SA, SL, BV, …) and filler words (the, and, shop, …), and treats & as and. So “Joe’s Pizza & Co.” and “Joe’s Pizza and Co LLC” are the same name. Single letters and digits are kept: Radio C and Radio X, or Radio 3 and Radio 5, in the same city are different accounts.
Phone compares digits only, so formatting doesn’t matter, and a national number matches the same number with its country code (931 234 567 = +34 931 234 567). Numbers shorter than 7 digits are never compared.
Website compares the domain only: https://www.acme.com/contact and acme.com match.

The imported ID is only compared when the incoming account carries one — in practice on file imports where you mapped that column, and on the upsert API, where it is the key.

What happens on a match

No path merges automatically. A match either stops the new account from being created or is left for you to resolve; the only way two accounts become one is a merge you run yourself, in Find duplicates (below) or through the merge API. What happens depends on how the account is being created:

How the account arrives	Place ID, imported ID or name + city (95–100%)	Phone, website or similar name (85–94%)
File import (CSV / XLSX)	The row is skipped: no account is created and its values are not copied into the existing account. It counts under Duplicates skipped, and a dry run lists it.	The row is imported as a new account, with no flag. Run Find duplicates afterwards to merge it.
New account form, or `POST /api/accounts/`	Not saved. LeadHunter points you to the existing account it matched, so you can open it instead; the API’s `force_create` creates a separate account when it really is a different one. (The imported ID isn’t checked here.)	Same: not saved, the existing account is shown.
Google Maps research	Only the Place ID is checked: a place already in the project is counted as a duplicate and not added again.	Not checked. A place whose business you already have without a Place ID (from a file, say) becomes a second account; Find duplicates catches it.
Tavily research	The result is dropped and counted as a duplicate.	The result is dropped and counted as a duplicate.
Upsert API (`upsert-by-imported-id`)	No duplicate check: the same imported ID updates that account, anything else creates one. A Google Place ID that another account already has is refused (400).	Not checked.

A file import also checks each row against the rows above it in the same file (since 2026-10-03), with the same rules: a shop listed twice with the same name and city is imported once; two rows that only share a phone number are both imported.

Because the 85–94% matches are created rather than skipped, run Find duplicates after a big import or research run.

Finding duplicates on demand

LeadHunter surfaces the same scanner from two places:

Accounts → More → Find duplicates — scans the whole project. Use this for periodic clean-up after large imports.
Campaigns → (campaign) → Review → Find duplicates (a button at the top of the campaign’s Review tab) — scans the same way but only returns groups that contain at least one account in this campaign. Use this when you’re reviewing scored accounts and notice two rows that look like the same business. Cross-campaign twins still show up in the same group so you see the full picture before merging.

Both work at any project size — a project with 50,000 accounts is scanned in a few seconds (the old “too many accounts” refusal above 10,000 is gone) — and show the groups 25 at a time, highest confidence first, with Previous / Next to page through. Two tabs:

Exact matches (default) — accounts that share a Google Place ID, the same name + city (after the normalisation above), the same phone (the last 9 digits) or the same website domain. A website on a shared host — a Facebook or Instagram page, a Linktree, a Google Maps link, a free site-builder address — never counts as a shared domain. The counts per kind sit next to the tabs.
Similar names — names that read alike (≥ 85% similar) in the same city, for the cases exact matching can’t see (Ciclos Martínez Hermanos and Ciclos Martinez Hermano). Identical names are left to the first tab. Only accounts with a city are compared, and only with accounts in the same city. In a city with more than 400 accounts, names are compared only with names that start with the same two letters, and a group still above 400 is compared in slices of 400, so in a very large city two similar names can land in different slices and not be paired. Exact matches have no such limit: that tab covers every account, however many. A group’s percentage is that of its least similar name, so Merge ≥90% on this page only takes groups where every name is at least 90% similar.

An account appears in one group only: if it shares a name + city with one account and a phone with another, it shows in the name + city group (the stronger match). Merge that one and scan again — the phone twin then shows up with the merged account. After every merge or delete the current page reloads, so the list always matches the data.

Each group shows:

Every account in the group with its name, location, phone, website, and key identifiers side-by-side.
The proposed survivor (the “golden record” — the row that will absorb the others).
A field-by-field diff so you can see exactly what gets merged where.

AI-assisted review for large lists

When Find duplicates returns dozens of groups, an AI review button runs the groups on the current page through an LLM pass that gives each a confidence and a one-line rationale. Use it to filter out obvious false positives before merging — the AI doesn’t decide, it just helps you skim faster.

Bulk merge

Merge ≥90% on this page merges every group on the current page whose confidence (or AI confidence, after an AI review) is 90% or more. It first opens a Bulk merge preview: one line per group with the account that survives (the oldest), the accounts it absorbs, and what the merged account will hold — city, phone, email, website, lifecycle stage. Nothing is written until you continue and type the confirmation. In a bulk merge the survivor keeps its own name (no AI name pick); to choose another name, survivor or field, cancel and merge that group on its own with Merge. A group too large to list in full (more than 50 accounts) is never bulk-merged.

What a merge actually does

When you confirm a merge (one group, or several with Bulk merge):

Every unique field from every duplicate is preserved on the survivor. If one duplicate has a phone and the other has a website, the survivor ends up with both.
Smart picks on conflicts when both rows have a value:
- Email — prefers a non-generic address (maria@acme.com wins over info@acme.com).
- Website — prefers the company’s own domain over an aggregator URL (e.g. acme.com over a Yelp listing).
- Notes — concatenated, de-duplicated, with attribution.
- Latitude + longitude — picked as a pair from the same source row (never half from one, half from the other).
- Website content — the longer, fresher scrape wins.
- Numeric / rating fields — the higher value wins.
- Custom fields — deep-merged dict; multi-select arrays union.
The survivor’s merge_history JSONB grows an entry recording: which accounts were absorbed, their full snapshot (so you can read what was on each row before the merge), who ran the merge, when, and which fields were overridden.
Everything pointing at the absorbed rows is re-pointed at the survivor: campaign-account rows, conversation messages, research records, scores, and Client Pulse usage history (events and daily counts are combined; first-seen is the earliest, last-seen the latest). No history is lost.
- Campaign decisions keep their history. When both accounts are in the same campaign and only the absorbed one was decided (Shortlist, Not a fit or Defer), the survivor takes that decision together with its history. A bulk decision still counts in its batch, and Undo on that batch undoes it on the merged account. When the survivor was already decided, its own decision stays and the absorbed one is dropped; undoing the absorbed account’s batch never changes the survivor. The dropped decision is still listed in the merge record.
- Client contact settings survive too: the survivor keeps the most recent last contacted date and the latest snooze, and its own contact cadence (or the duplicate’s, if the survivor had none). A merged account is archived only if every merged account was archived — one active duplicate keeps the result active.
Status escalates, never demotes. The internal order is prospect < contacted < in_negotiation < lost < in_trial < customer < do_not_contact. So:
- do_not_contact always wins (GDPR / CAN-SPAM stickiness — an opt-out survives every merge).
- customer beats lost — a recorded customer relationship outranks a never-sold lost deal even if the customer is older.
- The audit trail records the previous status of every absorbed row.
- The status can’t be picked in the merge dialog (since 7 October). If the merged account needs another status, change it afterwards on the account page (the Status chip in its header), so the change is recorded with its reason.
Acquisition channel sticks to the survivor’s value by default. Use the field-override controls on the merge dialog if the absorbed row’s channel was actually correct (e.g. you imported the outbound-discovered row first, but the inbound Adwords row was the real origin).
AI picks the canonical name when the duplicates have different names — “Joe’s Pizza” over “Joe’s Pizza Restaurant LLC Holdings 2014”. You can override it manually in the merge dialog before confirming.
Absorbed rows are deleted at the end. Their data lives on in the survivor’s fields and in merge_history, but the Account rows themselves are gone.

Preview before you commit

Every merge in the UI runs through a preview step first — same logic as the real merge, but no writes. The preview shows you the field-winner picks (with the source row that won), the LLM-selected name, the merge_history entry that would be written, and any overrides you’ve set. Confirm to commit, or back out to adjust.

For programmatic merges, hit the preview endpoint before the merge endpoint — same shape. The bulk merge has its own preview (a dry run of the same request), which is what the Bulk merge preview shows.

Why we merge instead of delete

Deleting loses information. Even a “clearly bad” duplicate often has one field the survivor doesn’t — a phone number, a typo correction in the address, a different decision-maker contact, a different acquisition channel. Merging keeps every signal and the audit trail.

The cost of merging conservatively (keep everything) is one bigger row. The cost of deleting aggressively is permanent data loss. We optimise for the first.

Merges aren’t reversible — but the history is preserved

When you confirm a merge, the absorbed accounts are deleted. There’s no “unmerge” button — the survivor row is now the truth and re-creating a clean split is hard to do automatically (which campaign-account row belonged to which side? which conversations?). The merge_history JSONB records the absorbed rows’ full snapshots so you can always read what was on each side, but restoring them is a manual rebuild.

In practice this rarely bites — most merges are obvious by the time they happen — but it’s worth knowing before you bulk-merge 50 fuzzy candidates without spot-checking.

When in doubt

The trade-off:

Merging two accounts that turn out to be different organisations: rare, but messy when it happens. The survivor row carries fields from both, conversations mix, the merge_history is the record. Manual rebuild required.
Not merging two accounts that are actually the same: common, low cost initially. Eventually, you reach out to the same place twice and look like a spammer.

Bias toward merging when the AI’s confidence is high and you’ve spot-checked the fields. Bias toward leaving them when names look similar but the addresses are clearly different (two unrelated Acme Bikes in two countries).

Common pitfalls

Bulk-merging without spot-checking the AI review. AI review is a triage helper, not a decision. Skim the uncertain group at minimum before clicking bulk-merge.
Expecting an import to have merged everything. An import only skips rows that match on Place ID, imported ID or name + city — and it skips them without copying their values over. Rows that only share a phone, a website or a similar name are imported as separate accounts, so Find duplicates after an import usually finds phone and website twins. A skipped row whose extra details you wanted (a phone the existing account lacks) has to be added to the existing account by hand.
Manually merging accounts the system flagged as different. The AI is wrong sometimes, but when it confidently says two accounts are different, it’s worth re-reading the field diff before overriding.
Treating merge_history as a backup. It’s an audit trail. Restoring an absorbed row from the snapshot is a manual exercise — possible, not push-button.