Finding duplicate parts in the part master: matching rules that actually work
Part masters that grew over many years carry duplicates no description search will find. Matching keys, a scoring model, survivorship rules and how to merge without breaking BOMs.
Not ready to talk yet? Take the 3-minute Data Foundation Check
TL;DR
- Exact duplicates are easy. The expensive ones are near-duplicates: same part, different description, maker or number.
- Match on normalised attributes and manufacturer part numbers, not on raw text.
- Score candidate pairs, auto-merge only the obvious ones, and send the grey zone to someone who knows the parts.
- Merging is a process, not a delete: pick a survivor, redirect BOMs, stock and open orders, and keep the mapping.
What types of duplicates are there?
| Type | Example | How to find it |
|---|---|---|
| Exact | Same description and maker part no., two material numbers | Group by normalised key |
| Near-duplicate | “Hex screw M8x20 8.8 zinc” vs. “SCREW ISO4017 M8X20 8.8 ZN” | Attribute match after parsing |
| Cross-supplier | Same bearing bought from two distributors under two numbers | Manufacturer part number, attributes |
| Functional equivalent | Different makers, same form, fit and function | Attributes + engineering approval |
| Drawing copy | Drawing copied for a new machine, part unchanged | Drawing comparison, where-used |
Why duplicates cost money
Every duplicate means separate stock, separate safety stock, smaller order quantities, and a service desk that can’t see the part is already on the shelf under another number.
How does matching work?
Match score (0–100) = 40 × same manufacturer part number (normalised) + 30 × share of key attributes that match + 20 × description similarity after normalisation + 10 × same class ≥ 90 auto-merge candidate 70–89 human review < 70 keep separate
The weights are a starting point. Calibrate them on a few hundred pairs that someone from engineering has labelled by hand. Blocking matters for speed: comparing every part with every other part doesn’t scale beyond a few thousand records.
What does a reviewer need to see?
Review is where most deduplication projects stall. Give the reviewer one screen with everything needed to decide in under a minute.
- Both records side by side, differing attributes highlighted
- Stock, open orders and 24 months of consumption per number
- Where-used: BOMs and machine types each number appears in
- Drawing or image, if available
- A proposed survivor and the reason for it
How do you merge without breaking things?
- Pick the survivor. Usually the number with the most where-used and the cleanest data, not simply the oldest.
- Block the others for new use, but keep them readable.
- Redirect BOMs to the survivor with a change number so the history stays traceable.
- Move stock and open orders, or let them run out if moving is expensive.
- Keep a mapping table old → new. Service, catalogues and customers will quote old numbers for years.
Typical survivorship rules
| Field | Rule |
|---|---|
| Material number | Most where-used, otherwise oldest active |
| Description | Rebuilt from attributes |
| Manufacturer part no. | Keep all, flag the preferred one |
| Info records / prices | Keep the best active one per supplier |
| Spare part flag | Set if any merged record was a spare part |
Checklist
- A labelled sample of pairs to calibrate the score
- Manufacturer part numbers captured for purchased parts
- Where-used and stock available per material
- A named approver from engineering
- A change number and mapping entry for every merge
FAQ
How many duplicates does a typical part master have?
It varies with age and history. Part masters that grew over many years, through acquisitions or several ERP systems, often show a noticeable share of duplicates among standard and purchased parts. A profiling run on your own data gives the real number.
Can duplicates be found automatically?
Exact and most near-duplicates, yes, with normalisation and attribute matching. Functional equivalents need someone from engineering to confirm form, fit and function.
Should we delete duplicate materials?
No. Block them for new use, redirect BOMs and stock to the survivor, and keep the old numbers readable through a mapping. Deleting breaks history and service lookups.
What is a survivor record?
The material number that stays active after a merge. It is chosen by rules, typically the record with the most where-used and the best data quality.
Does classification help to find duplicates?
Yes. Classification with attributes is the best input for matching. Without it, matching relies on text similarity, which produces many false positives.
How many duplicates are in your part master?
Send us a part master extract under NDA. We’ll return a duplicate report with scores and proposed survivors.





