BOM & Data Foundation

Finding duplicate parts in the part master: matching rules that actually work

Part masters that grew over many years carry duplicates no description search will find. Matching keys, a scoring model, survivorship rules and how to merge without breaking BOMs.

Not ready to talk yet? Take the 3-minute Data Foundation Check

SAME BEARING, THREE RECORDS10023Bearing 6204100%10023-ABEARING 6204 2RS96%BRG-6204Rillenkugellager 620491%Golden record10023Bearing 6204-2RS1 price, 1 supplierMatch score on description, attributes and supplier

TL;DR

  • Exact duplicates are easy. The expensive ones are near-duplicates: same part, different description, maker or number.
  • Match on normalised attributes and manufacturer part numbers, not on raw text.
  • Score candidate pairs, auto-merge only the obvious ones, and send the grey zone to someone who knows the parts.
  • Merging is a process, not a delete: pick a survivor, redirect BOMs, stock and open orders, and keep the mapping.
For:Master data managementEngineering standardsPurchasingSpare parts

What types of duplicates are there?

TypeExampleHow to find it
ExactSame description and maker part no., two material numbersGroup by normalised key
Near-duplicate“Hex screw M8x20 8.8 zinc” vs. “SCREW ISO4017 M8X20 8.8 ZN”Attribute match after parsing
Cross-supplierSame bearing bought from two distributors under two numbersManufacturer part number, attributes
Functional equivalentDifferent makers, same form, fit and functionAttributes + engineering approval
Drawing copyDrawing copied for a new machine, part unchangedDrawing comparison, where-used

Why duplicates cost money

Every duplicate means separate stock, separate safety stock, smaller order quantities, and a service desk that can’t see the part is already on the shelf under another number.

How does matching work?

DUPLICATE DETECTION PIPELINENormaliseCase, units,abbreviations, stopwordsBlockGroup by class andkey dimension tocut pairsCompareAttributes, makerpart no., textsimilarityScoreWeighted 0 to 100per pairDecideAuto-merge, reviewor keep
Match score (0–100) =
  40 × same manufacturer part number (normalised)
+ 30 × share of key attributes that match
+ 20 × description similarity after normalisation
+ 10 × same class

≥ 90    auto-merge candidate
70–89   human review
< 70    keep separate

The weights are a starting point. Calibrate them on a few hundred pairs that someone from engineering has labelled by hand. Blocking matters for speed: comparing every part with every other part doesn’t scale beyond a few thousand records.

What does a reviewer need to see?

Review is where most deduplication projects stall. Give the reviewer one screen with everything needed to decide in under a minute.

  • Both records side by side, differing attributes highlighted
  • Stock, open orders and 24 months of consumption per number
  • Where-used: BOMs and machine types each number appears in
  • Drawing or image, if available
  • A proposed survivor and the reason for it

How do you merge without breaking things?

  1. Pick the survivor. Usually the number with the most where-used and the cleanest data, not simply the oldest.
  2. Block the others for new use, but keep them readable.
  3. Redirect BOMs to the survivor with a change number so the history stays traceable.
  4. Move stock and open orders, or let them run out if moving is expensive.
  5. Keep a mapping table old → new. Service, catalogues and customers will quote old numbers for years.

Typical survivorship rules

FieldRule
Material numberMost where-used, otherwise oldest active
DescriptionRebuilt from attributes
Manufacturer part no.Keep all, flag the preferred one
Info records / pricesKeep the best active one per supplier
Spare part flagSet if any merged record was a spare part

Checklist

  • A labelled sample of pairs to calibrate the score
  • Manufacturer part numbers captured for purchased parts
  • Where-used and stock available per material
  • A named approver from engineering
  • A change number and mapping entry for every merge

FAQ

How many duplicates does a typical part master have?

It varies with age and history. Part masters that grew over many years, through acquisitions or several ERP systems, often show a noticeable share of duplicates among standard and purchased parts. A profiling run on your own data gives the real number.

Can duplicates be found automatically?

Exact and most near-duplicates, yes, with normalisation and attribute matching. Functional equivalents need someone from engineering to confirm form, fit and function.

Should we delete duplicate materials?

No. Block them for new use, redirect BOMs and stock to the survivor, and keep the old numbers readable through a mapping. Deleting breaks history and service lookups.

What is a survivor record?

The material number that stays active after a merge. It is chosen by rules, typically the record with the most where-used and the best data quality.

Does classification help to find duplicates?

Yes. Classification with attributes is the best input for matching. Without it, matching relies on text similarity, which produces many false positives.

How many duplicates are in your part master?

Send us a part master extract under NDA. We’ll return a duplicate report with scores and proposed survivors.

Similar Posts