eduKate Singapore

How Classification by Similarity Works | Prototypes, Distance, Clusters and Nearest Neighbours

Similarity-based classification begins with a practical idea: if a new thing looks sufficiently like known members of a category, that resemblance can become evidence for membership.

Humans do this constantly. A child recognises a new dog without knowing a formal definition of dog. A radiologist compares an image with familiar patterns. A librarian encounters a new book and notices that it resembles a known subject cluster. An AI system embeds a new document into a vector space and checks which labelled examples sit nearby.

Similarity is powerful because many useful categories have no single perfect rule. But resemblance is also dangerous: two things can be close in one representation and importantly different in another.

Quick answer: how does similarity-based classification work?

  1. Represent each item using relevant features.
  2. Define what “close” or “similar” means.
  3. Compare the new item with prototypes, clusters or labelled examples.
  4. Score candidate categories.
  5. Apply a threshold or ranking rule.
  6. Return one or more categories, or mark the item uncertain or novel.
  7. Test whether the similarity measure preserves the distinctions the task actually needs.

This method sits beside the broader framework in How to Categorise Anything. It is especially useful when categories behave like clusters rather than perfectly defined boxes.

1. Similarity depends on representation

Nothing is simply similar in the abstract. Two objects are similar with respect to selected features.

Two cars may be similar in colour and different in engine type. Two documents may be similar in vocabulary and different in legal status. Two students may have similar scores and very different misconceptions.

2. Feature choice creates the comparison space

A feature is any characteristic used to compare items: size, shape, words, measurements, behaviours, metadata, frequencies or learned numerical representations.

The chosen features determine which kinds of resemblance the system can see.

3. Irrelevant features add noise

If the task is to classify plant species, the colour of the pot may be irrelevant. If the task is to classify archival access risk, the document’s subject may matter less than rights and confidentiality.

Similarity systems need features aligned with the classification purpose.

4. Feature scaling matters

If one numerical feature ranges from 0 to 1 and another from 0 to 1,000, the larger scale can dominate naive distance calculations.

Normalisation or appropriate distance design may be needed so scale does not silently decide importance.

5. Distance is a formalised notion of difference

Similarity-based systems often convert resemblance into distance: smaller distance means greater similarity.

Different distance measures emphasise different geometry.

6. Euclidean distance is intuitive but not universal

Euclidean distance treats the feature space like ordinary geometry.

It is useful for many numeric problems but can be inappropriate when features are categorical, sparse or differently scaled.

7. Cosine similarity compares direction

For text and embeddings, cosine similarity is often used to compare orientation rather than raw magnitude.

Two documents can therefore be treated as semantically similar even if one is much longer.

8. Categorical similarity needs different rules

Eye colour, document type, jurisdiction and status do not behave like continuous coordinates.

Use matching, weighted overlap or domain-specific similarity rather than forcing every feature into numeric distance carelessly.

9. Prototypes represent category centres

A prototype is a representative example or abstract centre of a category.

New items can be classified according to which prototype they most resemble.

10. Prototypes explain typicality

Some members sit near the centre of a category and feel very typical. Others sit near the edge.

This connects to the prototype discussion in How Categories Work.

11. One prototype may be too simple

A category may contain several distinct subtypes.

One average prototype can fall into a region where no real member exists. Multiple prototypes or cluster-based models may represent the category better.

12. Nearest-neighbour classification uses actual examples

Instead of comparing with an abstract prototype, a nearest-neighbour method compares the new item with stored labelled examples.

The labels of the closest examples provide evidence for the new item’s category.

13. One neighbour can be fragile

A single unusual or wrongly labelled example can mislead a one-neighbour classifier.

Using several neighbours can make classification more robust.

14. Majority vote can hide local minorities

If one category is much more common, nearby majority examples can overwhelm a small but legitimate cluster.

Class balance and local density matter.

15. Weighted neighbours can reward closeness

Closer neighbours can be given more influence than distant ones.

This makes the local geometry of the space more important than raw vote count.

16. Clustering finds structure without labels

Clustering groups similar items before category names are necessarily assigned.

This can reveal candidate categories, subcategories or anomalies hidden inside a dataset.

17. A cluster is not automatically a meaningful category

Algorithms can find mathematical clusters that have no operational or conceptual value.

Human interpretation must ask what the cluster means and whether the distinction matters.

18. Cluster count changes the story

A dataset can often be partitioned into two, five or twenty clusters depending on algorithm and parameters.

Granularity remains a design choice.

19. Dense regions and sparse regions behave differently

A point inside a dense cluster may be easy to classify. A point in a sparse region may be uncertain even if one category is technically nearest.

Distance alone should not always imply confidence.

20. Boundaries emerge between clusters

Similarity classification creates decision regions between category centres or examples.

Items near those boundaries deserve explicit uncertainty handling.

21. Embeddings create learned similarity spaces

Modern AI systems can transform text, images and other inputs into learned numerical vectors.

Semantically similar items often become neighbours even when they do not share exact surface features.

22. Embedding similarity is powerful for retrieval

A query about “Add Maths” can retrieve content about “Additional Mathematics” even if the wording differs.

Similarity helps bridge language variation.

23. Similarity is not authoritative identity

Two documents can be semantically similar and belong to different legal jurisdictions or publication years.

Use controlled vocabulary, facets or ontology constraints when exact identity matters.

24. Similarity thresholds determine acceptance

A system may classify only if the best similarity score exceeds a minimum threshold.

Below the threshold, the item can be marked unknown or novel instead of forced into the nearest category.

25. Relative margin matters too

A top score of 0.82 may look strong, but if the second-best category scores 0.81, the classification is less decisive than a case scoring 0.82 versus 0.40.

Compare the leading candidate with alternatives.

26. Similarity can support multi-label classification

An item may be close to several category prototypes and legitimately receive several labels.

See How Multi-Label Classification Works.

27. Similarity can expose novelty

If an item is far from every known category, that distance can become evidence that the taxonomy may not contain an adequate home.

Novelty detection is therefore the inverse of similarity classification.

28. Similarity can encode bias

If the feature space reflects biased samples or proxies, “nearest” can reproduce those distortions.

Test subgroup and category-level performance using How Classification Bias Works.

29. Typicality can privilege majority examples

Minority but valid forms may sit farther from the dominant prototype.

Use diverse prototypes and representative examples so category centres do not become cultural or sampling stereotypes.

30. Similarity spaces drift

New technologies, language and behaviours can change what typical examples look like.

Monitor recent data and refresh reference sets when necessary.

31. Labels can drift even if vectors remain stable

A category definition may change while the underlying representation model remains unchanged.

Version reference examples and label definitions with the taxonomy.

32. Evaluate nearest-neighbour confusion

Inspect which categories repeatedly become each other’s nearest alternatives.

Persistent confusion may indicate overlapping boundaries or weak features.

33. Visualisation can reveal geometry

Dimensionality-reduction plots can help humans inspect clusters and outliers.

They are diagnostic aids, not proof that high-dimensional categories truly separate cleanly.

34. Gold test sets remain essential

Similarity systems should be evaluated against reviewed examples, including boundary and novel cases.

Measure precision, recall, confusion and rejection behaviour.

35. Rejection quality matters

A classifier that safely says “none of the known categories fit” can outperform one that always chooses the closest label.

Unknown detection is part of classification quality.

36. Similarity can complement rule-based classification

Rules can enforce hard constraints while similarity ranks plausible candidates inside those constraints.

Hybrid systems often combine the strengths of both.

37. Similarity can complement ontology

Ontology supplies explicit structure. Similarity helps discover candidate entities and concepts when wording varies.

Semantic closeness can propose; ontology can constrain and validate.

38. A practical similarity-classification protocol

  1. Define classification purpose.
  2. Select relevant features or representation.
  3. Choose similarity or distance measure.
  4. Collect representative labelled examples.
  5. Choose prototype, neighbour or cluster method.
  6. Set thresholds and rejection rules.
  7. Test boundary cases.
  8. Test minority and rare forms.
  9. Measure confusion and calibration.
  10. Monitor drift and novelty.

39. When similarity is the wrong tool

If membership is defined by a precise legal, mathematical or physical rule, similarity may be unnecessary or unsafe. Use rule-based criteria when exact conditions determine membership.

40. The deeper idea

Similarity classification works by turning experience into geometry.

To say that two things are similar is already to say which differences we have decided to ignore.

Final answer

Use similarity-based classification when categories are best represented by patterns, prototypes or neighbourhoods rather than strict rules. Choose features carefully, define distance appropriately, use thresholds and margins, preserve uncertainty, test bias and novelty, and version the reference space as the domain changes.

Continue through the series

Explore the connected learning guides

Choose the question that brought you here. Open one useful guide, try a small task, and stop when you have what you need.

Take one question further

The same learning habit can travel across subjects, while each subject keeps its own methods. These routes help you notice a difficulty, understand one part of it, and return to something you can do.

A word is familiar, but using it is difficult.

Move from recognising a word to retrieving it in a new context. Understand vocabulary plateaus.

Try it without the guide: Choose one word you already know. Close the guide and use it in a new sentence. Explain why it fits; try another context tomorrow.

A piece of writing has ideas, but the reader loses the thread.

Make the order of events and the links between sentences clear. Explore composition writing.

Try it without the guide: Choose one short paragraph. Read the relevant explanation, close it, and revise the paragraph. Ask someone to tell you what happened and why.

The Mathematics seems familiar, but marks still disappear.

Find the first point where the working stops being reliable. Find Secondary 4 A-Math mark leakage.

Try it without the guide: For a Secondary 4 A-Math question you have attempted, locate the first uncertain line. Repair that step, then try a comparable question without the worked answer.

A Science fact is remembered, but the explanation is incomplete.

Connect the evidence to a scientific idea and the resulting change. Follow the Primary Science learning route.

Try it without the guide: Choose a familiar Primary Science example. Explain the evidence, the idea and the result without notes. Then change one condition and explain your prediction.

Two accounts of the world seem to disagree.

Check the question, source, date and evidence before combining claims. Explore the World Knowledge research library.

Try it without the guide: Take one claim. Find the source best placed to support it, note its date, and state what remains uncertain. Return to your original question.

There is plenty of help, but independence is hard to see.

Check what the learner can understand and do after support is removed. Understand how education works.

Try it without the guide: Choose one small task the child has practised. Agree on a calm, brief attempt without prompts. Use what happens to choose one next step, then stop.

For the structure behind these connections, read the eduKateSingapore runtime manifest and the eduKate ecosystem boot contract. The reader map describes public navigation; those manifests preserve the wider ownership and return rules.