Module 3 · 3.5

English

Nederlands English Deutsch

Canonical and duplicate content

Module 3: The technical basics of a page · Reading time: 12 minutes

Learning objective: After this lesson you can spot duplicate content on your site, choose which address is the real (canonical) one, and pass that on to search engines properly with redirects and rel="canonical".

Key concepts: duplicate content, canonicalisation, canonical address, variants (http and https, with and without www, slash, parameters, capital letters), rel="canonical", absolute address, canonical as a hint, 301 redirect as a stronger signal, Google-selected canonical, no penalty for duplicate content

One page, five addresses

To you, the bridal bouquets page is one page. To a search engine, every address is a separate page. Look at how the same content at Bloemenatelier Lotte can be reached at five addresses:

http://example.com/bruidsboeketten
https://example.com/bruidsboeketten
https://www.example.com/bruidsboeketten
https://www.example.com/bruidsboeketten/
https://www.example.com/bruidsboeketten?utm_source=nieuwsbrief

This is called duplicate content: the same (or almost the same) content at more than one address. Google lists the most common causes:

Cause	Example
Protocol variants	`http://` and `https://` both work
Variants with and without `www`	the domain and `www.` show the same site
Site features	sorting and filtering: `?sorteer=prijs`, `?kleur=rood`
Regional variants	the same Dutch text for `/nl-nl/` and `/nl-be/`
Device variants	a separate mobile site, such as `m.example.com`
By accident	a test copy of the site (`test.example.com`) that is open to search engines

On top of that: a slash at the end or not, capital letters in the path (/Contact and /contact), index.html after the address, and parameters from newsletters and adverts.

Is that a problem?

No penalty. Google writes that some duplicate content on a site is normal and does not break its spam policies. You won't be penalised for it. Things only go wrong when someone deliberately creates copies on a large scale to fill the search results; that is a different story (lesson 8.2).

So why sort it out? Because otherwise the search engine chooses for you. Google groups pages with the same content and picks one canonical address from them: the representative that appears in the search results and is crawled most often. That choice is called canonicalisation. Leave it to Google and it may pick exactly the address with ?utm_source=nieuwsbrief on the end. On top of that, your figures in Search Console get spread across five variants, and the crawler wastes time on copies.

The three signals, from strong to weak

You tell the search engine which address is the real one. Google names three ways, in order of strength:

Redirects (301 or 308): a strong signal that the target of the redirect should become canonical. Use this for variants nobody needs: http to https, and the version with or without www to the other one. You set this up once for the whole site (lesson 4.1).
rel="canonical": also a strong signal. A line in the <head> of the page that says which address is the real one. Use this where the variant needs to keep working, such as an address with a filter or a newsletter parameter.
The sitemap: a weak signal. Addresses you put in your sitemap are put forward as canonical (lesson 3.4). So only include the real addresses.

A canonical looks like this:

<head>
  <title>Bruidsboeketten op maat | Bloemenatelier Lotte</title>
  <link rel="canonical" href="https://www.example.com/bruidsboeketten">
</head>

Every variant of the page carries the same line, including the canonical address itself. That last one is called a reference to itself (self-referencing canonical). It isn't required, but it is useful: if someone arrives via ?utm_source=nieuwsbrief, the real address is right there.

A hint, not an order

Google is clear about it: a canonical is a hint, not a rule. Google can choose a different address from you. This mainly happens when your signals contradict each other. In URL inspection in Search Console (lesson 2.4) you see two lines side by side: "User-declared canonical" and "Google-selected canonical". If they differ, look for a contradiction:

the canonical points to https://www.example.com/..., but the sitemap lists https://example.com/...;
the canonical points to https://, but https:// redirects to http://;
the internal links in your menu point to a different variant from the canonical;
the pages are not similar enough: a canonical from a product page to the homepage doesn't work, because that is not a copy.

With identical content, Google prefers the https version as the canonical address, unless something is wrong with it, such as an invalid certificate.

How to write a canonical properly

Rule	Why
Full address with `https://` and the host name	Google recommends absolute paths; with a relative path a small mistake can point to the wrong address
Exactly one canonical per page	Two different canonicals contradict each other; Google may then ignore both
Only in the `<head>`	Google only accepts the link in the `<head>` of the HTML
Points to a page that returns 200	A canonical to an address that redirects or returns 404 is a contradictory signal
The target has no noindex	"This is the real address" and "don't include the real address" don't go together
The target points to itself	A chain (A points to B, B to C) makes it unclear
Choose one method per page	The HTML line or the HTTP header `Link`; using both at once is more error-prone according to Google

For files without HTML, such as a PDF, a canonical can also be sent as an HTTP header:

Link: <https://www.example.com/prijslijst.pdf>; rel="canonical"

What not to do

No robots.txt for canonicalisation. Google says explicitly: don't use robots.txt for this. If you block the variants, Google can't read them and so won't see your canonical either.
No noindex to force a canonical. Google advises against this within a single site too. Noindex says "don't include this at all", not "prefer that other one".
Not everything to the homepage. A canonical on every page pointing to the homepage (a well-known mistake in themes and plugins) tells Google that your whole site is one page.

In practice: who sets up what?

You set up the site-wide redirects with your hosting. In DirectAdmin at WebOké, the settings for a domain include the options for SSL, https redirection and "with or without www"; other providers call it something else, but it is the same kind of setting. The canonical in the page is created by your website builder. WordPress adds a canonical to posts and pages itself, and SEO plugins take that over. WebOké's AI Builder adds a canonical to every published page pointing to that page's own address, and deliberately none on the error page (404). After every change of domain or www choice, check that canonical, redirect and sitemap still name the same address.

Summary

Duplicate content is normal and not a reason for a penalty, but without a signal the search engine decides for itself which address is canonical.
Three signals: redirects (strong), rel="canonical" (strong) and the sitemap (weak). Make them all name the same address.
A canonical is a hint: Google can choose differently, especially with contradictory signals. Check URL inspection.
Write it in full, once, in the <head>, pointing to a page that returns 200, has no noindex and points to itself.
Don't use robots.txt or noindex to force a canonical.

Quiz

Exercise

Exercise A3.5: Does your canonical point to the right place? Enter the address of a page on your site. The program reads the page, finds the canonical and checks whether there is exactly one, whether it is complete and https, whether the target returns 200, has no noindex and points to itself, and whether the variants with http and with or without www redirect to the same host. No site of your own? Use https://european-union.europa.eu/index_nl.

Cookies at WebOké

Cookie settings

Necessary

Marketing