Web Page Change Detection: How It Works, Step by Step

Detecting that a web page changed sounds trivial: download it twice and compare. Try it and you'll find that almost every page "changes" on every load. Session tokens, nonces, ad slots, timestamps, A/B tests and lazy-loaded images all differ between two requests made a second apart, even when a human would say nothing changed at all.

Web page change detection is the work of separating those meaningless differences from the ones that matter. This guide walks through how it's done, step by step, with the trade-offs at each stage. It's the same pipeline MyKavo runs, and the same ideas apply if you're building a script of your own.

What is web page change detection?#

Web page change detection is the automated process of checking a page on a schedule, comparing it with a saved reference version and reporting meaningful differences. Those differences can be in content, layout, SEO metadata, HTTP status, links, scripts or specific elements like a button or a form.

If you want the non-technical overview first, read what is website change monitoring. If you want to compare products, see the best website change detection tools. This article is about how it works.

The pipeline at a glance#

1. Fetch the page, with or without a browser.
2. Stabilize it so it looks the same every time.
3. Normalize the HTML so meaningless differences disappear.
4. Extract the parts you care about.
5. Fingerprint each part with a hash.
6. Compare with the baseline: text, tags, pixels.
7. Filter noise.
8. Rate severity and alert.
9. Approve expected changes into a new baseline.

Step 1: Fetch the page#

There are two ways to get a page.

A plain HTTP request downloads the HTML the server sends. It's fast and cheap, and it's enough for server-rendered pages and for checking status codes, redirects and headers. But it doesn't run JavaScript, so on a React, Vue or Shopify storefront you may get an almost empty shell, and you won't see scripts injected by a tag manager.

A headless browser (Chrome driven by a tool like Playwright or Puppeteer) loads the page as a visitor would: it runs JavaScript, loads fonts and images and builds the final DOM. It's slower and uses far more memory, but it's the only way to capture what visitors actually see and to take a screenshot.

Most serious tools use a browser for the full check and plain requests for lighter work like checking whether linked URLs still respond.

While fetching, record the HTTP status, the final URL after redirects, the response time, the number of requests and the total bytes. Those become the availability and performance signals later.

Step 2: Stabilize the page#

A page that's still loading will look different every time you capture it. Stabilization means:

• Wait for the DOM to load, then for web fonts.
• Wait for a short network quiet period, but with a hard limit. Some pages poll a server forever, so "wait until the network is idle" can hang.
• Disable CSS animations and transitions and hide blinking text cursors, so a screenshot doesn't catch a slide halfway through.
• Scroll the page to trigger lazy-loaded images, then return to the top.
• Use a fixed viewport, device scale, locale and timezone on every check. A different window size or language is a different page.

Skipping this step is the single largest source of false alerts.

Step 3: Normalize the HTML#

Never compare raw HTML. Instead, build a cleaned version where meaningless differences are removed:

• Remove comments, and script and style contents (scripts are tracked separately).
• Remove volatile attributes: CSRF tokens, CSP nonces, framework hydration markers and build hashes in class names where it's safe.
• Sort attributes so class="a" id="b" and id="b" class="a" are equal.
• Collapse whitespace.
• Remove elements the user has marked as ignored (a rotating banner, a "last updated" line, a live chat widget).
• Optionally normalize dates and times to a placeholder.

After this, two loads of an unchanged page should produce identical output.

Step 4: Extract the parts that matter#

A page isn't one thing to compare. Split it into parts, because each kind of change means something different:

You can see most of these for any page right now with free tools like the meta tag checker and the script detector.

Step 5: Fingerprint each part#

Hash each normalized part (for example with SHA-256): one hash for the raw HTML, one for the normalized DOM, one for the visible text, one for the screenshot.

Hashes make the common case cheap. On most checks, most pages haven't changed, so if the text hash matches the baseline's text hash you know the text is identical and can skip the text diff. Only parts whose hash changed need a real comparison.

Comparing several hashes also tells you what kind of change happened. If the raw HTML hash changed but the normalized DOM hash didn't, only noise changed. If the DOM changed but the text didn't, the change is structural or stylistic.

Step 6: Compare with the baseline#

Compare against an approved baseline, not the last check#

The reference version should be a baseline you approved, not simply the previous check. If you compare with the previous check, a change detected on Monday disappears on Tuesday, because Tuesday is compared with Monday's already changed page. With an approved baseline, the change keeps showing until someone fixes it or approves it.

Text diffs#

For visible text, use a line or word level diff (the classic Myers algorithm behind git diff and most diff libraries). Show removed text and added text side by side. Diffing by word or sentence rather than by character keeps the output readable.

Field comparisons#

SEO fields are compared as values: old title vs new title, old canonical vs new canonical, index vs noindex, old status vs new status. These are exact comparisons and they're the most reliable signals you have.

Set comparisons#

Links and scripts are sets. Compare them as added and removed items, not as a diff of the page. "3 scripts added, 1 removed: googletagmanager.com" is useful. A raw diff of <script> tags is not.

Screenshot diffs#

Compare the new screenshot with the baseline pixel by pixel, which is what libraries like pixelmatch do. Two settings matter:

• A per-pixel tolerance, so tiny color differences from font smoothing and image compression aren't counted.
• A percentage threshold for the whole page. Under about 1% of pixels changed is usually noise. Larger differences are reported with a diff image that highlights the changed areas.

If the page height changed, compare the overlapping area and report the size change separately, otherwise everything below the change looks different.

Step 7: Filter the noise#

Even after stabilization and normalization, some content changes by design. Good systems give you controls:

• Ignored selectors remove elements from the text and DOM comparison.
• Screenshot masks blank out areas before the pixel diff.
• Thresholds per page or per site.
• Confirmation scans: re-check before alerting on something that may be temporary, like a single timeout.
• Grouping: one alert per scan, with "17 internal links now return 404" instead of 17 emails.

Noise control is a product feature, not a setting. If the alerts are noisy, people stop reading them, and then the one alert that matters gets missed too.

Step 8: Rate severity and alert#

Not every change is equal. A rules engine turns differences into severity levels:

Then send immediate alerts only for the levels that need action, and keep the rest in a dashboard.

Step 9: Approve the new normal#

When a change is expected, such as a new pricing page you just shipped, approve it. The current snapshot becomes the new baseline and the next check compares against it. Keep old baselines as history so you can always see what the page looked like before.

Detecting web page changes yourself#

You can build a basic version with a script and a scheduler:

1. On a schedule, load the page in Playwright and wait for it to settle.
2. Extract the visible text from the main content area and the title, canonical and robots meta.
3. Hash them and compare with the hashes saved last time you approved the page.
4. If a hash differs, diff the values and send yourself an email.

That works for a few pages. The hard part comes later: false alerts from dynamic content, screenshots that never match exactly, browsers running out of memory, pages that block automated traffic, storing screenshots, and doing it all safely without letting a user-supplied URL reach your internal network. That's where a dedicated tool earns its cost. For a comparison of approaches, from manual checks to browser extensions to monitoring services, see how to monitor website changes.

To try change detection on a page right now with no setup, use the free website change detector.

Summary#

Web page change detection is a pipeline, and each step removes a source of false alerts: render the page properly, let it settle, normalize away tokens and timestamps, split it into parts, hash them, compare each part the right way against an approved baseline, filter what's left and rate it by severity. Done well, you get few alerts, and each one is worth opening.