How MyKavo crawls, and how it protects your network

Identifying itself

Scans use a user agent naming MyKavoBot and linking to a page explaining what it is, so your logs and your firewall can tell what the traffic is. MyKavo does not disguise itself as a normal browser.

Respecting robots.txt

robots.txt is read during discovery and respected. MyKavo does not include a setting to bypass it.

Why every URL is re-validated

A monitoring service fetches URLs supplied by its users, which makes it an attractive way to reach things the internet cannot otherwise see - internal dashboards, cloud metadata endpoints, services on a private network. MyKavo therefore treats every URL as untrusted, no matter where it came from.

• Only HTTP and HTTPS are allowed; URLs containing credentials are rejected.
• DNS is resolved and the resulting address checked before the request is made.
• Localhost, loopback, private, link-local and reserved ranges are blocked, as are cloud metadata endpoints.
• Redirects are re-validated at every hop, with a maximum redirect count - a public URL that redirects to an internal one is stopped at the redirect.
• Request timeouts, maximum response size, and per-workspace rate limits are all enforced.

Bounded crawling

Discovery has maximum page counts, maximum discovered URLs and a maximum scan duration. MyKavo will not crawl a large site indefinitely, which protects your server as much as its own.

More in plans, security and data

• Plans and limits
• Your data, and what deletion actually deletes
• All documentation