Website Crawler and Broken Link Checker

A website crawler and broken link checker for up to 100 pages: broken links, redirects, missing titles and duplicates. 5 credits per run.

Most SEO and site problems do not live on the homepage. They sit on page 40 of a blog archive, in an old product page that now redirects three times, or in a set of pages that all share one title. You rarely find these by clicking around, because nobody visits most of the pages on a site more than once.

The Whole-Site Crawl from Dynamic Growth is an automated service. It costs 5 credits and needs only your website URL. Our software follows the links on your public site, visits up to 100 pages, and checks each one. The report appears in your dashboard to download. You get findings on broken links, redirect chains, titles, descriptions, headings, thin pages, indexing, click depth, and slow or large pages. Charts show the overall picture, and a CSV file lists every page it visited so you can sort and filter it yourself.

What you get

Report on up to 100 pages: broken links, missing titles, duplicates and slow or thin pages.

Broken link report

A list of links on your site that lead to pages returning errors, with the page each link sits on. You can fix the link, restore the page, or add a redirect where an old address has a clear replacement.

Redirect chain report

The URLs that pass through more than one redirect before reaching their final page, with each hop listed. Shortening a chain to a single redirect removes extra steps for visitors and crawlers.

Titles, descriptions and headings review

For every crawled page, whether the title, meta description and main heading are present, repeated across pages, or unusually long or short. These are quick to fix and show how each page presents itself.

Thin page and indexing findings

Pages with very little text on them, and pages that carry a noindex directive or a canonical tag pointing elsewhere. The report lists them so you can confirm each one is deliberate.

Click depth and site structure view

How many clicks it takes to reach each page from the homepage. Pages buried deep are harder to find for crawlers and visitors, and the chart shows where your structure gets deep.

Slow and large page list

Pages that responded slowly or came back with a heavy file size during the crawl, ranked so you can see which ones to look at first. This is a pointer to investigate, and a full performance study is a separate job.

Charts and a CSV list of pages

Charts summarize the results across the site. The CSV lists every page crawled with its status, title, description, heading, depth and size, ready to open in a spreadsheet and sort.

What we need from you

Your website URL.

Up to 100 pages crawled
Broken links and redirect chains
Titles, descriptions and headings
Thin pages, indexing and click depth
Slow and large pages
Charts and a CSV list of pages

Why crawl the whole site

A single-page check tells you about one page. Sites break in patterns, and patterns only show up when you look at many pages together. If a template leaves the meta description empty, every page that uses it has the same gap. If a menu link points to a deleted page, the dead link appears on every page that carries the menu. A crawl surfaces these repeats so you can fix one cause and clear many findings.

Some problems are invisible to visitors. A redirect chain still loads, but each hop adds delay and gives a search engine one more step to follow. A page set to noindex still looks normal in a browser. A page that sits eight clicks from the homepage still exists, but crawlers and people reach it rarely. The only way to see these is to follow the links the way a crawler does and write down what each page says about itself.

A crawl also gives you a list to work from. Most sites have never had a full inventory of their pages. Having one in a spreadsheet, with status codes, titles, headings and sizes next to each URL, makes cleanup, migrations and content planning easier. Software runs the scan, so read the findings with your own knowledge of the site: some pages are meant to be thin or hidden, and the report cannot know which.

How it works

Enter your website URL
Type in your site address and confirm the crawl. It costs 5 credits and needs no uploads and no connection to your accounts. The URL is the only input.
Automatic crawl and analysis
Our software starts at your homepage, follows internal links and visits up to 100 pages. It records the status, redirects, tags, headings, size and response time of each one, then checks the results for problems.
Read and download the result
The report appears in your dashboard. Read the summary and charts first, then open the findings by type. Download the report and the CSV list of pages for your records or to share with a developer.
Apply the fixes yourself
Work through the findings in your CMS or hand them to your developer. Check each one before you change it, since software wrote the list and a flagged page may be intentional. Your site stays as it is until you edit it.

Common problems the crawl looks for

Broken internal links

Menus, footers and old blog posts often link to pages that no longer exist. Visitors land on an error page and crawlers waste time. Update the link or redirect the old address to the closest live page.

Redirect chains

After several site changes, one old URL can redirect to another that redirects again. Point the first URL straight to the final page, and update internal links so they skip the redirect entirely.

Missing or repeated titles and descriptions

Pages created from the same template can share a title or leave the description blank. Write a distinct title and description for each page that you want people to find.

Pages set to noindex by mistake

A noindex tag left on after a redesign keeps a page out of search results. The report lists every page that carries one so you can confirm each is intended.

Slow or heavy pages

Large images and unneeded scripts make some pages slow to load. The list shows which pages stood out during the crawl, so you can compress images and remove what you do not use.

What the crawl measures

Pages crawled and their status codes. How many pages were visited, and how many returned success, redirect or error responses. This gives a quick read on the overall health of the site.
Broken and redirected links. The count of links pointing to error pages and to redirected addresses. A lower count means fewer dead ends and fewer extra hops.
Title and description coverage. The share of crawled pages with a title and a meta description, and how many of those are repeated. This shows how complete your page labeling is.
Click depth. The number of clicks from the homepage to each page, shown as a distribution. It shows how deep your structure goes.
Response time and page size. How quickly each page answered during the crawl and how large the response was. It helps you spot the slowest and heaviest pages.

Do it yourself or run the automated crawl?

You can crawl a site yourself. Desktop crawler tools exist, some with free tiers, and Google Search Console reports on indexing for sites you own. If you are comfortable installing software and reading its output, and you have the time to set it up, you will get detailed data. Search Console also shows what Google itself sees, which an outside crawl cannot.

The automated crawl suits owners who want the answer without setup. Enter a URL and read a report with charts and a page list. It visits up to 100 pages, so a larger site gets a sample from the start of the link structure, not a full inventory. It reads public pages only and does not reach pages behind a login. Treat the findings as a starting list and confirm anything important before you change templates or redirects.

Who it's for

Best for

Site owners who have never had a full list of their pages
Businesses that recently redesigned, migrated or changed their URLs
Marketers preparing a content or SEO plan who want a clean base
Developers and agencies who need a CSV of pages to work through

Not the right fit if

Sites with thousands of pages that need every URL checked
Pages behind a login or paywall, which the crawler cannot reach
Anyone who wants traffic or ranking data, since the crawl reads page content only

A site audit with a link checker and SEO crawler built in

The Whole-Site Crawl is a website crawler that follows your internal links from the page you give it. It works as a broken link checker, a site crawler and an SEO crawler in one report, and the same run doubles as a site audit for missing titles, duplicate titles, thin pages and redirect chains. A link checker that reads one page at a time misses most of these problems.

Teams in the USA, the UK, Europe, the Middle East and Latin America use a crawl before a migration, after a redesign and when rankings slip. The crawl follows links on the public page, so it sees what a visitor or a search engine bot would reach without logging in.

Frequently asked questions

How many pages does the crawl cover?

It visits up to 100 pages of your site, starting from the URL you enter and following the links it finds. If your site has more pages than that, the report covers the first 100 it reaches.

What does it cost and what do I need to provide?

It costs 5 credits and runs automatically from your URL alone. You do not connect any accounts, and we never log in to your site.

Will the crawl fix the problems it finds?

No. It finds and lists them, and each finding says what to look at. The edits are yours or your developer's to make.

What is in the CSV file?

One row for each crawled page, with its address, status code, title, description, main heading, click depth and size. Open it in a spreadsheet to sort, filter and share it.

How is this different from the SEO Audit?

The SEO Audit is a technical and on-page review that costs 2 credits. The Whole-Site Crawl costs 5 credits and goes through up to 100 pages one by one, with a page-level CSV and charts across the site.

Does it show my traffic or rankings?

No. It reads the content and structure of your public pages. It has no access to your analytics or Search Console and shows no traffic or ranking data.

How much does whole-site crawl cost?

Whole-Site Crawl costs 5 credits. On the Starter plan ($29 for 10 credits) that is about $14.50, on Growth ($59 for 25 credits) about $11.80, and on Scale ($129 for 60 credits) about $10.75. Ad spend is always paid separately to the ad platforms.

What do you need from me?