The Screaming Frog SEO Spider is a website crawler that allows you to scan websites for technical and onsite SEO issues.
By default, every website in the free version of Screaming Frog has about 500 URLs that can be crawled without paying. If you want to analyze larger websites that contain more URLs, Screaming Frog asks for a £149 upgrade (good 170 euros).
Review: Is Screaming Frog worth its money?
For an SEO specialist Screaming Frog is definitely a must to have in house. This tool helps you to easily discover very important information about websites, regarding their SEO problems. Some examples.
- Page depth: How many clicks does it take to visit a web page? Search engines usually don’t crawl deeper than 4 clicks. If you have interesting content that is too “deep” on your site, try to get it 1 or more levels higher. This can be done for example by: a right column with a section ‘Interesting posts’ or internal linking via webpages at a higher level.
- Status Code: to detect non-working links. Here you can also discover the 404-pages.
- Duplicate content: Search engines don’t like duplicate content. With the right filters and hashtags you can look up duplicate content and duplicate meta data via Screaming Frog.
- Images: are they correctly indexed? Aren’t they too big? Especially for mobile traffic, too large images are not recommended. And which images don’t have an alt-tag?
- Indexing: which sitepages are blocked for the search engines? Look at the tab ‘Directives’ and filter on ‘Noindex’.
- Response codes: Fast and mobile friendly sites have more than an edge in the SERPs. Through the tab ‘response codes’ you can quickly know the response times of the URLs present on the site.
- And much, much more: Redirects, External Links, Internal Links, Anchor Links, Protocol (URLs https secured or not), Onpage missing information such as H1, H2, meta descriptions, …