Deployment Execution Blueprint

---
title: Multi-Page Internal Link Crawler and 404 Finder (PHP)
description: A bare-metal PHP script that crawls local site URLs to detect dead layout routing references and broken 404 links.
category: Server Config
slug: php-internal-link-crawler-404-finder
keywords: php site crawler, check broken links, find 404 errors php, automated link spider script, backend seo audit tool
---

### Overview & Problem Matrix
Dead hyperlinks and broken internal routing pathways severely damage your domain authority profiles, increase user bounce rates, and negatively impact search engine indexing visibility. 

Manually parsing every single anchor element across large content repositories or flat-file engines to spot broken links is practically impossible. You need a fast, zero-dependency server-side crawler utility that automatically map-scans your internal link architecture recursively, evaluates server response signatures, and exposes broken endpoints instantly.

### Implementation Guide & Setup Steps
To initialize this deep site auditing spider across your host nodes, execute these administrative steps:

1. Create the Script Asset: Save the optimized automation logic below to your site directory infrastructure path as `link_spider.php`:
   $ touch link_spider.php

2. Configure Your Source URL Target: Adjust the domain testing parameter at the absolute bottom of the script block to point directly to your own web application path:
   # Define your core target boundary inside the runner module:
   # trace_internal_link_health('https://yourdomain.com/');

3. Execute via Command-Line Interface (CLI): Trigger the crawler routine straight from your server console node to watch the recursive validation pipeline crawl paths and catch errors in real time:
   $ php link_spider.php

function trace_internal_link_health($base_domain_url) {
    $scanned_links = [];
    $crawl_queue = [$base_domain_url];
    $broken_links_found = 0;

    print "Starting recursive verification process across: {$base_domain_url}\n";

    while (!empty($crawl_queue)) {
        $current_url = array_shift($crawl_queue);
        
        if (in_array($current_url, $scanned_links)) continue;
        $scanned_links[] = $current_url;

        // Fetch target raw document mapping matrix via cURL
        $ch = curl_init($current_url);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
        curl_setopt($ch, CURLOPT_TIMEOUT, 6);
        curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true); // Follow redirects to trace final destinations
        $html_payload = curl_exec($ch);
        $http_code = curl_getinfo($ch, CURLINFO_HTTP_CODE);
        curl_close($ch);

        if ($http_code !== 200) {
            print "[ALERT] Route Broken: {$current_url} returned Code {$http_code}\n";
            $broken_links_found++;
            continue;
        }

        // Parse HTML markup for links safely without dependencies
        dom_regex_find_links($html_payload, $base_domain_url, $crawl_queue, $scanned_links);
    }

    print "Link verification matrix processed. Identified {$broken_links_found} dead anchors.\n";
}

function dom_regex_find_links($html, $base, &$queue, $scanned) {
    preg_match_all('/href=["\']([^"\']+)["\']/i', $html, $matches);
    if (!empty($matches[1])) {
        foreach ($matches[1] as $href) {
            // Trim whitespace and remove hash fragments to avoid duplicate page scanning loops
            $href = explode('#', trim($href))[0];
            
            if (empty($href)) continue;

            // Resolve relative paths safely to stay on the target domain
            if (strpos($href, '/') === 0) {
                $href = rtrim($base, '/') . '/' . ltrim($href, '/');
            }
            
            // Clean up duplicate slash formatting anomalies
            $href = preg_replace('/([^:])\/{2,}/', '$1/', $href);
            
            // Filter queue to ensure it doesn't leave the internal root domain boundaries
            if (strpos($href, $base) === 0 && !in_array($href, $scanned) && !in_array($href, $queue)) {
                $queue[] = $href;
            }
        }
    }
}

if (php_sapi_name() === 'cli') {
    trace_internal_link_health('https://howtodocument.com/');
}