{
  "name": "How to Diagnose and Fix Website Crawlability Problems",
  "step": [
    {
      "name": "Step 1: Identify Affected URLs and Patterns — Start by looking at your Google Search Console reports. Are there specific pages or sections of your site that Google isn't crawling or indexing? Note any errors or warnings related to coverage, like 'Indexed, though blocked by robots.txt' or 'Excluded by ‘noindex’ tag.' This helps narrow down where to focus your efforts.",
      "text": "Step 1: Identify Affected URLs and Patterns — Start by looking at your Google Search Console reports. Are there specific pages or sections of your site that Google isn't crawling or indexing? Note any errors or warnings related to coverage, like 'Indexed, though blocked by robots.txt' or 'Excluded by ‘noindex’ tag.' This helps narrow down where to focus your efforts.",
      "@type": "HowToStep",
      "position": 1
    },
    {
      "name": "Step 2: Inspect Individual URLs in Search Console — For a few representative problem URLs, use the URL Inspection tool in Google Search Console. Request a live test to see exactly how Googlebot renders the page and if it encounters any issues accessing resources. This can reveal if JavaScript isn't loading correctly or if there are other access barriers.",
      "text": "Step 2: Inspect Individual URLs in Search Console — For a few representative problem URLs, use the URL Inspection tool in Google Search Console. Request a live test to see exactly how Googlebot renders the page and if it encounters any issues accessing resources. This can reveal if JavaScript isn't loading correctly or if there are other access barriers.",
      "@type": "HowToStep",
      "position": 2
    },
    {
      "name": "Step 3: Review `robots.txt` and `noindex` Directives — Check your `robots.txt` file to ensure you're not accidentally blocking critical parts of your site that you want indexed. Also, confirm that `noindex` tags aren't present on pages meant for public search results. A common mistake is using `robots.txt` to try and deindex pages, when `noindex` is the proper method for that.",
      "text": "Step 3: Review `robots.txt` and `noindex` Directives — Check your `robots.txt` file to ensure you're not accidentally blocking critical parts of your site that you want indexed. Also, confirm that `noindex` tags aren't present on pages meant for public search results. A common mistake is using `robots.txt` to try and deindex pages, when `noindex` is the proper method for that.",
      "@type": "HowToStep",
      "position": 3
    },
    {
      "name": "Step 4: Check HTTP Status Codes and Server Accessibility — Ensure your server is responding with 200 OK for all indexable pages. Look for 4xx (client errors) or 5xx (server errors) codes, which tell Google that pages are unavailable. Also, make sure Googlebot can consistently access your server without timeouts or other intermittent issues.",
      "text": "Step 4: Check HTTP Status Codes and Server Accessibility — Ensure your server is responding with 200 OK for all indexable pages. Look for 4xx (client errors) or 5xx (server errors) codes, which tell Google that pages are unavailable. Also, make sure Googlebot can consistently access your server without timeouts or other intermittent issues.",
      "@type": "HowToStep",
      "position": 4
    },
    {
      "name": "Step 5: Verify Sitemaps and Canonical Signals — Your XML sitemap should only list pages you want indexed, and it shouldn't contain redirects or canonicalized URLs. For pages with duplicate content, ensure canonical tags are correctly pointing to your preferred version. Incorrect sitemap entries or canonicals can confuse search engines about which version to prioritize.",
      "text": "Step 5: Verify Sitemaps and Canonical Signals — Your XML sitemap should only list pages you want indexed, and it shouldn't contain redirects or canonicalized URLs. For pages with duplicate content, ensure canonical tags are correctly pointing to your preferred version. Incorrect sitemap entries or canonicals can confuse search engines about which version to prioritize.",
      "@type": "HowToStep",
      "position": 5
    },
    {
      "name": "Step 6: Correct Technical Errors and Request Recrawling — Once you've identified and fixed any blocking `robots.txt` rules, `noindex` tags, server errors, or sitemap issues, it's time to inform Google. Use the URL Inspection tool to request re-crawling for specific fixed URLs, or resubmit an updated sitemap if changes are site-wide. Keep monitoring your Search Console reports for improvements in crawl and index status.",
      "text": "Step 6: Correct Technical Errors and Request Recrawling — Once you've identified and fixed any blocking `robots.txt` rules, `noindex` tags, server errors, or sitemap issues, it's time to inform Google. Use the URL Inspection tool to request re-crawling for specific fixed URLs, or resubmit an updated sitemap if changes are site-wide. Keep monitoring your Search Console reports for improvements in crawl and index status.",
      "@type": "HowToStep",
      "position": 6
    }
  ],
  "@type": "HowTo",
  "@context": "https://schema.org",
  "speakable": {
    "@type": "SpeakableSpecification",
    "cssSelector": [
      "h1",
      ".howto-step",
      "[itemprop=\"step\"]"
    ]
  },
  "totalTime": "PT30M",
  "areaServed": [
    {
      "name": "New York City",
      "@type": "City"
    },
    {
      "name": "Los Angeles",
      "@type": "City"
    },
    {
      "name": "Chicago",
      "@type": "City"
    },
    {
      "name": "Houston",
      "@type": "City"
    },
    {
      "name": "Phoenix",
      "@type": "City"
    },
    {
      "name": "Philadelphia",
      "@type": "City"
    },
    {
      "name": "San Antonio",
      "@type": "City"
    },
    {
      "name": "San Diego",
      "@type": "City"
    },
    {
      "name": "Dallas",
      "@type": "City"
    },
    {
      "name": "San Jose",
      "@type": "City"
    },
    {
      "name": "Austin",
      "@type": "City"
    },
    {
      "name": "Jacksonville",
      "@type": "City"
    },
    {
      "name": "Fort Worth",
      "@type": "City"
    },
    {
      "name": "Columbus",
      "@type": "City"
    },
    {
      "name": "Charlotte",
      "@type": "City"
    },
    {
      "name": "San Francisco",
      "@type": "City"
    },
    {
      "name": "Indianapolis",
      "@type": "City"
    },
    {
      "name": "Seattle",
      "@type": "City"
    },
    {
      "name": "Denver",
      "@type": "City"
    },
    {
      "name": "Oklahoma City",
      "@type": "City"
    },
    {
      "name": "Boston",
      "@type": "City"
    },
    {
      "name": "El Paso",
      "@type": "City"
    },
    {
      "name": "Nashville",
      "@type": "City"
    },
    {
      "name": "Detroit",
      "@type": "City"
    },
    {
      "name": "Portland",
      "@type": "City"
    },
    {
      "name": "Las Vegas",
      "@type": "City"
    },
    {
      "name": "Memphis",
      "@type": "City"
    },
    {
      "name": "Louisville",
      "@type": "City"
    },
    {
      "name": "Baltimore",
      "@type": "City"
    },
    {
      "name": "Milwaukee",
      "@type": "City"
    },
    {
      "name": "Albuquerque",
      "@type": "City"
    },
    {
      "name": "Tucson",
      "@type": "City"
    },
    {
      "name": "Fresno",
      "@type": "City"
    },
    {
      "name": "Sacramento",
      "@type": "City"
    },
    {
      "name": "Kansas City",
      "@type": "City"
    },
    {
      "name": "Mesa",
      "@type": "City"
    },
    {
      "name": "Atlanta",
      "@type": "City"
    },
    {
      "name": "Omaha",
      "@type": "City"
    },
    {
      "name": "Colorado Springs",
      "@type": "City"
    },
    {
      "name": "Miami",
      "@type": "City"
    },
    {
      "name": "Raleigh",
      "@type": "City"
    },
    {
      "name": "Long Beach",
      "@type": "City"
    },
    {
      "name": "Virginia Beach",
      "@type": "City"
    },
    {
      "name": "Oakland",
      "@type": "City"
    },
    {
      "name": "Minneapolis",
      "@type": "City"
    },
    {
      "name": "Tampa",
      "@type": "City"
    },
    {
      "name": "Tulsa",
      "@type": "City"
    },
    {
      "name": "Arlington",
      "@type": "City"
    },
    {
      "name": "New Orleans",
      "@type": "City"
    },
    {
      "name": "Wichita",
      "@type": "City"
    },
    {
      "name": "Cleveland",
      "@type": "City"
    },
    {
      "name": "Bakersfield",
      "@type": "City"
    },
    {
      "name": "Aurora",
      "@type": "City"
    },
    {
      "name": "Anaheim",
      "@type": "City"
    },
    {
      "name": "Honolulu",
      "@type": "City"
    },
    {
      "name": "Santa Ana",
      "@type": "City"
    },
    {
      "name": "Riverside",
      "@type": "City"
    },
    {
      "name": "Corpus Christi",
      "@type": "City"
    },
    {
      "name": "Lexington",
      "@type": "City"
    },
    {
      "name": "Stockton",
      "@type": "City"
    },
    {
      "name": "Henderson",
      "@type": "City"
    },
    {
      "name": "Saint Paul",
      "@type": "City"
    },
    {
      "name": "St. Louis",
      "@type": "City"
    },
    {
      "name": "Cincinnati",
      "@type": "City"
    },
    {
      "name": "Pittsburgh",
      "@type": "City"
    },
    {
      "name": "Greensboro",
      "@type": "City"
    },
    {
      "name": "Anchorage",
      "@type": "City"
    },
    {
      "name": "Plano",
      "@type": "City"
    },
    {
      "name": "Lincoln",
      "@type": "City"
    },
    {
      "name": "Orlando",
      "@type": "City"
    },
    {
      "name": "Irvine",
      "@type": "City"
    },
    {
      "name": "Newark",
      "@type": "City"
    },
    {
      "name": "Durham",
      "@type": "City"
    },
    {
      "name": "Chula Vista",
      "@type": "City"
    },
    {
      "name": "Toledo",
      "@type": "City"
    },
    {
      "name": "Fort Wayne",
      "@type": "City"
    },
    {
      "name": "St. Petersburg",
      "@type": "City"
    },
    {
      "name": "Laredo",
      "@type": "City"
    },
    {
      "name": "Jersey City",
      "@type": "City"
    },
    {
      "name": "Charlotte",
      "@type": "City"
    },
    {
      "name": "Madison",
      "@type": "City"
    },
    {
      "name": "Chandler",
      "@type": "City"
    },
    {
      "name": "Buffalo",
      "@type": "City"
    },
    {
      "name": "Lubbock",
      "@type": "City"
    },
    {
      "name": "Scottsdale",
      "@type": "City"
    },
    {
      "name": "Reno",
      "@type": "City"
    },
    {
      "name": "Glendale",
      "@type": "City"
    },
    {
      "name": "Gilbert",
      "@type": "City"
    },
    {
      "name": "Winston-Salem",
      "@type": "City"
    },
    {
      "name": "North Las Vegas",
      "@type": "City"
    },
    {
      "name": "Norfolk",
      "@type": "City"
    },
    {
      "name": "Chesapeake",
      "@type": "City"
    },
    {
      "name": "Garland",
      "@type": "City"
    },
    {
      "name": "Irving",
      "@type": "City"
    },
    {
      "name": "Hialeah",
      "@type": "City"
    },
    {
      "name": "Fremont",
      "@type": "City"
    },
    {
      "name": "Boise",
      "@type": "City"
    },
    {
      "name": "Richmond",
      "@type": "City"
    },
    {
      "name": "Baton Rouge",
      "@type": "City"
    },
    {
      "name": "Spokane",
      "@type": "City"
    },
    {
      "name": "Des Moines",
      "@type": "City"
    },
    {
      "name": "Tacoma",
      "@type": "City"
    },
    {
      "name": "San Bernardino",
      "@type": "City"
    },
    {
      "name": "Modesto",
      "@type": "City"
    },
    {
      "name": "Fontana",
      "@type": "City"
    }
  ],
  "description": "Step 1: Identify Affected URLs and Patterns — Start by looking at your Google Search Console reports. Are there specific pages or sections of your site that Google isn't crawling or indexing? Note any"
}