Introduction
If you're managing a WordPress site with Cloudflare, you may encounter a puzzling problem. Your images display perfectly when you view them directly in your browser, yet SEO tools like Ahrefs flag them as broken.
This can be quite frustrating, especially when you're trying to boost your SEO health score or resolve technical issues. The problem often isn't with the images themselves; it usually stems from how Cloudflare interacts with requests from bots, crawlers, or external tools.
In this guide, we'll explore why this issue occurs, how to diagnose it, and most importantly, how to fix it step by step.
Understanding the Problem
Before diving into solutions, it's crucial to grasp what's really going on.
When you access an image directly in your browser, you're making a request as a regular user. In contrast, when tools like Ahrefs or other SEO crawlers scan your site, they act like bots.
Cloudflare treats these two types of requests differently.
Your images aren't broken; they're simply being blocked or restricted for certain types of requests.
Common Causes of Broken Images in SEO Tools
Hotlink Protection
Cloudflare has a feature called Hotlink Protection that stops other websites from directly linking to your images. While this is great for preventing bandwidth theft, it can also inadvertently block legitimate crawlers. If Ahrefs attempts to access your images without the right headers or referrer info, Cloudflare might deny access.
Firewall Rules Blocking Bots
Cloudflare's firewall rules can block or challenge traffic based on various factors like user agents, countries, and request patterns. SEO tools often deploy custom crawlers, which Cloudflare might flag as suspicious traffic.
Bot Fight Mode
Cloudflare features a Bot Fight Mode that aggressively blocks any non-human traffic. This can create issues for Ahrefs, SEMrush, Screaming Frog, and other SEO crawlers.
Caching or CDN Misconfiguration
At times, images might be cached incorrectly or not served properly through the CDN. This can lead to 403 Forbidden errors, 404 Not Found responses for bots, and inconsistent behavior between browsers and crawlers.
Missing or Restricted HTTP Headers
Some SEO tools fail to send complete browser-like headers. Cloudflare may reject requests that lack a referrer, have unusual user agents, or don't comply with security rules.
How to Diagnose the Issue
Before jumping into fixes, it's crucial to pinpoint the root cause.
Check Image URL Response
Open your image URL in a browser, incognito mode, and Curl or Postman. If it works in the browser but not through the tools, it's likely a restriction from Cloudflare.
Use Curl Command
Run this command and look for the response code — 200 OK means working, 403 Forbidden means blocked, 404 means missing or restricted.
Check Cloudflare Firewall Logs
Navigate to Cloudflare Dashboard → Security → Events and look for blocked requests from AhrefsBot, SemrushBot, or other crawlers.
Step by Step Solutions
Now, let's tackle the issue.
Fix 1: Disable or Adjust Hotlink Protection
Head over to Cloudflare Dashboard → Scrape Shield. Temporarily turn off Hotlink Protection or allow trusted bots. If disabling resolves the issue, consider creating exceptions instead of keeping it off permanently.
Fix 2: Whitelist SEO Crawlers
Set up a firewall rule in Cloudflare using the User Agent field with the appropriate operator to allow trusted SEO crawlers like AhrefsBot and SemrushBot through without being blocked.
Fix 3: Temporarily Disable Bot Fight Mode
Head over to Security → Bots and turn off Bot Fight Mode and give it another shot. If this does the trick, you might want to consider switching to Super Bot Fight Mode with your own custom rules.
Fix 4: Tweak Your Firewall Rules
If you've set up custom rules, take a look for country blocking, rate limiting, and challenge rules. Adjust these rules so they don't block genuine crawlers.
Fix 5: Double-Check Your Image URLs
Make sure that images are returning HTTP 200, there are no redirect loops, and your HTTPS configuration is spot on. Also, ensure that your WordPress URLs align with your Cloudflare settings.
Fix 6: Clear Your Cloudflare Cache
Navigate to Caching → Purge Everything. This will make sure that updated image responses are served correctly to all visitors and crawlers.
Fix 7: Verify MIME Types
Check that your server is sending the right headers: image/jpeg, image/png, or image/webp. Incorrect MIME types can throw crawlers off and cause them to flag images as inaccessible.
Practical Tips for WordPress Users
Choose Reliable Hosting
Subpar hosting can lead to inconsistent responses for bots.
Be Cautious with Security Plugins
Plugins like Wordfence or iThemes Security might clash with Cloudflare.
Test with Various Tools
Don't just stick to one SEO tool. Try using Ahrefs, Screaming Frog, and Google Search Console.
Keep an Eye on Server Logs
Look at access logs to see if crawlers are being blocked at the server level.
Utilize Cloudflare Page Rules
Set up rules to bypass security for image directories. For example, for yourdomain.com/wp-content/uploads/*, set Security Level to Essentially Off.
Common Mistakes to Avoid
Blocking All Bots
A lot of users accidentally block all bots except Googlebot, which can hurt SEO analysis and third-party tools.
Neglecting Firewall Logs
Cloudflare offers detailed logs, but many users overlook them. This is the quickest way to pinpoint the issue.
Overusing Security Features
Turning on every security feature without proper configuration can disrupt normal functionality.
Skipping Testing After Changes
Always test after making adjustments. Don't just assume the problem is resolved.
Advanced Optimization
If you're looking to dive a little deeper, consider these options:
Creating Custom Rules Based on Paths
Restrict bots to image directories instead of allowing access to the entire site.
Using Cloudflare Workers
You can tweak request headers to make it easier for bots to access your content.
Monitoring with Real-Time Tools
Utilize tools that mimic bot behavior to keep a constant eye on accessibility.
FAQ Section
Conclusion
If your images are loading perfectly in the browser but appear broken in SEO tools, the problem is likely linked to how Cloudflare manages bot traffic. By adjusting hotlink protection, firewall rules, and bot settings, you can easily resolve the issue and ensure your images are accessible to crawlers. If you're running a business website and want to steer clear of these technical headaches, collaborating with professionals can save you time and prevent SEO setbacks. At CodesGarage, we specialize in helping businesses build, optimize, and maintain high-performance WordPress websites that are both secure and SEO-friendly. Whether it's fixing Cloudflare issues or enhancing technical SEO, our team is dedicated to making sure your website runs smoothly for both users and search engines.