Today your task in the 31 Days to Build a Better Blog Project is something that most bloggers who have been blogging for a while could probably benefit from doing – go on a dead link hunt.
Blogging is built on the ‘link’. One blog links to another blog who links to another who makes comment on another. This is a wonderful thing – but what happens when one of the blogs that you’re linking to is retired, is deleted, changes it’s link structure, moves etc?
The link is a dead one (also known as Link Rot) and can cost your blog on two fronts:
1. Readability – clicking on a dead link can mean your readers can end up on error pages or being redirected to other irrelevant content to the one they were expected to get to. This can lead to reader frustration or giving the impression that your blog is old and/or out of touch.
2. SEO – I’m not sure of the technicalities of it or what the latest research shows but from what I can tell a dead link is not looked upon favorably by search engines and you run the risk of penalties.
So how do you detect dead links on your blog?
The most obvious ‘solution’ is to surf every page on your blog and manually check all the links. This is something that might be achievable on a new blog – but on older blogs with hundreds or thousands of posts it’s just not feasible.
There are many link checking tools available but to be honest I’m yet to find one that I’m really happy with. I do hear that Xenu’s Link Sleuth is well regarded. I’ve also used the free version of Dead-Links.com (which only checks to a reasonably shallow depth) – but I’d be keen to hear from readers on their suggestions of other options.
Other dead link checking tools:
here are a few that are recommended in the WordPress Codex:
- Siteowner’s Link Checker
- Link Valet
- W3C free check link free utility
- AnyBrowser’s Link Checker
- HTMLHelp Valet Link Checker
- NetMechanic’s Link Check
- Squarefree’s Bookmarklets – javascripts for checking links
Feel free to suggest others below.
What to Do When You find a Dead Link?
There are a few options for what to do with dead links. They include:
- fixing/updating them – if the link is simply wrong or pointing to the wrong place update it so that it works
- deleting them – if they are dead and you can’t find a correct one then you can delete the link. I usually add an ‘update’ note to say that I’ve done this. I sometimes also update with new relevant links so that the post is still relevant.
- delete the post – on occasion I’ve done this if the whole post’s main point is to link to someone else’s post. A dead link makes this type of post obsolete so I consider deleting them rather than updating.
Whether you use a tool or just tackle the task manually a few posts at a time – finding and fixing dead links can be well worth the effort.
What do you Do
How do you find dead links? What do you do when you find them? Got any cool tools to share? Feel free to discuss here, or share with everyone in the forum post for this task.
Want More?This task is a sample of one of the tasks in the 31 Days to Build a Better Blog Workbook – a downloadable resource designed to reinvigorate and revitalize blogs. Join over 14,000 other bloggers and Get your Copy Today. |
One of the more annoying things involving dead links are when you link to images that get taken down. As a preventative measure, I always keep a local copy to put in its place as a precaution and email the original owner of the image (if it’s from a private site) to ask if I can use it with a link back to their blog instead of directly to the image.
Again Dareen a very usefull tips. It help us . I will do this .
Thanks for the post
Great post, Darren. I have been wondering about this for a while (Years, maybe?) and have finally found the solution. Thank you! :D
That’s very annoying when readers clicks on link taht go no where,I have to found tools to find all the dead links.I will try the tools you mentioned also.
I’ve never really bothered with dead links before. There have been a couple of occasions where I came across them accidentally and having found them I fix the problem, but to actively search for them, man that would take way to much time. I think I would rather just put up with a bit of link rot. That or pay my readers a dollar for every dead link they find. Surely there couldn’t be that many. Now there is an idea for a good competition ;)
I just did this 3 days ago!
Only thing I found odd is that some of the dead links were in the comments area. I have been taking out those dead links and leaving the comment.
I also found that some of my NoFollow links were shown as dead but, they were alive and well when I hand checked them.
My challenge for today is done, guess that means I have time to watch the horse race & the Nascar race.
If you’re using WordPress,
then try this plugin
it can save you a lot lot of time from checking the link
wordpress.org/extend/plugins/broken-link-checker/
it’s simple and it’s easy…
we can update it directly from control panel…
if you got the better plugin, please let me know…
That’s the though part. I didn’t even know that there was software to such entrepreneur. Having a blog with over a 1000 posts it’s hard to go and check one-by-one. I will keep my eyes peeled to see if someone comes up with a better solution.
I am enjoying the series. Thank you!
It’s actually quite easy (if you have strong programming knowledge of javascript or php) to create universal code that notifies or redirects dead links automatically.
For the less tech-savvy of you, you can keep a notebook of any changes you make to your blog and follow them right through your whole structure.
Even with a small blog, going through every page checking links is a rather monotonous task!
Cheers,
Sam
HazardousPaste: that’s not annoying, that’s bandwidth theft.
On my webpage (not blog) I made small, but useful script which e-mail me on every 404 and 403 entry. So when someone is coming to my non-existing page, they get nice styled error message with links to homepage, blog or microblog.
When someone is requesting non-existing image it simply display image with text, that image is removed.
Example; http://rolandinsh.lv/aefqwreqwsdfygsdhebqljahedv.png (image does not exist!)
On first few days I’ve got hundred e-mails with error messages, but after fixing HTACCESS file with dead links, now i’m getting maybe 1 e-mail per day.
About linking out… well on that I’m working now.
I use broken links checker http://w-shadow.com/blog/2007/08/05/broken-link-checker-for-wordpress/ which is pretty useful because it checks old posts on a regular basis and new posts too, so when I mistype a link it doesn’t impact all my readers because I have a message about a dead link in my dashboard just after publication. It gives enough time to correct it before it goes public.
I used Free Link Checker v4.3 to perform broken link check. It will give a broken link report after validation. It’s a simple but powerful free broken link tool.
Great idea – I have not thought about dead links since my blog is only about a year old. No dead links – hurrah!
I use “broken link checker” also and I find it works really well. It has drawn my attention to a few dead links that I then just fixed up.
If you decide to delete a link, you should also and a Redirect 410 /link/ to your .htaccess file so the search engines will stop looking for it. Google respects this, and will take the link out of their index. I am less sure about MSN and Yahoo!
Same applies to deleted categories and tags: do a 410 redirect.
Without the 410, you can wind up with a bunch of 404 errors that could, depending on the number, impact your quality score with the search engines.
I use the WordPress Broken Link Checker. It seems to do a good job. Runs in the background, and after a few days seems to go all the way through the site, picking up dead links.
Dead Links are a killer, for sure. I know when I am checking new sites out and I find a dead link or two, I usually blacklist the site and do not visit again.
This is another helpful article Darren. With Free Samples, they run out of stock sometimes and this creates problems with my older links often as they will lead to offers that are no longer going… but that should only be the ones that are a few weeks old and so on. I will have to check this out on my http://www.ezfreesamples.com blog. Thanks.
Xenu worked great for me – great tips!
Will also check the WP-plugin
WordPress has a plugin called bad link checker that works really well for this.
Good idea, Darren. And thanks, Sayz. I installed the WordPress plugin. Where on the Dashboard will it show any broken links?
I’m new enough to be able to do this manually… over a few days.
I found an incorrect link on a friend’s blog and let her know.. thought i should share that with her, I did it through email rather than a public comment.. figured she’d want to correct that as well…
So, as a courtesy I think it’s a good idea to do that as well.
I’m also , going back to an earlier lesson, doing a lot more linking of old posts to new ones.
A once in a while i will check for dead links but most of the time i don’t because it can be time consuming and boring to check every link on your site. I didn’t know they had tool that could help you with this. I will try some of them out.
I’ve used Google Webmaster for sometime but found it annoying soemtimes and not very helpful. I’m trying Xenu, the one recommended by Darren. let’s see how it goes.
Thanks for the ideas on programs that might find dead links. I’ve wondered how to remove dead links, but the thought of looking through nearly 1,000 posts on three blogs seemed daunting.
Rita, The Survive and Thrive Boomer Guide
Although I understand that a tool might be necessary for a larger blog with many, many posts, one additional advantage to hand checking for links is spending some time rereading older posts. It’s a way to look with fresh eyes at things written a long time ago – which might prompt either ideas for new posts or the need to rework the older posts.
It seems as though this day’s task could relate to the previous task to help another blogger. If I ever find a dead link on a blog of someone I read regularly, I’ll e-mail to let that person know. We may be more apt to find the dead links on someone else’s blog. We all help each other.
Thanks for the reminder that we all need to keep working to keep things fresh.
I’ve always wondered about this. Thanks for sharing how.
I used Dead-Links.com, and the errors are still coming up after 35 minutes (blog has been up since 2002 with at least one post a day. There’s a huge list appearing. Like I said it stills going after 35 minutes.
What is (? query string)?
I’m getting these two lines — a link and then X visited – X in the host – X out.
The X visited number keeps growing? Do I start at the first one and work through them.
Wow, the list is so long. I’ll be here for a year cleaning this up. Ouch.
I use TypePad. Does anyone know of a plug-in that works with it I can use on a weekly basis to keep it clean?
Thanks for the links Darren, I’m sure it’ll help a lot when I’m going to start digging dead links. Another great lesson here.
Time to put on my camouflage and go hunting for dead links. This is another great idea that I had overlooked. Thanks for sharing.
@ Sayz – Thanks heaps mate; I used that plugin you recommended and it worked a charm. I was able to remove 12 dead links.
I had no idea that there were websites for tracking dead links. I usually just check them manually. Those sites should prove to be time savers.
That’s right. Dead links will cost your site. I’d tried dead-link.scom and it’s works =D
Never bothered with dead links, takes too much time! But will try out the WP plugin thanks Sayz
I’ve used Xenu’s Link Sleuth. It’s an open source download. It takes awhile to scan your site, but once it does, you have a list of every broken link.
I didn’t do this for almost the first two years of my site and found literally hundreds of dead and broken links – news sources that were no longer there, blogs that closed up shop, etc. A common source of dead links was in the comments section from people linking to their sites which are no longer live, or people who left their site as “yahoo” or “google.” I delete those now as soon as I see them, but it never occurred to me for the first few months of blogging.
Budget several hours to clean up dead links if you have a lot of content and have never done it before. You are likely to have a lot of dead links. I worked on mine for half hour at a time over the course of a few days. Now I just run Xenu every other month or so and I only have to do a couple corrections.
I highly recommend cleaning your dead links, regardless of which plugin or software program you use.
I found that the carefully crafted index to important information on my site returned a 404 every time I tried to look for it (still don’t know why that happened). And I’d linked to it several times in subsequent messages.
I was able to get to it via WordPress’s admin-edit menu. Then I had to copy-paste the whole contents into another message, change the post date to match the original, and go change all the messages that linked to it. Royal pain, but at least now it’s visible if someone clicks a link.
I used http://www.bad-neighborhood.com/text-link-tool.htm some time ago to verify my dead links, and I found it pretty cool.. :)
I actually never looked at dead links but its good to have 301 redirects.
When you construct a sitemap via Google Webmaster tools the report generates that identifies broken links site wide. (Find this under diagnostics.)
It gives you are report with a variety of different errors but if there is a better option I would love it!
Like a few others up above, we rely on Google Webmasters tools and a sitemap. We do it a bit differently though as the tool we use to generate that sitemap also tells us what’s broken. We use the sitemap generator at auditmypc.com s that gives you the broken link report at the same time as a sitemap. I;ve recommended it previously and find it a great tool.
Hello!
Thanks for the information! I’ve discovered just two dead links in a post of mine. I am going to update it with new links.
Salvatore
I use ‘Broken Link Checker’: http://w-shadow.com/blog/2007/08/05/broken-link-checker-for-wordpress/ from Janis Elsts in my WordPress blogs. Very effective, shows broken links automatically in your Dashboard and you can edit and change them in the overview.
I have 9 broken links to edit now.
I have not done this previously, but I will try the pilugin suggested by Sayz. This is a task that I can add to my editorial calander, along with other blog maintenance.
Very cool. I did this manually when I converted from Blogger to WordPress. Nice to see that there are better ways to do this.
Darren, I use Broken Link Checker, a WordPress plugin which scans the blog for any broken links being pointed to. It’s an efficient tool to do the job and a few times I had to edit some parts of my blog to cover up.
One instance is, I use Linux with the help of VMWare. And there are 2 tiny, but very important files which route the commands from the Shortcut to the actual Linux ISO file. These were actually open source and it was fine when I posted about how to run linux on a windows machine. As these files were taken down from the actual server, my readers had come back to me shouting that there were dead links.
So, I had to provide those two files from the backup I had made a few months ago and they were happy. After this scenario, I came across the plugin I have mentioned and from then on, life was easier.
Great post Darren, my site offers links to everything from free ppv to mlb games I know I check all my links twice, nothing pisses a person off when they dont get whats advertised, and they dont care if it’s not my fault, tonight people want to watch the Ricky Hatton Paqciuo fight for free, the last thing they want are my links to be broken. Thanks for making a great point again Darren, If you want to save $50 go ahead and watch Hatton smash Manny, he’s the underdog but he’s gona shock the world.
I’ve read blogs with dead links to other pages on their blog. It takes very little time to run xenu, I’ve done it. I think you should probably do this once a month to keep up. Xenu makes finding dead links quick and painless. I’ve never tried the others.
I use a plugin called Link Checker that detects any broken links through automatic searches every so often and notifies me if any exist. Useful tool.
The link checker I use is ‘Broken Links’ – which you can find here:
http://w-shadow.com/blog/2007/08/05/broken-link-checker-for-wordpress/
It gives you various options (ignore, delete, fix, edit post) when it finds broken links.
It only runs when you’re actually logged into your admin screens. This has the advantage of reducing the effect on page load times (if it was set up as a Cron job) but the disadvantage that if you don’t log in regularly it may be some time before broken links are picked up.
It’s doing a great job for me and picks up new broken links very quickly – as I discovered when I deleted some of my own pages that links were going out to :)
Cheers,
Martin.
Funny… I saw this post and thought “I really need to do that”… then happened to surf to one of my most popular posts and found a bad link on that page. Duh.
http://www.craftacraft.com/sandblaster_build_your_own
This is the 1st time I have heard about the broken link checker. Will be installing it on my site so that it reduced the amount of dead links.
I have noticed that most of the people are quite careful about their own internal links, however spend little time checking their external links.
If the site you are linking to no longer exists (or the page), this will also your SEO too, just as would a broken link on your site.