Written on May 2nd, 2009 at 08:05 pm by Darren Rowse

Hunt for Dead Links [Day 27: 31DBBB]

Today your task in the 31 Days to Build a Better Blog Project is something that most bloggers who have been blogging for a while could probably benefit from doing – go on a dead link hunt.

Blogging is built on the ‘link’. One blog links to another blog who links to another who makes comment on another. This is a wonderful thing – but what happens when one of the blogs that you’re linking to is retired, is deleted, changes it’s link structure, moves etc?

The link is a dead one (also known as Link Rot) and can cost your blog on two fronts:

1. Readability – clicking on a dead link can mean your readers can end up on error pages or being redirected to other irrelevant content to the one they were expected to get to. This can lead to reader frustration or giving the impression that your blog is old and/or out of touch.

2. SEO – I’m not sure of the technicalities of it or what the latest research shows but from what I can tell a dead link is not looked upon favorably by search engines and you run the risk of penalties.

So how do you detect dead links on your blog?

The most obvious ’solution’ is to surf every page on your blog and manually check all the links. This is something that might be achievable on a new blog – but on older blogs with hundreds or thousands of posts it’s just not feasible.

There are many link checking tools available but to be honest I’m yet to find one that I’m really happy with. I do hear that Xenu’s Link Sleuth is well regarded. I’ve also used the free version of Dead-Links.com (which only checks to a reasonably shallow depth) – but I’d be keen to hear from readers on their suggestions of other options.

What to Do When You find a Dead Link?

There are a few options for what to do with dead links. They include:

fixing/updating them – if the link is simply wrong or pointing to the wrong place update it so that it works
deleting them – if they are dead and you can’t find a correct one then you can delete the link. I usually add an ‘update’ note to say that I’ve done this. I sometimes also update with new relevant links so that the post is still relevant.
delete the post – on occasion I’ve done this if the whole post’s main point is to link to someone else’s post. A dead link makes this type of post obsolete so I consider deleting them rather than updating.

Whether you use a tool or just tackle the task manually a few posts at a time – finding and fixing dead links can be well worth the effort.

What do you Do

How do you find dead links? What do you do when you find them? Got any cool tools to share? Feel free to discuss here, or share with everyone in the forum post for this task.

Want More?

This task is a sample of one of the tasks in the 31 Days to Build a Better Blog Workbook – a downloadable resource designed to reinvigorate and revitalize blogs.

Join over 14,000 other bloggers and Get your Copy Today.

What next?

74 Responses to “Hunt for Dead Links [Day 27: 31DBBB]” - Add Yours

HazardousPaste

May 2nd, 2009 8:30 pm

One of the more annoying things involving dead links are when you link to images that get taken down. As a preventative measure, I always keep a local copy to put in its place as a precaution and email the original owner of the image (if it’s from a private site) to ask if I can use it with a link back to their blog instead of directly to the image.
mohan

May 2nd, 2009 8:32 pm

Again Dareen a very usefull tips. It help us . I will do this .
Thanks for the post
Fyre Vortex

May 2nd, 2009 8:58 pm

Great post, Darren. I have been wondering about this for a while (Years, maybe?) and have finally found the solution. Thank you! :D
venkat

May 2nd, 2009 8:59 pm

That’s very annoying when readers clicks on link taht go no where,I have to found tools to find all the dead links.I will try the tools you mentioned also.
Sire

May 2nd, 2009 9:02 pm

I’ve never really bothered with dead links before. There have been a couple of occasions where I came across them accidentally and having found them I fix the problem, but to actively search for them, man that would take way to much time. I think I would rather just put up with a bit of link rot. That or pay my readers a dollar for every dead link they find. Surely there couldn’t be that many. Now there is an idea for a good competition ;)
Sheryl Loch

May 2nd, 2009 9:04 pm

I just did this 3 days ago!

Only thing I found odd is that some of the dead links were in the comments area. I have been taking out those dead links and leaving the comment.

I also found that some of my NoFollow links were shown as dead but, they were alive and well when I hand checked them.

My challenge for today is done, guess that means I have time to watch the horse race & the Nascar race.
Sayz

May 2nd, 2009 9:08 pm

If you’re using Wordpress,
then try this plugin
it can save you a lot lot of time from checking the link
wordpress.org/extend/plugins/broken-link-checker/

it’s simple and it’s easy…
we can update it directly from control panel…
if you got the better plugin, please let me know…
Milton Ramirez

May 2nd, 2009 9:08 pm

That’s the though part. I didn’t even know that there was software to such entrepreneur. Having a blog with over a 1000 posts it’s hard to go and check one-by-one. I will keep my eyes peeled to see if someone comes up with a better solution.

I am enjoying the series. Thank you!
Samuel

May 2nd, 2009 9:33 pm

It’s actually quite easy (if you have strong programming knowledge of javascript or php) to create universal code that notifies or redirects dead links automatically.

For the less tech-savvy of you, you can keep a notebook of any changes you make to your blog and follow them right through your whole structure.

Even with a small blog, going through every page checking links is a rather monotonous task!

Cheers,

Sam
Sue

May 2nd, 2009 9:40 pm

HazardousPaste: that’s not annoying, that’s bandwidth theft.
`rolandinsh`

May 2nd, 2009 9:43 pm

On my webpage (not blog) I made small, but useful script which e-mail me on every 404 and 403 entry. So when someone is coming to my non-existing page, they get nice styled error message with links to homepage, blog or microblog.

When someone is requesting non-existing image it simply display image with text, that image is removed.
Example; http://rolandinsh.lv/aefqwreqwsdfygsdhebqljahedv.png (image does not exist!)

On first few days I’ve got hundred e-mails with error messages, but after fixing HTACCESS file with dead links, now i’m getting maybe 1 e-mail per day.

About linking out… well on that I’m working now.
Nathalie Hamidi

May 2nd, 2009 9:47 pm

I use broken links checker http://w-shadow.com/blog/2007/08/05/broken-link-checker-for-wordpress/ which is pretty useful because it checks old posts on a regular basis and new posts too, so when I mistype a link it doesn’t impact all my readers because I have a message about a dead link in my dashboard just after publication. It gives enough time to correct it before it goes public.
Money4Invest

May 2nd, 2009 9:52 pm

I used Free Link Checker v4.3 to perform broken link check. It will give a broken link report after validation. It’s a simple but powerful free broken link tool.
Viviana

May 2nd, 2009 9:57 pm

Great idea – I have not thought about dead links since my blog is only about a year old. No dead links – hurrah!
MJ Doyle

May 2nd, 2009 10:16 pm

I use “broken link checker” also and I find it works really well. It has drawn my attention to a few dead links that I then just fixed up.
Bruce Keener

May 2nd, 2009 10:27 pm

If you decide to delete a link, you should also and a Redirect 410 /link/ to your .htaccess file so the search engines will stop looking for it. Google respects this, and will take the link out of their index. I am less sure about MSN and Yahoo!

Same applies to deleted categories and tags: do a 410 redirect.

Without the 410, you can wind up with a bunch of 404 errors that could, depending on the number, impact your quality score with the search engines.

I use the WordPress Broken Link Checker. It seems to do a good job. Runs in the background, and after a few days seems to go all the way through the site, picking up dead links.
ezFreeSamples.com

May 2nd, 2009 10:50 pm

Dead Links are a killer, for sure. I know when I am checking new sites out and I find a dead link or two, I usually blacklist the site and do not visit again.

This is another helpful article Darren. With Free Samples, they run out of stock sometimes and this creates problems with my older links often as they will lead to offers that are no longer going… but that should only be the ones that are a few weeks old and so on. I will have to check this out on my http://www.ezfreesamples.com blog. Thanks.
N!cklas

May 2nd, 2009 11:03 pm

Xenu worked great for me – great tips!
Will also check the WP-plugin
Steven-Sanders

May 2nd, 2009 11:31 pm

Wordpress has a plugin called bad link checker that works really well for this.
Clay

May 2nd, 2009 11:34 pm

Good idea, Darren. And thanks, Sayz. I installed the Wordpress plugin. Where on the Dashboard will it show any broken links?
delicate flower

May 2nd, 2009 11:39 pm

I’m new enough to be able to do this manually… over a few days.
I found an incorrect link on a friend’s blog and let her know.. thought i should share that with her, I did it through email rather than a public comment.. figured she’d want to correct that as well…
So, as a courtesy I think it’s a good idea to do that as well.

I’m also , going back to an earlier lesson, doing a lot more linking of old posts to new ones.
BlogDirectory

May 2nd, 2009 11:41 pm

A once in a while i will check for dead links but most of the time i don’t because it can be time consuming and boring to check every link on your site. I didn’t know they had tool that could help you with this. I will try some of them out.
Guillermo

May 2nd, 2009 11:45 pm

I’ve used Google Webmaster for sometime but found it annoying soemtimes and not very helpful. I’m trying Xenu, the one recommended by Darren. let’s see how it goes.
Rita

May 3rd, 2009 12:00 am

Thanks for the ideas on programs that might find dead links. I’ve wondered how to remove dead links, but the thought of looking through nearly 1,000 posts on three blogs seemed daunting.

Rita, The Survive and Thrive Boomer Guide
Vicki@collegeparentcentral

May 3rd, 2009 12:02 am

Although I understand that a tool might be necessary for a larger blog with many, many posts, one additional advantage to hand checking for links is spending some time rereading older posts. It’s a way to look with fresh eyes at things written a long time ago – which might prompt either ideas for new posts or the need to rework the older posts.

It seems as though this day’s task could relate to the previous task to help another blogger. If I ever find a dead link on a blog of someone I read regularly, I’ll e-mail to let that person know. We may be more apt to find the dead links on someone else’s blog. We all help each other.

Thanks for the reminder that we all need to keep working to keep things fresh.
Catherine Franz

May 3rd, 2009 12:03 am

I’ve always wondered about this. Thanks for sharing how.

I used Dead-Links.com, and the errors are still coming up after 35 minutes (blog has been up since 2002 with at least one post a day. There’s a huge list appearing. Like I said it stills going after 35 minutes.

What is (? query string)?

I’m getting these two lines — a link and then X visited – X in the host – X out.

The X visited number keeps growing? Do I start at the first one and work through them.
Wow, the list is so long. I’ll be here for a year cleaning this up. Ouch.

I use TypePad. Does anyone know of a plug-in that works with it I can use on a weekly basis to keep it clean?
ITrush

May 3rd, 2009 12:14 am

Thanks for the links Darren, I’m sure it’ll help a lot when I’m going to start digging dead links. Another great lesson here.
Gayle

May 3rd, 2009 12:26 am

Time to put on my camouflage and go hunting for dead links. This is another great idea that I had overlooked. Thanks for sharing.
Sire

May 3rd, 2009 12:29 am

@ Sayz – Thanks heaps mate; I used that plugin you recommended and it worked a charm. I was able to remove 12 dead links.
Java Bean Rush

May 3rd, 2009 1:30 am

I had no idea that there were websites for tracking dead links. I usually just check them manually. Those sites should prove to be time savers.
alone

May 3rd, 2009 1:45 am

That’s right. Dead links will cost your site. I’d tried dead-link.scom and it’s works =D
Bill Masson

May 3rd, 2009 1:54 am

Never bothered with dead links, takes too much time! But will try out the WP plugin thanks Sayz
Patrick

May 3rd, 2009 2:51 am

I’ve used Xenu’s Link Sleuth. It’s an open source download. It takes awhile to scan your site, but once it does, you have a list of every broken link.

I didn’t do this for almost the first two years of my site and found literally hundreds of dead and broken links – news sources that were no longer there, blogs that closed up shop, etc. A common source of dead links was in the comments section from people linking to their sites which are no longer live, or people who left their site as “yahoo” or “google.” I delete those now as soon as I see them, but it never occurred to me for the first few months of blogging.

Budget several hours to clean up dead links if you have a lot of content and have never done it before. You are likely to have a lot of dead links. I worked on mine for half hour at a time over the course of a few days. Now I just run Xenu every other month or so and I only have to do a couple corrections.

I highly recommend cleaning your dead links, regardless of which plugin or software program you use.
infmom

May 3rd, 2009 2:51 am

I found that the carefully crafted index to important information on my site returned a 404 every time I tried to look for it (still don’t know why that happened). And I’d linked to it several times in subsequent messages.

I was able to get to it via WordPress’s admin-edit menu. Then I had to copy-paste the whole contents into another message, change the post date to match the original, and go change all the messages that linked to it. Royal pain, but at least now it’s visible if someone clicks a link.
Daniel Pereira

May 3rd, 2009 3:30 am

I used http://www.bad-neighborhood.com/text-link-tool.htm some time ago to verify my dead links, and I found it pretty cool.. :)
Make Money Online

May 3rd, 2009 4:25 am

I actually never looked at dead links but its good to have 301 redirects.
Ark Lady

May 3rd, 2009 4:31 am

When you construct a sitemap via Google Webmaster tools the report generates that identifies broken links site wide. (Find this under diagnostics.)

It gives you are report with a variety of different errors but if there is a better option I would love it!
Dr. Mike Wendell

May 3rd, 2009 4:48 am

Like a few others up above, we rely on Google Webmasters tools and a sitemap. We do it a bit differently though as the tool we use to generate that sitemap also tells us what’s broken. We use the sitemap generator at auditmypc.com s that gives you the broken link report at the same time as a sitemap. I;ve recommended it previously and find it a great tool.
grassosalvato86

May 3rd, 2009 4:51 am

Hello!
Thanks for the information! I’ve discovered just two dead links in a post of mine. I am going to update it with new links.

Salvatore
Ben Licher

May 3rd, 2009 4:58 am

I use ‘Broken Link Checker’: http://w-shadow.com/blog/2007/08/05/broken-link-checker-for-wordpress/ from Janis Elsts in my WordPress blogs. Very effective, shows broken links automatically in your Dashboard and you can edit and change them in the overview.
I have 9 broken links to edit now.
Clara Mathews

May 3rd, 2009 5:14 am

I have not done this previously, but I will try the pilugin suggested by Sayz. This is a task that I can add to my editorial calander, along with other blog maintenance.
Kosmo @ The Casual Observer

May 3rd, 2009 6:19 am

Very cool. I did this manually when I converted from Blogger to Wordpress. Nice to see that there are better ways to do this.
Team Nirvana

May 3rd, 2009 6:36 am

Darren, I use Broken Link Checker, a Wordpress plugin which scans the blog for any broken links being pointed to. It’s an efficient tool to do the job and a few times I had to edit some parts of my blog to cover up.

One instance is, I use Linux with the help of VMWare. And there are 2 tiny, but very important files which route the commands from the Shortcut to the actual Linux ISO file. These were actually open source and it was fine when I posted about how to run linux on a windows machine. As these files were taken down from the actual server, my readers had come back to me shouting that there were dead links.

So, I had to provide those two files from the backup I had made a few months ago and they were happy. After this scenario, I came across the plugin I have mentioned and from then on, life was easier.
FREEHATTONFIGHT

May 3rd, 2009 7:55 am

Great post Darren, my site offers links to everything from free ppv to mlb games I know I check all my links twice, nothing pisses a person off when they dont get whats advertised, and they dont care if it’s not my fault, tonight people want to watch the Ricky Hatton Paqciuo fight for free, the last thing they want are my links to be broken. Thanks for making a great point again Darren, If you want to save $50 go ahead and watch Hatton smash Manny, he’s the underdog but he’s gona shock the world.
Laura Simmons

May 3rd, 2009 9:13 am

I’ve read blogs with dead links to other pages on their blog. It takes very little time to run xenu, I’ve done it. I think you should probably do this once a month to keep up. Xenu makes finding dead links quick and painless. I’ve never tried the others.
Robby G

May 3rd, 2009 9:43 am

I use a plugin called Link Checker that detects any broken links through automatic searches every so often and notifies me if any exist. Useful tool.
Martin Malden

May 3rd, 2009 9:45 am

The link checker I use is ‘Broken Links’ – which you can find here:

http://w-shadow.com/blog/2007/08/05/broken-link-checker-for-wordpress/

It gives you various options (ignore, delete, fix, edit post) when it finds broken links.

It only runs when you’re actually logged into your admin screens. This has the advantage of reducing the effect on page load times (if it was set up as a Cron job) but the disadvantage that if you don’t log in regularly it may be some time before broken links are picked up.

It’s doing a great job for me and picks up new broken links very quickly – as I discovered when I deleted some of my own pages that links were going out to :)

Cheers,

Martin.
Bruce Dillahunty

May 3rd, 2009 10:13 am

Funny… I saw this post and thought “I really need to do that”… then happened to surf to one of my most popular posts and found a bad link on that page. Duh.

http://www.craftacraft.com/sandblaster_build_your_own
Dominique

May 3rd, 2009 10:38 am

This is the 1st time I have heard about the broken link checker. Will be installing it on my site so that it reduced the amount of dead links.
Nicolas Prudhon

May 3rd, 2009 11:48 am

I have noticed that most of the people are quite careful about their own internal links, however spend little time checking their external links.

If the site you are linking to no longer exists (or the page), this will also your SEO too, just as would a broken link on your site.
Dee Wilcox

May 3rd, 2009 12:14 pm

Thanks to all for the recommendations regarding the Broken Link Checker Wordpress plugin. I installed it this evening, and it immediately generated three very old posts with broken links. I likely never would have found them unless someone had pointed them out to me. This is a great tip, and the plugin is a very helpful tool. Thanks!!
sulumits retsambew

May 3rd, 2009 1:03 pm

I have checked my blog’s links using dead-links com, thanks for the info.
TechThinker

May 3rd, 2009 1:47 pm

Thesis is a great theme. I have seen a lot of good looking and functional blogs developed on Thesis.

For those who do no want to invest right away on a theme, try Vigilance. Its a great theme that looks similar to Thesis. The theme is free. But in order to make use of the advanced features you have to join the Vigilance members forum to get help and support (for a fee). But if your have good coding skills you can tweak the theme by yourself.

I just switched my blog to the vigilance theme. I love it!
Work From Home Opportunities

May 3rd, 2009 2:23 pm

I use broken link checker plugin on my blogs, works great and displays on the dashboard.
Dominique

May 3rd, 2009 4:13 pm

Just installed the WP dead link plugin one commentor suggested..yikes 61 dead links to clear.. off to do some housekeeping.
L.J.T.

May 3rd, 2009 5:06 pm

I installed the suggested wp plugin from another reader and it looks to be working well.
Lee Ka Hoong

May 3rd, 2009 11:31 pm

I’ve used W3C and dead-links before, but I found W3C link checker is better than dead-links. I think Google hates blog which has lot dead links because we share something unavailable, right? :)
Thanks for sharing other dead links checker tool Darren!

Regards,
Lee
Brad Wiederholt

May 4th, 2009 12:32 am

I just tried this with 4 blogs using the dead-links.com tool. As Darren expected there were dead links on 2 of the 4. Now they are nice and tidy.

I also started using Google Webmaster tools for my latest blog. It seems to be doing a pretty good job at checking link mismatches as well, though it is not as immediate as these tools above.
titan

May 4th, 2009 2:49 am

huhu. need to work hard to get better blog. Seo and hunt for dead links another my project too. Hope it will success.
spocrep

May 4th, 2009 3:21 am

thanks Darren for the post :) but I have a question, If I updated the post many times to update the links does that make search engine thinks of my post as spam or something ?
I would love for you answer:) thanks a lot
Jasmino924

May 4th, 2009 4:39 am

Thanks for telling us about checking for dead links. I don’t know about everyone else but I know that sure as heck wouldn’t have though about checking for dead lins in a million years!
Albert

May 4th, 2009 6:05 am

I’ve learned quite a bit through out this series, not only from the author, but from many of the folks posting comments. It has been a great learning experience!

Regards,
Albert
The Rasch Outdoor Chronicles.
The Range Reviews: Tactical.
David Eedle

May 4th, 2009 10:45 am

Darren

A great post, especially as it highlights the housekeeping component of operating a successful content based web site. Dead outbound links are a pain and traffic negative. Plus of course ensuring links within your own site are still valid. Housekeeping ain’t glamorous, but it’s critical to long term success.

One of our websites used to link to a very significant quantity of news websites, who of course have a terrible tendency of moving pages around without using proper redirects. Easy example is when they move something into their paid archive after 30, 60 or 90 days of free view. Used to drive us made trying to keep a couple of hundred thousand links up to date.

I’ve tried a few of the link services but never really been overwhelmed. I fall back on the tried and true Google Webmaster Tools for internal links, and have written a few utilities to check outbound links.

Would always suggest if at all possible people have a custom 404 page, to ensure if you inadvertently trash an internal link your visitor still sees something valid instead of the bog standard ‘can’t find that page’ message.

Cheers…David

http://www.NicheContentMillionaire.com
John

May 4th, 2009 7:53 pm

Google webmaster tools will help you. It will let you know if you have any deadlinks for any of your sites.
cara membuat blog

May 5th, 2009 12:32 am

I never think about finding dead link in my blog but, after i read your article i will try it, because maybe dead link will make our site bad in visitor eyes.. thanks.
Jason of Repairmycreditrating

May 5th, 2009 5:13 am

Thanks for that guideline. Will check my websites and blogs right after this to check on dead links.
elizof

May 5th, 2009 9:07 pm

Thanks for the suggestion… I don’t think I have any dead links but will be happy to hear from anyone who finds one on my blog… I will check again when I have a moment. Great tip too Darren!
العاب شمس

May 6th, 2009 9:47 pm

Thanks for link Dareen, I’m waiting your newest post.
Sam sall

May 8th, 2009 1:13 am

Yay me . no dead links found in my post or blog . i used to do frequently and i geuss i did it well :)
Unionhawk

May 21st, 2009 4:09 am

dead-links.com is full of crap… It said that pretty much every link on my blog was a 404 (most of the rest recognized it as a web spider, and blocked it)

Examining these links, they were not dead at all.
Unionhawk

May 21st, 2009 11:04 am

The first, downloadable option worked fantastically. Not quite as easy to use as Wikipedia toolserver’s Checklinks, which will let you automatically tag or remove dead links, but it still works.
Diane Schubach

May 28th, 2009 10:12 am

I check manually for dead links, but what I did find recently was that my e-mail address was mis-typed and I had been cutting and pasting the link for months. Ahhhhhhhrg.
N!cklas

May 28th, 2009 6:37 pm

@diane
If uou’re using mySQL as database you might find this handy:

UPDATE tablename SET tablefield = REPLACE(tablefield,”findstring”,”replacestring”);

Just set all low-case to suit your settings and needs
Albert

May 29th, 2009 7:45 pm

This is quite a conundrum.
You need to link to others in order to be relevant and achieve better Search Engine rankings, but then you are at the mercy of the linked site staying alive.
The fact of the matter is many blogs which are not still born die at a rather youthful age… Links die
So yes this is good advice… Thankfully my blog is new so I can do it manually, but later… I’ll cross that bridge when I come to it.

Comments will be closed off on this post 90 days after it is published. Apologies to those this impacts but it's a regrettable and temporary measure to combat a growing comment spam problem. See our most recent posts where you can comment here.

Written on May 2nd, 2009 at 08:05 pm by Darren Rowse

Hunt for Dead Links [Day 27: 31DBBB]