Monday, June 18, 2007

Link Rot

The link rot in the community color family of directories had gotten out of hand. I've performed several link purges of the populare pages on the sites. I decided, however, that I needed to do a systematic purge of the site.

So, for my weekend project, I wrote a program that allows me to look through every link in the directory and delete the rot. This project will take several days. The Old Links page summarizes where I am on this project.

Currently there are 17,044 active links in the directory. My records show that I have deleted 2,731 links. It looks like 13% of the sites listed in the directory have met their untimely (in may cases timely) end.

My records are very inaccurate. I deleted several thousand links before it dawned on me that I should keep a record of link deletions. The program I use to add and remove affiliate links deletes things with recording the event.

My goal in this project is to check every link in the directory that is more than a year old. I am checking from oldest to newest. So far, I have checked 802 links. I've deleted 124 of the checked links. Ouch, 15% of the links checked were in need of deletion.

I suspect that this percentage will drop as I move along with the project. Regardless. There were far too many broken links in the directory for my taste.

BTW, if your site was inaccessible at the time I checked it. There is a good chance that I would have deleted it. You can go to the add link page. Enter your domain in the "Check Domain Box" and press the "Check Domain" box. This will show if your site is still in the directory. If I deleted it, you can always re-add it.

2 comments:

Lars said...

I have a solution so that you only have to do your link purge one last time.
Hi,

I got tired of broken links on my websites. And I got tired of finding link rot everywhere on the web. So I created a free website for archiving links. This service is designed to prevent link-rot. In other words you will be able to create permanent links to other websites that will not change or go away once you create them.

http://www.stayboystay.com


This should help prevent this problem in the future.


Lars

y-intercept said...

The archiving idea works great for research. A community directory needs to show what is currently active. Archiving a page for a theatre company that went kaput simply creates market confusion for the world.

What I wish to do now is simply create a process where I check 1/12 of the links in the system each month. That way I will capture the dead links within a reasonable (average 6 month) time frame.