Apart from the obvious answer that maybe you don't stop working on it ever, you stop working on it when it's achieved its goals. What? You don't have a goal for your website? Hmmm… so how are you going to decide what you should be working on? How will you tell whether your website is a success?

Goals are an essential part of any project. They should be set as part of the project planning process really, but at the very least you should have a goal as soon as you're up and running.

You don't have to set complicated goals for your website, you can have one goal and it can be as simple as something like a certain number of visitors per day or a specific amount of income per month.

To be honest, the goal itself doesn't matter nearly as much as having a goal in the first place, because without a clear goal you're just guessing, and that means you'll be all over the place.

Yep, I still forget to set goals, and when I do, without exception stuff just doesn't work out the way I expected it to.

Imagine you've set a goal of 100 visitors a day to your website. Once you have that, you can start doing what you need to do to achieve it. As you're thinking what to do next, you can ask yourself how it will help you reach your goal. Because your goal is clear, the answer will be clear.

Concentrate on the things that help you reach your goal and you'll find yourself spending less time on peripheral stuff and more time on the things that count. You will become more productive and you will become more successful. I guarantee it.

So, make sure you have a goal for your website. If you don't have one, set one today. Don't worry about whether your goal will be relevant next month or the month after. That's the good thing about goals – you can have as many as you like. Once you've reached a goal, you're free to set another one.

Having goals really does make all the difference to your success. I can't remember the last success I had that didn't have some sort of goal. Goals give you something specific to aim for, but they also give you much needed motivation when you reach them. Will you experience success without setting goals? I'd say it's unlikely.

What the plugin does

You probably know that when a search engine spider visits your site, one of the first things it does is look for a file called robots.txt which tells it which files and folders it can go and look at. By default, WordPress lets every robot go everywhere. That might be ok for some people, but I prefer to exercise a bit more control over things.

For example, if the robot identifies itself as a bad bot – yes, some of them do – then I don't want it to go anywhere. All it's probably going to do is trawl for email addresses to add to a spam list somewhere. And I don't really want any robots poking their noses into such places as the WordPress admin folder. Control freak? Me? Don't know what you mean…

The solution is to add the names of bad bots to your robots.txt file and disallow them from going anywhere, and add the names of common search engine spiders and specify which locations or files they are allowed to visit.

This plugin will do that in a completely hands free way by setting up a virtual robots.txt file for your blog as soon as it's activated. Whenever a request for a robots.txt file comes in, WordPress will display the contents of your virtual robots.txt file. No physical file is created on your site but one is shown to the search engine bot.

By default, your virtual robots.txt file will have Google's Mediabot allowed, a bunch of spam-bots disallowed, and a few of the standard WordPress folders and files disallowed. The default collection of bad bots is borrowed from http://www.clickability.co.uk/robotstxt.html.

Ok, even though it's completely automated and hands free I admit there are times when I want to tweak what's contained in the virtual robots.txt file. There's now a handy options page which lets you edit the contents.

Oh yeah, and if you mess up your robots.txt file you can just deactivate and reactivate the plugin and it will revert back to the default list of rules.

Also, if the plugin detects an existing sitemap.xml file (or if you are using my XML Sitemap plugin) it will add a reference to your sitemap.xml to the end of the robots.txt file. I'm told this helps with the discovery of your sitemap.xml and indexing of your pages. That's got to be a good idea.

How to use the plugin

With the plugin now being hosted on WordPress, the easiest way to install this baby is to visit your blog admin pages, click the Plugins menu, and then click the Add New menu. In the search box type something like "robots.txt" and with a bit of luck you should see PC Robots.txt in the list that appears. To the right of it you'll see a link to Install the plugin. Click that.

If you happen to be using the version that was hosted on this site, please delete it and install a new version using the instruction above. That way you'll always have the latest version and you'll get notified of updates and such by WordPress.

The official download page is at http://wordpress.org/extend/plugins/pc-robotstxt/

And please do give me a shout to let me know if it works for you or not :-)