For this I recommend a program that's designed specifically for email backup: MailStore Home. It's free, and it works with all the aforementioned clients, plus several others. The only thing you need is a flash drive or external hard drive to store the backup.
Backing up everything else for free
The cloud has its merits, but any good backup solution should include a local component, just in case you lose Internet access (or don't have a fast connection to begin with). You can use any number of free backup utilities to preserve your important data on an external hard drive, but those drives aren't free.
Instead, consider creating a "backup network" that leverages the other PCs in your house, syncing important files between them. All you need is Cubby, a free tool from LogMeIn that automatically syncs unlimited folders between two or more PCs. (You also get 5GB of free cloud storage that you can use for other stuff.)
This is a great way to sync, say, photo and music libraries between your PC and your spouse's, creating a backup of both in the process. You can even sync with a trusted friend or far-flung relative, thereby establishing an off-site backup to protect against local disasters like theft and fire.
The upshot of all this is that by focusing on your data by type instead of lumping it all together, then directing it to different places, you can keep everything safe and sound--all for little or no money. I've been using this approach for years, and so far it's been flawless.
Sign up for CIO Asia eNewsletters.