How to do dirt-cheap, cloud-based, encrypted backups (Part 1)

I've been dabbling on code with the final aim of building a peer-to-peer distributed filesystem for a while now and I seem to keep hitting diversions along the way. In an internal monologue somewhat along the lines of "What would MacGyver do?", the idea for this concoction of open-source software and cloud services was born. In about 20 minutes or so of messing about (assuming you're comfortable with linux), I'll explain how you too can sleep sounder at night for $0.125/GB/month.

The basic idea is to store an encrypted copy of your files on Amazon S3 and use lsync (which uses the Linux inotify feature) to automatically push changes to your local files to "the cloud" as soon as you make them.

If you're not fussed with encryption, you can certainly skip the EncFS step below but given I don't trust cloud storage providers not to snoop on my files, I would encourage others to go the extra yard and run with EncFS too.

Now assuming you're running ubuntu, the easiest way to get things off the ground is:

1. Sign up for Amazon AWS. It takes a few minutes for the S3 setup process to complete so do this first if you haven't already.
2. Install all the apt-based software you'll need:
   $ sudo apt-get install build-essential encfs libfuse-dev lsyncd fuse-utils libcurl4-openssl-dev libxml2-dev mime-support 
   
3. Download, build and install s3fs:
   $ wget http://s3fs.googlecode.com/files/s3fs-1.61.tar.gz
   $ cd s3fs-1.61
   $ ./configure && make && sudo make install
4. By now you should have your Amazon Access Key. Create ~/.password-s3fs as follows:
   $ echo ACCESSKEY:SECRETKEY > ~/.password-s3fs
   $ chmod 600 ~/.passwd-s3fs
5. Head over to the AWS Console and create an S3 bucket for yourself then run:
   $ sudo mkdir /mnt/s3fs && sudo s3fs mybucket /mnt/s3fs -ouse_cache=/tmp,passwd_file=~/.password-s3fs
6. Create a new encrypted filesystem using s3fs as the storage point:
   $ sudo mkdir /mnt/encfs && sudo encfs /mnt/s3fs /mnt/encfs
   <p>
   New Encfs Password:
   Verify Encfs Password: 
7. Almost there. Now just set up the directory you want synced:
   $ mkdir ~/secure_files && lsyncd -rsync ~/secure_files /mnt/encfs
8. You're done! Just start dumping your files into ~/secure_files and they'll be encrypted and uploaded to the cloud. 

For 10 GB/year I expect I will pay about $20USD. Obviously your mileage will vary depending on how active you are in your backup directory and how much data you have.

Automating this shebang is clearly something that you'll want to do but I'll leave that as an exercise for the reader. If you come up with any clever short cuts or additions to this, I'd love to hear about them.