Using Online Storage In A Raid Format

[thaimite]

There are many companies offering on-line storage such as Wuala (my favourite) DropBox and others, and being a little bit paranoid and totally crazy I was wondering if it is possible in Linux (Ubuntu) to configure multiple online storage providers to act as a single storage in much the same way as multiple hard drives form a RAID array such as RAID 5 or similar.

The advantages of this would be

1. Having the data spread over the cloud in chunks it is more secure from prying eyes

2. If a storage provider is taken down (remember. Megaupload) my data is still recoverable

3. All my storage is available as a single chunk.

I know with conventional RAID there are limitations regarding using the same size disks but I wonder is this feasible or what other issues could be out there to trap the unwary?

Edit

Further thoughts. Create multiple virtual drives, configure them as a RAID5 and then sync each one individually with a different cloud provider. Is this a way.forward

Edited April 8, 2012 by thaimite

[matt111]

Create multiple virtual drives, configure them as a RAID5 and then sync each one individually with a different cloud provide

Not sure that will work as the OS see it as a single partition. I cant see Dropbox losing your data, but I wouldnt consider Dropbox ultra secure, given the massive fck up they made with any password works.

Something like JungleDisk tied to Amazon S3 will give you very high reliability, you can check the stats out but its 99.99999% (or something like that) over a given a year. mean it would be 1 in a million or ten that S3 fails.

Personally I've add a 50gb EBS to my free Linux EC2 node (costs about $6pm with the add-on) and use rsync nightly.

Edited April 9, 2012 by matt111

[thaimite]

Create multiple virtual drives, configure them as a RAID5 and then sync each one individually with a different cloud provide

Not sure that will work as the OS see it as a single partition. I cant see Dropbox losing your data, but I wouldnt consider Dropbox ultra secure, given the massive fck up they made with any password works.

Something like JungleDisk tied to Amazon S3 will give you very high reliability, you can check the stats out but its 99.99999% (or something like that) over a given a year. mean it would be 1 in a million or ten that S3 fails.

Personally I've add a 50gb EBS to my free Linux EC2 node (costs about $6pm with the add-on) and use rsync nightly.

hanks Matt.

I agree that most online storage is very reliable, but there is always the chance of it going off line due to circumstances beyond their control rather than failure of their own equipment.

To be honest I use multiple clouds for different things and have a good set of backups at home for the really importnat stuff, so this is more of a theoretical exercise to see if it could be done easily

[mowgus]

interesting concept...Someone should develop software to do just that. You couldn't raid across various providers natively though because you don't have access to the drives on a bit level...only on a file level. Someone could develop software to store chunks of files on various cloud based services (say, in 4K chunks) using the Raid 5 - 1 parity chunk concept. It might be painfully slow though.

[Crushdepth]

Practically speaking, you'd be better off setting up your own backup drive with versioning. Have a google for rsnapshot. Not sure if public cloud providers will let you have that kind of access, but I use it to backup multiple websites and file servers. Brilliant, lets you step back in time to recover different versions of the same file.