The Sorry State of Open-Source Dropbox Clones

For starters, I use and love Dropbox. It is one of, if not THE, best file synchronization tool I have used or know of. I’m going to make the claim that no open-source Dropbox clone comes close to its functionality at the moment. But first, here is a list of the requirements for my personal file synchronization utility which Dropbox satisfies:

  1. a) It must keep a version-controlled copy of all of the files I am synchronizing. b) I should be able to easily revert to an earlier version of a file if I wish, independently of the state of the version of the rest of my files.
  2. It must work automatically and behind-the-scenes to keep my files in sync. I don’t want to have to think about synchronization or backups -that should be handled for me.
  3. File synchronization should begin as soon as I save the file locally. If I update a file I’m sharing with a friend, I want them to get the new version as soon as possible, which brings me to…
  4. Sharing files with others should be easy and intuitive.
  5. It should have an easy-to-use API which makes it easy for others to build applications around.
  6. My files must be accessible on my local machine, even if I am disconnected from the Internet.

With all of the above compliments, why would I want to replace Dropbox? Well, even Dropbox isn’t perfect, and I have a few additional requirements my ideal personal synchronization tool would meet:

  1. A high level of privacy and security. Although Dropbox claims to keep your files private and secure, there have been a few cases where that seems to be wearing thin. Many people are searching for a replacement. It seems to me that encrypting files locally before transmission (and not storing the private key anywhere except locally) is a good solution.
  2. I should be able to control where my files are stored. This probably doesn’t matter for your average user, but as a superuser, I want control. I.e., I should be able to setup my own server and store my files there if I want. This also means my available storage space is only limited by the size of my hard drive. Although related with privacy and security, I think this warrants a separate mention.
  3. The ability to share a public link with people who don’t have Dropbox accounts. This would make file sharing much easier. (A shout out to Spence for this idea)
  4. A proper access control facility. This would probably ideally mirror Unix-like ACLs.
  5. I should be able to revert all my files to a certain point in time. This is not (I don’t think) currently an option with Dropbox.
  6. Finally, it should be open source for the usual reasons.

Existing “Solutions”

Looking at the set of above requirements, it becomes clear that a) I am very picky, and b) there is no existing solution which can satisfy all of my requirements. There are a great number of open-source Dropbox clones out there. Below is a partial list of what I see as the major competitors in this space, along with their shortcomings on the above items. Feel free to leave a comment if I have missed one, or made false claims about one I have listed.

  • SparkleShare – It’s based on git. Git is fantastic, but not for all the image and music files I want to sync. You’re limited to storing your files in a git repository.
  • ownCloud – No client synchronization tool. They tell  you to use use WebDAV (see below).
  • Syncany – Doesn’t support sharing files with others.
  • duplicity – Only synchronizes when you tell it to.
  • NFS and WebDAV – No access to files when you are offline. This is a must.

What’s Next?

After searching for a synchronization client which meets my requirements and failing, I did what every self-respecting techno-geek would do, and began writing my own. I know I will catch some grief for starting yet another project and not contributing to an existing one, but I believe that the above projects all have core design limitations which will inhibit them from meeting all of the above without changing key components of their infrastructure.

I am aiming to eventually hit all of the above requirements. In the first pass, I am aiming to meet Dropbox requirements numbers 1.a., 2, 3, 6 and non-Dropbox requirements 2 and 6. These are, in my opinion, the subset of the above requirements that is in the middle of the Venn Diagram containing easy-to-implement features on the one side and important features on the other.

For the moment, I am calling my project “Asink” for both the asynchronous method of file synchronization, the fact that it aims to be a sink for all of my data, and that I should be able to store everything and the kitchen sink in it. You can follow my development on github at http://github.com/aclindsa/asink. I am planning to write another post to explain my initial design decisions, as well as an update as soon as I have a version with the basic functionality properly working (hopefully within the next week).

Feel free to let me know what you think in the comments.

Tags: , , , , ,

10 Responses to “The Sorry State of Open-Source Dropbox Clones”

  1. Tailor Vijay says:

    Is this practically working already cross-platform (using the Python script) with S3?

    Kind regards,
    Tailor Vijay

    • Aaron says:

      Tailor: The initial version *seems* to be mostly working, with a few restrictions (and I’m sure a bug or two). The server currently doesn’t handle more than one user, and you don’t have to login to the server portion of it (however, the server portion of asink only keeps the revision information based on the SHA1 hash of your files, and NOT your actual files, so no one will be able to gain access to your files this way, they can just mess with the revision information – this is a something I’m working on fixing). If you would like to try it out, you are welcome to by checking out the code at https://github.com/aclindsa/asink. Follow the instructions in the INSTALL file to install both the server and the client. Let me know if you do this and run into any problems or have any questions. Feel free to email me as well (available under the contact tab).

  2. konaya says:

    If the only problem with Syncany is no public sharing, why not simply address that problem alone? It would be a trivial matter to write a simple web application in PHP/Python/Ruby/Chooseyourpoison which with some .htaccess sorcery could generate and deal with on-the-fly links to files of your choosing. You could set an expiry time on individual links, have optional password protection, only allow certain IP ranges… The possibilities are endless.

    Hm. I need something similar to Syncany, as I do not particularly need the public sharing feature. However, it would be a nice project to write something like what I just described… Hm…

    • Aaron says:

      I did look at the Syncany code once upon a time. If I remember correctly, I found it to be not extremely clean and, worst of all, in Java. Also, to be honest, I wanted to play around with content-addressable storage and synchronization protocols myself, so my reasons against the competitors didn’t have to be bulletproof to justify creating my own pet project!

  3. Rama says:

    ownCloud forces you to use the same MySQL username as the admin username! – Regardless if you specify the actual credentials you use (it will ignore them). Terrible security. And… then the installation is still not smooth. (The installation script assumes too much – not all servers are set up the way the ownCloud devs expect – I had to do a fair bit of “fixing” and work arounds). Tried and deleted. ownCloud may be a valiant idea, but the development execution seems quite poor.

  4. John says:

    Sounds like just what the world needs. Did anything ever get started with this project?

    Thanks, John

    • Aaron says:

      John: I did quite a bit of work on it last summer, but since then I’ve gone back to school for my master’s and haven’t found much (any) time to work on it.

      It mostly works, but is still a little rough around the edges (i.e. don’t trust it for important things). It’s available as source at https://github.com/aclindsa/asink, and there’s an INSTALL document there as well if you would like to give it a try. As always, I heartily welcome contributions, and this is especially true since I have so little time at the moment.

  5. Anony says:

    Hey bro, this is promising but seemingly dead, have you long abandoned this?

    • Aaron says:

      While I did abandon the Python version of this project a while ago (partially because I got busy and partially because my design just wasn’t working well), I have recently picked it up again and am making solid progress towards a working re-written version in Go. I will be sure to post again when I get something working well enough for others to test.

Leave a Reply to Aaron