Did you know you can run remote computations from your Windows/Mac/Linux box without any special client software installed, just by dragging and dropping? And it even doesn’t matter if it’s not online all the time…
It’s a great idea from Ian Cottam at The University of Manchester, and it makes some powerful points.
The trick uses Dropbox, which is software that syncs your files across your computers. This is incredibly handy – as time goes on we all use more PCs, laptops (and indeed iPhones!) and Dropbox synchronises the contents of your Dropbox folder across all these for you. Note this is quite different from having some centralised filestore (or WebDAV drive) mounted on everything – it doesn’t need you to be online at time of use and it doesn’t need a sysadmin to set it up. Dropbox is very easy to install and incredibly easy to use – there really is no need to read a manual and the benefits are immediate. (Other synchronising software exists, but Ian prefers the simpliciity and ease of Dropbox.)
With the “Drop and Compute” model you just drag and drop your “job” into the appropriate Dropbox folder. Later Dropbox notifies you about new files and when you look you find the results. This is a totally familiar interface for file and data management. Behind the scenes, the server spotted the job – via a simple monitoring script – and did its thing. To find out all the details of how Ian makes this work with Condor job submission, check out the Drop and Compute Wiki page for instructions and a video.
Couldn’t we have done this before? Yes, but nothing like this easily for all concerned. If we say to researchers “mount this network drive and put all your files there on every machine you use” then we are creating an extra burden and perhaps worsening a version control problem, and obliging them to be online to use it. However, if we say “you can use Dropbox to keep your files in sync between all your machines – and your iPhone too” then the user has an immediate benefit at next to no extra work. Dropbox is a solution that makes things simpler while other solutions make things more complicated – this is the only acceptable direction!
I think there’s some interesting psychology involved here too. Asking people to put their files somewhere central stops them feeling they are their personal files any longer, whereas syncing them across personal machines keeps them close and personal. Of course, behind the scenes, Dropbox is indeed putting them somewhere central, but that’s an implementation detail (and has the benefit you can also manage them from a Web browser). Fundamentally, the user model is empowering rather than disempowering.
Is there a catch? Just one little issue at the moment – if Dropbox puts its data in a different territory then it may be subject to different laws, the best known example being the Patriot Act in the US.
The Condor example has exercised the model well, and in principle this approach could be used for any remote processing. But best of all it’s an example of understanding that ease of use really matters: above all, it’s a solution which actually makes peoples lives simpler.