Mirroring Local Directories to Google Drive

Since Google Drive has significantly lowered its pricing scheme, earlier this year (), it has become an appealing medium for mirroring/backing up even large folders.

The Google Drive web-based interface is rich and mature, but has some annoying limitations when it comes to the directories upload function. Without loss of generality let’s describe a situation that illustrates the lack of adequacy of the web-based interface for performing what I believe are popular operations. When access via Chrome, the interface proposes an option to upload folders (should you, for some obscure reasons, not be able to use Chrome, there exists other ways—see the official documentation on this matter, Upload files and folders). There is, nevertheless, no easy way to incrementally update (or overwrite) an already uploaded folder.

Let’s assume we want to backup both our Music¹ directory (i.e., our iTunes library) and our Pictures directory into Drive. Both directories contain plenty of files arranged in many subfolders. We prefer not move those directories into the Google Drive Folder (i.e., the directory that is synchronized), but instead create a “Backups” directory in Drive that we do not synchronize ().

Why not move those directories into the Google Drive Folder? Well, the post Finally an Affordable Cloud-Based Storage Solution provides some elements of response: we may not want to synchronize! Besides, those folders are used by other applications that might make some assumptions about their default location. And even assuming that those applications offers the possibility to configure the path of the folders they require (which, presumably, most of applications have), we may prefer not to mess up the default configuration (which should remain consistent over different machines and between updates, thus conserving it unchanged might be a wise decision). Another valid reason is that the sync function might come with some defects that prevent it from properly handling a massive quantity of files. For instance, a few months ago, on my mac mini, the Google Drive application was not able to synchronize its dedicated folder after I added a subfolder, which, I must admit, sort of dwarfed the original contents in terms of both number of files and size… The sync function did not like it, it was seemingly indefinitely scanning the web… That appears to be a known issue. As a result, I needed to erase the folder, to remove the Google application, to reinstall it and finally to re-sync the local folder from Drive; consequently, the subfolder I wanted to upload could not be uploaded that way… Plus, needless to mention, I was rather pissed off.

Let’s refocus on our goal consisting in remotely mirroring the two substantial folders Music and Pictures to Drive. The first time, the web-based interface (via Chrome) appears to do the job; the two folders can easily be remotely duplicated (only patience is required). After a while, when several new music tracks, movies and TV series have been purchased and many new pictures taken, the local and the remote folders start to substantially differ. We are, accordingly, interested in updating the remote folders in Drive from their local counterpart… Here is the problem, because we only want to upload the new files, and to update the files that have been locally modified (e.g., iTunes’s setting files), and what we certainly do not want is to upload the entire very large folders in Drive all over again. Unfortunately, the web-based interface does not provide any reasonable means of doing that.

There is indeed a way to manage files versions (), but it is not designed for dealing with an enormous amount of files distributed through a wide and deep tree structure (for you must operate on each file individually)…

In Drive, the directories’ and files’ name are not used as a unique key, and therefore two directories or files with the same name can reside contiguously in the tree² structure (they can be siblings)… Therefore, when one attempts to upload again a local folder with the intention of overwriting the existing outdated remote version, she/he ends up getting two different folders that have the same name. Such an approach has many advantages, though (e.g., it is very efficient to move files elsewhere in the logical structure, which is not as easy in Swift storage, for example - ).

With the purpose of filling this gap, I have recently started to look at the Drive API. To be continued…