Previously we have described how to encrypt local files and upload them to Amazon. Before that we also demonstrated layering UnionFS-FUSE over the Amazon mount in order to:

  • Cache new files.
  • Preserve consistency of directory structure

Although we mentioned about keeping the directory structure consistent between the local files and the files already uploaded to Amazon Cloud Drive, we can make this less of an issue by simply writing over the directory structure which already exists on Amazon Cloud Drive and completing our filesystem layout described in the research post.

Filesystem layout

Completing this picture is simple, and I’d like to dig into how to achieve this in a bit more detail.

About UnionFS

Unionfs is a filesystem service for Linux, FreeBSD and NetBSD which implements a union mount for other file systems. It allows files and directories of separate file systems, known as branches, to be transparently overlaid, forming a single coherent file system. Contents of directories which have the same path within the merged branches will be seen together in a single merged directory, within the new, virtual filesystem.

Source Wikipedia.

Mounting your UnionFS-FUSE Filesystem

Luckily this is an extremely simple step of our final setup. Before proceeding, ensure you have UnionFS-FUSE installed and available and also ensure you have your local and remote encrypted mounts as described in my previous post.

unionfs-fuse -o cow /home/plex/local-sorted=RW:/home/plex/acd-sorted=RO /home/plex/sorted/

Lets break this simple line down:

  • unionfs-fuse - Self explanatory!
  • -o cow - Enables CoW (Copy on Write). This means that new files written will be written to your read-write layer, if you want to alter a file on the Read-only layer it will be copied over and written (although we do not use this feature).
  • /home/plex/local-sorted=RW - The first united layer. =RW means this layer is writeable.
  • : - Each subsequent layer is separated with a colon.
  • /home/plex/acd-sorted=RO - The second united layer. =RO means the layer is read only.
  • /home/plex/sorted/ - The UnionFS-FUSE mountpoint. This is the location where the combined files will be displayed, and where you should write to.

Note: If you were to over-write a file on Amazon Cloud Drive from the UnionFS-FUSE, and attempt to upload it with acd_cli upload, it would not upload unless you provide the --overwrite flag.

That’s All!

There’s really nothing more to it. You can continue to upload the /home/plex/.local-sorted directory contents as before however making new files is a lot simpler as you appear to write to a view which looks like Amazon Cloud Drive.

Reading new files is also improved, as you can choose to write files locally and preserve it locally for a set time. To delete files older than two weeks, you’d simply need to add a cron such as:

0 4 * * * find /home/plex/.local-sorted/ -type f -mtime +14 -exec rm -rf {} \;

Providing the file co-exists on Amazon Cloud Drive by then, /home/plex/sorted/ would appear unchanged, it may also be worth uploading via a cron also to ensure you never lose data.

If you’re using an application such as Filebot, you should sort into /home/plex/sorted/. The encfs mounts should only be used for directly uploading to Amazon Cloud Drive.