Living in a small space, as I do, it is important to maximize the utility of every inch of the place. Generally, I am not doing that so well as I might, but in one area I excel: Had I tried to bring with me my library of books, music and movies, there wouldn’t be any room in my apartment for a bed! Now, I’ll be the first to admit that there is a certain joy that can be taken in handling a real dead-tree book. I love the feel of the paper; I love the smell; I love being able to freely flip back and forth through the pages. I do not love being a slave to my bookshelf, though, so when I started down-sizing, I had a choice to make: Do I keep all of my wonderful books, or do I find an alternative? Ultimately, I decided that the books and DVDs and CDs had to go. There just wasn’t going to be room for them. What there is room for, though, is a small office NAS (Network Attached Storage). The device pictured here, the Seagate Blackarmor 440 was a little pricey at $600 for the enclosure and two 1 TB hard disks. It has since garnered some very unfavorable reviews at Amazon—mine’s been fine, but maybe shop around for a better one. It has enabled me to reduce two bookshelves’ worth of media to a box that is less than 10 inches in its longest dimension. It fits easily behind one of my computer monitors, and it streams fast enough to keep up with standard def video and h264 HD.
As I had the budget, I have upgraded the drives to four 2 TB disks, which gives me over 5 TB of storage (some space is lost to data redundancy, which I’ll cover in a bit). With 4 TB disks now on the market, I expect to continue upgrading the server to double its capacity again. My current library stands at 839 books. My movie collection contains 147 titles, and TV shows are 159 seasons of 33 programs, I have 484 comic books, and 725 tracks in my music library. Add an archive of all of my art school projects and freelance jobs, and I still have 1.8 TB free.
You can imagine how much space all of that would take up if I had physical copies of everything!
The biggest concerns with converting my entire entertainment collection to digital format and storing it on a single device are:
- Data integrity—what happens when a drive fails (as it will eventually do)?
- Backups—If the server is damaged or stolen, will I lose all of my books and movies?
- Management—How on earth do I keep track of so much stuff?
- DRM—Do I really trust Amazon and Comixology to be around forever?
Each of these things could be a huge problem, but they can all be overcome, and I’m going to teach you how I did it. First up: Data integrity!
File Servers and RAID
If you look up RAID on Wikipedia, you’re going to get a formidably technical article that will scare you off of ever wanting to mess with it. Relax; it’s not as bad as it seems. IT pros will try to convince you that setting up a RAID (Redundant Array of Independent Discs, if you’re curious, but you don’t have to know what it means to use one) is difficult and fraught with peril. If you’re trying to install a Windows server in a data center, that’s true. For a simple small-office file server like the one I use, it’s as simple as clicking a couple of buttons in the configuration menus.
I won’t walk you through the exact steps because they’ll be slightly different for every model of NAS, but this is the essential knowledge you need: Choose RAID-5.
The why’s are pretty simple:
- RAID-5 gives you redundancy. If one of your disks fails, your server will continue to operate. Just replace the bad disk with a new one with at least the same capacity, and the server will automatically restore itself. Don’t let it run too long with a bad disk, though, because if you lose two you lose everything!
- RAID-5 uses up less space for redundancy than other RAID levels. RAID-1 (mirroring) cuts your storage in half. RAID-5 reduces it by the size of one disk. RAID-6 reduces it by the size of two disks.
- RAID-5 is pretty fast. Writing to it takes a little longer, but reading from it, which is where the speed really matters for this purpose, is three times faster than a single disk (assuming a 4-disk array).
RAID-1 and -6 are slightly safer. RAID-1 is slightly faster. RAID-0 maximizes capacity and is fastest of all, but it has no redundancy. Of all the available levels, I find RAID-5 to be the all-around best choice for a media server. The downside is that you need at least three disks in order to use it. If you have a two-disk NAS, your only viable choice is RAID-1. RAID-0, while super fast and having a large capacity, is actually less safe than a single disk because if either of the disks fails, you lose the data on both of them.
Update: Crashplan has discontinued their Home product. I am on the hunt for a new backup solution for my media server.
Update to the update: In a perfect storm of misfortune, soon after Crashplan closed down their product, the server was stolen. I tried to migrate from Home to their Small Business plan to save my data, but they told me that the maximum size for a transfer was 2 TB, and I had almost 4. So all of my media was lost. Fortunately, the truly irreplaceable data is backed up on DVDs as well, but it’s going to take a lot of time and expense to restock my digital library.
My new NAS, a Synology DS720+, is backed up to Amazon Glacier, which costs about $5 per Terabyte per month. That makes it a little more expensive than Crashplan’s Home product, but less than their Small Business plan. The new server has a capacity of 5TB at the moment, so I shouldn’t spend more than $25 / month if I completely fill it. Of course, once I start expanding further, it may get way more expensive. Also, running the backup client on the NAS instead of my PC is an improvement. Crashplan sometimes hogged some resources.
Media Management: Calibre, Kodi and ComicRack
With thousands of files to keep track of, I need some software to make finding, categorizing, and consuming my media easier. Here are a handful of programs that I find very useful:
Calibre and the Amazon Kindle
If you are a fan of e-books at all, you’re probably already familiar with Calibre. But if you’re not, it’s a fantastic little utility to manage your electronic library. I can plug my Kindle into my computer via USB and transfer books to it easily. It can convert e-books to numerous different formats, and with the right plug-in, it can strip DRM from books sold by Amazon, Barnes & Noble, and others (more about that later—I don’t condone piracy, but I do believe that DRM is a needless and harmful restriction on media I have legitimately purchased). As you can see from the screenshot (click for a larger view), it manages tags, keeps track of novel series, and attaches extensive metadata to each title. Better still, Calibre will automatically download the metadata and book covers from Amazon and Google, making management of your library simple.
I use an Amazon Kindle (a generous Christmas gift from Zoic Studios five years ago) as my primary reader. I find the e-ink display to be much more comfortable for reading than the LCD screen of a tablet or phone. For material with lots of pictures requiring color (comic books, mostly) I also have a Kindle Fire (a generous Christmas gift from my ex-wife).
Kodi, Ember Media Manager and the Amazon FireTV
For playing and managing my movies, music and television programs, I have installed Kodi on my FireTV.
This is another rather involved technical process—sideloading an app onto the FireTV is not easy. Update: Installing Kodi is way easier these days. I found some useful instructions at troypoint.com. It was important to me to maintain access to the special features on all of the DVDs I ripped. I wanted to keep the menus and so forth intact, so I couldn’t just convert everything to h264 and stream through Plex (the simple choice, and a marvelous one if your needs aren’t like mine).
I ripped all of my DVDs to Video_TS folders and stored them on the media server. Kodi plays them as though I had the physical disc loaded into a player, and the FireTV has enough processing power to be able to deinterlace without stuttering. My previous playback device, an Apple TV 2, didn’t quite have enough CPU to manage that feat (also, support for Kodi on the Apple TV has ended). In order to further optimize my space, I attached the FireTV and a power bar to the back of my television with self-stick velcro. Not only does it save space, but it also lets me bundle all of the cables right behind the television, keeping them neatly coiled and out of sight.
Kodi does a fairly nice job managing movies, music and television programs on its own, but if you have other kinds of material, like my small collection of Gnomon Workshop training videos or home movies, you’ll need to supplement it with another media manager in order to easily access those files through Kodi. For that purpose, I like Ember Media Manager. It can add tags and other metadata to your videos, extract frames to use as thumbnails or poster art, and create informational files that Kodi and other similar programs will use to organize the files.
ComicRack and Comixology
Comixology has done a great job in making digital comics available affordably and conveniently. I am a huge fan and a loyal customer. Their recent decision to permit users to download some comics to the CBZ format just makes me like them even better.
For the most part, Comixology’s apps are my preferred way of reading comics. The guided view is a very nice feature, particularly when reading on small screens. However, since I prefer to maintain my collections using tools that I control on file systems that I have unfettered access to, I don’t like to leave everything up to them. Whenever possible, I download CBZ versions of my books and index them with ComicRack. The CBZ format is simply a series of numbered images inside a zip archive and renamed with a .cbz extension. This super-simple file structure ensures that even if every reader available suddenly stops working, I can still view my comics using any simple image viewer. There’s no encryption, no DRM, and no other funny business to get in the way of enjoying my books.
Digital Rights Management: We don’t need it, and we don’t want it.
Ever since the invention of the photocopier, media companies have been terrified of piracy. They’ve spent billions attempting to protect their property from copying and theft. Most of the schemes they come up with are broken by the pirates within hours of release. Blu-Ray’s AACS encryption is the champion in this regard, lasting almost a year from the time the standard was put into use until the first crack appeared.
Media companies have responded to this continuous circumvention of security by creating ever more restrictive schemes and lobbying for new laws to criminalize any act of cracking, even if the user simply wants to burn a CD of an audio book to listen to in the car (thankfully, Audible has relaxed their authorized device controls and now permit burning CDs right out of their software). The end result has been to hamper end users’ use of the media they pay for while simultaneously doing nothing to slow down the pirates. For a while, it was my standard procedure to immediately download a cracked version of every computer game I legally purchased because the crack was usually more stable and convenient!
To top it off, in the event that the company holding the encryption keys goes under, it is possible that it will become impossible to access the media they sold you. They cover themselves by claiming that what you’ve paid for is a “license” to view the media, not the media itself. Given that we’re not provided with much opportunity to negotiate the contract, I feel comfortable in rejecting that claim and taking steps to preserve access to the entertainment I have purchased. Though I suppose by that reasoning, I should be paying sales tax on my digital content. Hmm…
So, here are my solutions for avoiding and nullifying DRM:
- The DeDRM plug-in for Calibre can remove DRM from books downloaded through the Kindle4PC application. Kindle4PC decrypts the book, then DeDRM reencodes it without the encryption. This happens immediately upon import to your Calibre library. I prefer to then convert all of my books to the Mobi format, which can be read by all Kindle models currently available. DeDRM can also decrypt books from other services, but I only use Amazon at this time.
- CDs, DVDs and Blu-Rays can be decrypted and ripped. I have not yet found a reliable method of removing DRM from videos purchased from Amazon for digital delivery. I don’t have any experience with ripping Blu-Ray, but for DVDs I like Ideal DVD Copy.
- For backing up encrypted books from Comixology, try Cortys for Chrome. It automatically and swiftly takes a screenshot of each page and packages the images in a CBZ.
Let me reiterate that piracy is a Bad Thing. As someone who makes his living from media production, I am dead set against theft. I’m not making six figures and living in the lap of luxury here, and 98% of people working in production are in the same boat. So have a care to pay us for the joy we bring to you. But don’t let the giants push you around and tell you what you can and can’t do with the media that you have purchased.
There you have it. 2500 words about how I reduced my giant media collection from a set of bookcases to a little black box. I won’t say that it didn’t take me some time, nor will I claim that it was an inexpensive project. In the end, though, it’s greatly simplified things for me and saved me a lot of back pain from moving all of those books!