Welcome to our media collection

  This is the entry point for our media collection. Here you can find catalogs of our CD, DVD, and LaserDisc collections, as well as information about the hardware and software used and a brief history of the project.

• Our new ultra-cool CD Browser - a must-see!.
• CD's on Terry's servers.
• Terry's DVD collection.
• Terry's LaserDisc collection.

  You could also read this footnote about cataloging.

Hardware

  Since I've received many requests saying "so, what does all this stuff look like?", I decided to take some quick pictures to show you. Each of the following pictures is clickable to display an image twice as large. And yes, all of this stuff is really in my house...

  Note that this is only the latest incarnation in a long string of servers I've had here. For a historical perspective, you can click here for a view from January, 2000 and here for a view from February, 2003.

  This is the overall view of the cabinet. From the top down are:

• Cisco Catalyst 5505 switch. This has 24 10/100 Ethernet ports, two long-haul Gigabit fiber ports, and 9 Gigabit copper ports. It also has the RSM (Route Switch Module) so it functions as a Cisco 7500-series router as well. The Gigabit fiber ports connect to my offsite backup location, while the Gigabit copper ports connect to the various severs in this cabinet as well as the PCs in my house. The remaining 10/100 ports connect to various maintenance devices like the UPSs and console server. The coiled cable dangling on the left is an anti-static wrist strap.
• Two VoIP boxes, each with 2 ports. The left one is a Linksys RT32P2 from Vonage, while the right one is a Cisco ATA186 used with FWD.
• Cisco 2621 router. This handles my four outgoing T1 connections as well as 4 VoIP (Voice over IP) ports.
• A shelf with three 40/80GB DLT drive (attached to various systems below this shelf). One connects to the DS10, one to the PowerEdge 750, and one to the RAIDzilla.
• HP [Compaq [DEC]] DS10 server. This is the box I use for an email server. It runs VMS.The gold-colored bar below it is just used to support it (the bar is mounted on the center set of rails in the rack).
• One of two APC AP7900 8-outlet remote reboot boxes.
• A Dell PowerEdge 750 server (3.4GHz Pentium 4 CPU with 4GB of RAM) running FreeBSD 5.4. This system is the web server for www.tmk.com, among other uses.
• Two RAIDzillas, each with 4.8TB of data. They also have 2 hot spare drives. These servers are mirrored over Gigabit Ethernet to an offsite location for backup and redundancy.
• Cisco 2511RJ router. This has 16 serial ports which are connected to the serial console ports of the various systems in the rack.
• The second AP7900 reboot box.
• APC Symmetra UPS. This is a 6KVA unit which can run the equipment for about an hour and a half in the event of a power failure.

  A close-up of the top half of the cabinet.

  A close-up of the bottom half of the cabinet.

  The RAIDzillas in a close-up view.

  The PhatNoise car MP3 player. For more info about my car, click here.

Software

History

  Meanwhile, I started hanging out with Amanda, and we discovered that despite the difference in our ages (17 years), we had similar tastes in music and managed to have about a 30% overlap between our CD collections. Amanda was accepted to graduate school in Illinois, and didn't particularly want to move her 450-plus CD's along with all her other stuff (hey, she needed the space for clothes 8-). So, since she's really cool with technology, we decided that we should get her a RAID server as well, and we'd copy her CD's to it and she could listen to them all out there, while keeping the original CD's in her New York City loft. She was moving in August, 2001 and I soon discovered that I couldn't fit as many of her CD's on a server, compared to mine. That's because hers are mostly current dance CD's (70 minutes or so), while mine were mostly classic rock (46 minutes, originally to fit on 2 23 minute album sides). So I ordered more servers. And so the mania began...

  Since I was reading these CD's with a 12X (state-of-the-art at that time) CD recorder, I was spending 6 minutes just reading perfect discs. And that didn't include time to swap discs in the recorder, figure out what a CD was (there were a lot that were "naked" discs without jewel cases, and lots of those just had artwork on the disc and no identifying information), or deal with badly scratched discs that would only read at 1X. Having read in about 800 CD's (at 7 minutes each, that's 93 hours), I decided I did not want to do that again, so we decided that Amanda would keep a backup copy of my CD's on her servers in Illinois, and I'd keep a copy of her CD's on my server in New York. At this point we'd gone from 2 servers (one each planned for New York and Illinois) to 6 servers (3 each), and my CDW account rep was very happy with me.

  Amanda moved (as planned) in August 2001, and after I came back in mid-September from helping her get set up out there, I was greeted with a layoff from Verio. I was hoping there would be headlines like "Heartless Japanese firm throws employees out on street in 9/11 aftermath" and I'd get my job back, but the massive layoff gathered little notice outside the Internet community. I was forced to cut my Internet connection back to a T1 from a T3. A T1 isn't fast enough to stream a CD in real-time, so I couldn't play them in the Verio office where I no longer worked - no great loss.

  Since I was mostly unemployed at that time (I was doing consulting, which left me lots of spare time), I started hanging out with some fellow geeks, and of course the conversation drifted to my online CD collection. Most of them thought this was a great idea, and wanted to get involved. Ritz, Ylana, and Ben (in that order) joined the collection. Or is that the collective: We are the Snap! servers. Resistance is futile. Your CD collections will be assimilated.

  As the collection grew, so did the quantity of servers. I was still using a combination of 240GB (179GB usable) units in Illinois and 300GB (224 GB usable) units in New York. Around this time, I had a bad experience with the IBM Deskstar 75GXP drives. While I didn't have a failure in any of the RAID servers, I did have 2 of these drives go bad in PC's, with no notice, within a week of each other. I decided I had to do something to make the RAID servers more reliable, and to expand capacity.

  Fortunately, Quantum had just released their own 480GB model of the server, using 4 Western Digital 120GB drives, along with a new OS version that supported generic drives. So I ordered 4 drives as a test to upgrade a test server, and it worked well. So I called up my CDW rep and said "please send me 4 dozen of the Western Digital drives", which made him happy (again). The servers were upgraded by copying all the data to an empty, upgraded 480GB server and then pulling the drives from the old server, putting in the new ones, upgrading the OS, and using that server as a destination for the next copy, and so on.

  Eventually we wound up with 19 480GB servers, 9 here in New York, 6 at other locations in New York, and 4 in Illinois. We also have 3 120GB servers - Snap 1100's modified with 120GB drives - used to move music around between sites. Eventually the Snap servers became unworkable for this much data and I migrated everthing to the RAIDzillas.

  In the fall of 2002, we decided to generate MP3 versions of the music (in addition to keeping the full CD images) so we could listen to our music anywhere there was a reasonably fast Internet connection (DSL, cable modem, or faster). As I mentioned above, playing a CD images requires a link that is faster than a T1, so we picked MP3 encoding to save on bandwidth. We're using VBR encoding with a minimum rate of 192Kbit/sec. The CD images were processed directly from the Snap! servers into MP3 files. Amanda and I both have the PhatNoise Car Audio System (also called the PhatBox) in our cars now, so we have all our music at our fingertips. I've written an add-on package for the PhatBox, called PhatVoice. More information about it can be found here. We also generated "gapless" MP3s which are a single MP3 file per album. Due to the way MP3s work, if you listen to a mix album there will be a gap between tracks which can interrupt the flow of the music. Encoding a whole album to a single MP3 solves this problem. The PhatBox has support for seeking between tracks in MP3s, so it works well in our cars.

  Currently (as of early September, 2005) there are 2117 unique CD's in the collection, taking up about 1.2TB of space for the CD images, 160GB for the regular MP3s, and 185GB for the gapless MP3 versions. The gapless ones are larger because they are recorded at a constant bit rate (224Kbit/sec) since a constant bit rate is needed for reliable track seeking within the MP3 file. I should emphasize that we have the all of the original CDs in our possession - this server is a convenience feature, not a bootlegging project.

A few notes about criteria for inclusion of media

  Also, the DVD and LaserDisc listings aren't 100% complete. I've omitted the stuff that might be classified as erotica - for example, some of the more risqué stuff like erotic Japanese Anime isn't listed.

  Currently I'm only cataloging my own DVD's and Laserdiscs - in the future I may expand this to the other contributors' collections.

  Also, the DVD and LaserDisc entries are just catalog data - the media have not been loaded to the servers. While it is technically possible to do this for DVD's, they take between 4.5GB and 18GB each. LaserDiscs have analog video and there isn't any way to directly store them on digital media, which is a shame as there are titles only available on out-of-print LaserDiscs which suffer from Laser Rot. Perhaps at some point in the future I'll look at video encoders to see if it is worth trying to preserve these somehow. Unfortunately, true DVD video encoding appears to be quite expensive.