• When the pieces just fall into place

    There are things you are struggling with over a long period of time, only to see them resolved all of a sudden. Yesterday, I had three of those experiences on one day. It will not surprise people that know me that all three ended some kind of IT dilemma. So, if you’d indulge my techno babble, here’s my triple “Yay!”:

    Sometimes, there’s another choice

    If you’re running a big IT infrastructure as we do at freistil IT, it’s not enough to regularly check if a server still has enough disk space or free processing power. It’s equally important to be able to see how these metrics develop over time. These statistics help understanding past events (“Look, right before the server crashed, its free memory dropped to zero.”) and preventing future incidents (“At this rate, the disk will be full in three days.”). But building those statistics for hundreds of servers means collecting thousands of data points per minute.

    Of course, there are powerful solutions for this, for example statsd, developed by the devops team at Etsy whose virtual marketplace is powered by am impressive server farm. Although statsd looked very appealing, I restrained myself from trying it because it would make it necessary to start using node.js, a technology we don’t have much experience with. While I love entering new territories, I also need to focus on the work at hand. I’m still getting used to code in Ruby, the language in which our infrastructure automation tool, Chef, is written. So, I was sad to realize that statsd is a can of worms I just can’t open right now.

    Well, turns out the guys at 37signals were in the same situation, so they built batsd, a data collection agent that is compatible with statsd but written in Ruby. Let’s start gathering metrics. Yay!

    Sometimes, the choice between A and B is C

    Mac OS X is a great combination of graphical user interface and UNIX technology which makes it the ideal tool for a sysadmin come business owner. Unfortunately, some of its components are not as great as others, and Finder is a particularly sad example because it lacks many features that an experienced computer user simply expects from a file manager.

    I’ve been using PathFinder as a better substitute but it can’t replace Finder completely because it’s just not as integrated into the OS (for example, its missing the Dropbox item in its contextual menus and other applications still open Finder). The newest PathFinder version is a paid upgrade and I’ve been hesitant to shell out the money because I don’t want to have to work with two competing file managers.

    Yesterday, I found TotalFinder which takes another approach: It expands the Finder’s functionality instead of trying to replace it. So, more power but seamlessly integrated. Yay!

    Sometimes, you don’t have to choose at all

    I live my life as much paperless as possible. Paperless to me means efficient and flexible. And nowadays, there are strong bridges between the material and the digital world making the paperless life easy, for example my ScanSnap scanner and my Kindle ebook reader. David Sparks’ new “Field Guide” book for the iPad, appropriately named “Paperless”, is a great read on this topic and contains a lot of helpful advice in form of text, pictures and videos. Among many other things, it describes alternatives for organizing a growing number of digital files.

    A few years ago, I chose DEVONthink Pro Office for that purpose because it has great searching, tagging and sorting features and comes with built-in OCR to add even scanned documents to the search index. One of DEVONthink’s downsides is that it’s hard to share the database that contains all the documents. When we got help with accounting, I found it necessary to get our bank statements and invoices out of DT and put them into a nested folder structure in Dropbox. Since then, I had not been able to decide in which direction to go now – should I choose the power of DEVONthink or the flexibility of a simple folder hierarchy?

    As luck would have it, yesterday I read Fletcher Penney’s blog post about his PDF reading workflow in which he mentions that DEVONthink has an “index a folder” feature that indexes files inside a certain disk directory without moving the files themselves into the application’s database. Best of both worlds. Yay!

    Great findings on on single day. Let’s call it a Satur-yay! (Okay, it’s late. Please don’t sue me.)

  • Everyone. Needs. Backup.

  • Devops podcasts

    While there hasn’t been a shortage on IT engineering blogs for a long time, podcasts that deal with devops topics are a rare sight. That’s why I’d like to recommend the ones that I currently subscribe to:

    • DevOps Cafe Podcast: Damon Edwards, John Willis and guests talk about interesting news in the datacenter world.
    • The Changelog: A show that “covers what’s fresh and new in Open Source”, hosted by Wynn Netherland and Adam Stacoviak
    • The Food Fight Show is a bi-weekly podcast for the Chef community, or, as hosts Bryan Berry and Matt Ray put it, “The Podcast where DevOps chefs do battle”
    • itkanban’s podcast comes also bi-weekly and covers news about lean and agile IT management methods.

    Do you know any other podcasts a self-respecting system administrator should listen to? Please post them in the comments!

  • How to use a remote shell over flaky connections without losing your nerves

    Recently, I’ve stumbled upon the Mosh remote shell application, and since then I can’t stop talking about it! If you need to access your servers’ command line interface over slow, unreliable connections, you want to use the Mobile Shell.

    As the website describes it, Mosh is a…

    Remote terminal application that allows roaming, supports intermittent connectivity, and provides intelligent local echo and line editing of user keystrokes.

    And – spoiler alert – it works so well that I’ve completely replaced ssh for mosh for accessing our many servers. SSH is still necessary, though, because the Mosh client first opens an SSH connection to the target server and then launches its server component there. After establishing an UDP connection between the client and server components, the SSH connection is dropped.

    By using a new protocol called the State Synchronization Protocol (SSP) which is based on UDP, Mosh provides a shell connection that’s far more usable over slow and flaky connections than SSH, for example when using a 3G network from a train. It even survives reconnects that change the client’s IP address. I was really amazed when, on my first day with Mosh, all shell sessions I started in Starbucks simply resumed after I opened my laptop again in my homeoffice.

    Local Echo is another great feature that makes working over unreliable connections far less annoying. While SSH doesn’t display your keystrokes until they have been sent back from the server, Mosh shows them immediately without requiring the roundtrip. That way, you can spot and correct typos without wait and finally hit the Enter key with all confidence. It may take some time until you see the effect of your command due to your slow connection, but at least typing it was no hassle at all. Whily typing, Mosh gives you visual feedback about the synchronization process by underlining those parts of the command line that have not yet been acknowledged by the server.

    Of course, Mosh also uses encryption, so you don’t lose any security by switching from SSH to Mosh. Installation is easy, too, so don’t wait any more. Start moshing!

  • OSDC -- Meeting of the pro geeks

    On Wednesday and Thursday, I'll be in Nuremberg again for this year's Open Source Datacenter Conference. It has a great lineup, for example:

    • Kris Buytaert, Devops evangelist and fellow Drupalista
    • Kristian Köhntopp, with whom I shared an employer and still share the pleasure of drinking whisky
    • Kenny Gryp and Alexey Kopytov from the MySQL experts at Percona
    • Olivier Renault, cloud computing engineer at Eucalyptus

    And it looks like my Chef presentation at OSDC 2011 was well-received enough for host Netways to have me take part again, this time to talk about organizing IT teams with the Kanban method. Thanks, guys!

    I'm already excited and can't wait to get on the train on Tuesday. Not only will this be an amazing source of inspiration and knowledge. It's meeting all those great minds in person and reuniting with former colleagues like Kris, Peter and Martin (Andy: a pity that you won't be there) what I'll enjoy the most.