I’ve been sitting on this post for a long time, but thought I’d give it a go today as I’ve been working on some more generic type tasks that would apply outside of my own work environment. These seven things are apps I keep on my laptop to keep it the ultimate fire-fighting machine. Currently, my laptop runs Ubuntu, but most of these applications will run on any platform. While I find many things useful throughout the day, and there are many different applications and systems to use, I’m going to try and focus on portable swiss army knives and abstain from things requiring permanent installations or their own machine. I think that perhaps this list might be of usefulness to people just starting out in the trade as well as the old timer.

So, in no particular order…

Git Version Control

So maybe you don’t do a lot of scripting or coding. Maybe you don’t do any at all. Why would you need version control? The short answer is in the question of “How many times have you  made a minor configuration change and wondered how and why it got that way a year down the road?”. The long answer you’ll only find yourself after putting such a system into use. About everything I make a change to gets copied locally and committed into a git repo. Here’s a the rundown of the workflow:

  1. Copy the file, or the text of the file (say, a crontab entry) to a local file.
  2. Add it to git with “git add filename.ext”
  3. Commit to the git repo with an explination as to what it is: git commit -m ‘Adding crontab for oracle user from sys01-db’
  4. Make the change locally.
  5. Recommit to git: git commit -a -m ‘made the change for backup periods’
  6. Do the minor change on the target system.

Now, all your changes, times, and reasons are logged in the git repo. One thing to notice — I keep using the words “minor change”. To translate, you should really find better ways to push configurations that are bigger than small stuff, like the use of packaging and deployment systems. You should probably commit those to version control too.

Now that you’ve got this file of all the things you’ve done, you can trade with your friends! Say a cohort on the other side of the country works with the same system you do. You can use git to prepare an email with a patch of the change that was made without other tools. After you get to using the system for a while, you’ll find all sorts of social ways of sharing the things you’ve done, most of which, are more useful than the personal archive you’ve kept.

Don’t like git for some reason? Try another distributed version control system like mercurial or bzr. They both have their advantages, and I’ve been on the bandwagon for both of them at one time or another.

And the last note: this is absolutely not a replacement for a good backup system. At ALL. Under no circumstances.

A Really Good Editor

Undoubtably, you’re typing a lot. You’re probably repeating things you type a lot. We’re talking text here, not a word processor, not emails. Here are some good things to look for in a text editor:

  • Syntax highlighting. If you’re working on a apache config, make things that are particular to that config stand out for different reasons.
  • A right margin. No really. Lines that go out over 80 characters can be a total pain on a terminal later.
  • Auto-completion or snippets. Typing in a abbreviation and hitting tab as a shortcut for longer text.
  • Syntax checking. Missed a closing bracket? Wonder where that closing bracket starts? Click to find out!

Over the years I’ve gone pretty deep with my editors of choice. vi and emacs both do the things above and there’s loads of plugins to accomplish things in them, but here are some of my favorites that I use these days:

  • Netbeans. Yup, as an editor. It may be a bit heavy for most work loads, but I find a lot of its features invaluable. Crossplatform.
  • Textmate. If you’re on a Mac, this is the way to go. I bought it a few years back and feel like I’m ripping the author off for everything I’ve gotten out of it. It’s everything it needs to be and nothing it doesn’t. I hear good things about the E Text Editor on Windows, which supports Textmate bundles, but I haven’t used it (Windows or the Editor).
  • Gedit. The official editor of Gnome. Also a crossplatform tool, I use this in a lot of cases when I don’t feel like loading up the full Netbeans app. It’s pretty useful out of the box, but after you add a few plugins, it becomes amazing. Combined with the rest of Gnome on a *NIX box, it can get pretty out there with options, like editing of remote files, etc.

A Scripting Language That Thinks the Way You Do

If there was a real motivation to writing this post, it’s this right here. Ruby. It’s the glue that ties everything together for me. If you’ve toyed with other scripting languages in the past, this one is nothing like it, but probably takes the best parts of the one you toyed with. If you’re a certified perl monger, you’ll find your oneliners understandable and your regular expression syntax right where you left it. If you’re a python person, you’ll be almost at home.

One of the best parts of Ruby is its fantastic gems library (or not, depending on who you ask). A ruby gem is a program or library you can call on to extend the functionality of the language. Need a simple way to get the mean of a bunch of numbers? There’s a gem for that. Need to send a bunch of emails? Write PDFs? There’s gems for that too. Gems couldn’t be more simple to install and use as well. “gem install rubygemname” installs your gem, and using it is as simple as “require ‘rubygemname’” at the beginning of your script.

Another greate Ruby feature is not so much a feature, but an implimentation: JRuby. It’s a ruby intrepreter that runs on or in the JVM depending on how you see it, meaning any machine with Java can run Ruby. This also means that you can extend Ruby with Java, or if you’re not so inclined use some gems that already do this, like JDBC database connectors and other neato Java applications.

As a Sysadmin, you’ll find yourself doing all sorts of repetitive tasks and solving some not so easy or sometimes even precidented problems. Other languages require you to speak computer. Ruby understands English. You’ll never find it easier to get whats in your head into the computer and back out again.

I could, and would go on with the beauty of the Ruby way of doing things and its community for a really, really long time. But I’ve other things to do. Using Ruby.

A Place to Organize Data Structures

Log parsing, data corelations, pattern recognition, transformations, etc.. You’ll end up doing them all mid-outage looking through gigabyte sized logs. Note: optimizing sorting algorithms in the middle of an outage situation isn’t the best use of ones time.

Even in non-outage situations, perhaps you need to find out some infomation about a node’s activity. Perhaps you want to count a type of certain errors and co-relate them to a different source.

There are many free and commercial apps out there for log parsing (I highly recommend Splunk if you can afford the license), but in a lot of cases they doesn’t apply to to the problem at hand. So:

  • Become a regular expression artist.
  • Learn awk and grep.
  • Find the best way to dump the data in a parseable, workable format.
  • Know SQL.

With that said, I’ve currently got three database platforms of choice lately:

  • sqlite3. It runs everywhere. It interfaces with everything. It can handle large data sets. It’s self-contained, It’s free. It has some interesting features. No dedicated server required.
  • MySQL. Ok, server required, but that’s not a bad thing when it runs on everything for the most part. It’s fast and has some great tools that accompany it.
  • Oracle XE. Didn’t think I’d go there, did ya? While there are some limitations, the bundled APEX utility set can be useful for visualizing problems. It has all the connectivity options of its big brother which is another big plus.

So there’s some things to get you started and some toys to play with.