Monday, September 01, 2008

Standards for System Administration

My favorite article from the August ;login: magazine is online: "Standard Deviations" of the Average System Administrator (.pdf) by Alva Couch. I'd like to highlight some excerpts:

System administrators have a surprising amount in common with electricians. Both professions require intensive training. Both professions are plagued by amateurs who believe (erroneously) that they can do a good job as a professional. Both professions are based upon a shared body of knowledge.

But electricians can call upon several resources that system administrators lack. Electricians have a legally mandated mentorship/apprenticeship program for training novices. They have a well-defined and generally-accepted profession of job grades, from apprentice to journeyman to master. They advance in grade partly through legally mandated apprenticeship and partly through legally mandated certifications. These certifications test for knowledge of a set of standards for practice—again, mandated by law. The regulations are almost universally accepted as essential to assuring quality workmanship, function, and safety.

In short, one electrician can leave a job and another can take over with minimal trouble and without any communications between the two, and one can be sure that the work will be completed in the same way and to the same standard. Can any two system administrators, working for different employers, be interchanged in such a fashion?

At present, system administrators are at a critical juncture. We have functioned largely as individuals and individualists, and we greatly value our independence. But the choices we make as individuals affect the profession as a whole. I think it is time for each of us to act for the good of the profession, and perhaps to sacrifice some of that independence for what promises to be a greater good. This will be a difficult sacrifice for some, and the benefits may be intangible and long-term rather than immediate. But I think it is time now for us to change the rules.

From standards for distributions (e.g., the Linux Standard Base) to standards for procedures (e.g., those upon which Microsoft Certified Engineers are tested), I believe that — although standards may annoy us as individuals — standards for our profession (and certification to those standards) help build respect for system administration as a profession. Compliance with standards gives us a new and objective way to measure the quality of management at a site. Standards not only make the task easier but also enforce desirable qualities of the work environment and help to justify appropriate practices to management. Adoption of standards also has a profound effect upon our ability to certify system administrators and even changes the
meaning and form of such a certification.

Is a system administrator accorded the same respect as an electrician? I think the answer is an emphatic “no,” at least for those electricians who hold a master’s license. There are two factors that engender respect for a master electrician: legally mandated standards linked closely to legally mandated apprenticeship and certification.


I think the whole article is worth reading, but those are the key points. Now, I'm sure many of us have electrician horror stories. I know someone (not me) who was unlicensed and therefore had to hire an electrician to wire an addition to his house. The "electrician" did such a poor job that this person then rewired everything to code himself rather than bother with the electrician again. I don't think that's the norm, but I wonder if there is any research that might support Dr. Couch's statement that one electrician can leave a job and another can take over with minimal trouble and without any communications between the two, and one can be sure that the work will be completed in the same way and to the same standard?

Still, I guarantee that most every system administrator handles boxes differently. Even within the same company, I find systems horrendously maintained. I once assumed control of a set of "Linux appliances" built and operated by a managed security service provider. They were all built for the same purposes, but ran a variety of Linux kernels with different applications, versions, and configurations. These were all operated by the same small MSSP!

Perhaps one of the worst examples of our lack of standardization involves network diagrams. Sites like Rate My Network Diagram will make you laugh and cry. I usually cry because I took four years of architecture training in high school. We did most everything by hand (it was the late 1980s), to include learning how to write the various "architectural fonts" we were expected to use. (We did start learning how to operate AutoCAD on the Applie IIGS just before graduation!) The point, however, was all of our diagrams looked similar, if not the same. This standardization allows one architect to review and build using another's plans without wondering what the various lines and icons mean.

Incidentally, I know about Cisco's icons. I'm talking about a standard way to use such icons, not standardized icons themselves. That's only one step.

Don't get me started on standard terminology... Yes, the image on the left depicts my feelings about the maturity of our industry. It's still early days, so I hope we decide to professionalize during my working lifetime.

6 comments:

Matt Simmons said...

IANAE (I am not an electrician), so you'll have to forgive me for being under the impression that the typical "professional" systems administrator deals with a far more diverse set of parameters than an electrician.

I'm not sure what the best analog would be to systems administrators. It would have to be something where different locations have wildly differing requirements, different ideas of how things should be done, and different hardware.

At the risk of offending certain people, maybe 'priest' is a good analogue. Jewish rabbis might be a good parallel to the old wizened Unix mages. Linux and Christianity might be an ok fit, given the splintering of both into different denominations and sects.

Windows admins, of course, fall under voodoo. No real religion to speak of, just a series of superstitions ;-)

Christian Folini said...

IANAEE (I am not an electrician either), but I agree with Matt, that there is more to system administration than to wiring a building.

But this makes standards even more important. Richard, you are right in your statement, that this is only the beginning. People have been wiring houses for over a century. System administration as a discipline is a lot younger and standards are just evolving.

Maybe we would see more standards if things would not change so fast. But maybe there are a great many standards, but they are hidden from our eyes. I am hardly aware of LSB during my regular work - the package management does it all for me. I used to compile a lot. Now the package management does it all for me. Etc.

As automation takes over, the enforcement of the standard is being automated and the system administrator is left with the newer, non-standartized configuration choices. The author thinks this is where the "incidental complexity" sits. He's quite right about that.

The quoted article puts the finger where it hurts and points in the right direction. Way to go.

David Bianco said...

LTOCIANAE (Like The Other Commenters, I Am Not An Electrician). I agree that system administration is probably more complex than the average house rewiring job. For one thing, electricians wire to a specific code. System administrators often have to work cross-platform (Windows, OS X, plus however many different versions of Unix/Linux). Each of those platforms is effectively a different "electrical code".

Also, and lets be honest, even the best system admin probably has only a vague idea of what's actually going on inside their systems. Computers are basically big black boxes, and trying to manage them is a lot like playing black box: send in some inputs, see what output you get and try to infer the inner workings that lead you to that outcome.

I'm also not quite sure about this quote: Is a system administrator accorded the same respect as an electrician? I think the answer is an emphatic “no,” at least for those electricians who hold a master’s license. It's an apples-to-oranges comparison, as it's an explicit comparison between the most respected of one class (the master electrician) and the average of another (the system administrator). I've known some very highly respected system administrators, so I'm not really sure this comparison means anything except that someone with a master's license is presumed a priori to be more experienced than an apprentice or journeyman. Me, I've got certifications coming out of my ears, but I don't think they necessarily make me more respected in my profession.

red said...
This comment has been removed by a blog administrator.
Anonymous said...
This comment has been removed by a blog administrator.
stellathomas555 said...
This comment has been removed by a blog administrator.