Automation’s great for ops, no matter the size of your workplace. I’ve noticed, however, that there seems to be a certain assumption that automating deploying and maintaining servers with something like Chef is most useful for huge outfits with dozens, hundreds, or thousands of servers. Undoubtedly it’s great for them, but I think that same kind of automation is important even if you’re the only ops guy at your company and you only manage a small handful of servers. In fact, it may almost be more important, but there are a few considerations and best practices that are different with a small shop that I’ve noticed.
My experience is with Chef, but I imagine this applies to other automation systems like Puppet. These considerations would also apply to big places too, but I think they’d be easier to avoid with bigger staffs, and more servers may force better behavior.
The most important best practice with using Chef in a small shop is: Don’t cut corners with your cookbooks. If you only have a few servers, it’s easy to throw your hands up and leave important steps out of your cookbooks because it’s easier to just do it by hand when you deploy. I cannot stress how important it is that you not do this. It will bite you on the ass down the road, and you’ll waste a bunch of time tracking the problem down before you realize that you’ve been running ntpdate by hand rather than using the cookbook, or plunking some cron jobs in by hand rather than adding them to your cookbook. Even if you remember everything, eventually you’ll find a situation where you wish you had everything in place. Just bite the bullet and get it over with. Build your own packages if you need to.
I know better than anyone how easy it is to not follow this practice. Legacy setups are the hardest to be good about following the best practice here, but you really need to. Your cookbooks for legacy apps and systems don’t need to be beautiful examples of cookbook writing, but you’ll appreciate the effort later. Trust me.
“Grab-bag” cookbooks can be handy when you’re chef-izing an existing setup. Those little packages that are important but don’t need much to get them installed? Shove them in. If, later, you find they need some custom work, split them off to their own cookbook.
This isn’t as important as the above practice, and it applies to everyone, but if you’re on your own or working with one other person, you may need to Work hard at keeping your roles neat. Now that chef’s had environments for a while, it’s even easier. Rather than having separate roles for different subtly different installations, like “rails_staging” vs. “rails_production”, have a “rails” role and use “staging” and “production” environments to hold the differences. Some people prefer to use master cookbooks to keep these straight, but the point remains: minimizing the number of different versions of the same thing is good. Otherwise you may find, using “rails_staging” and “rails_production” as examples, that changes in one role may not make it to the other and strange differences may creep in that you don’t notice. The reason this is a small shop best practice when it applies to everyone is that without others to keep you in line, it’s easy to dash off that separate role when it isn’t really appropriate, and then you have all these extra roles that are just barely different from each other hanging around.
Resist the urge to be too clever for your own good with your cookbooks. Someday, some poor bastard might come along and have to figure out what on earth that cookbook you wrote is doing. It might even be you, so don’t make it too hard to figure out.
Finally: Make time to make sure all your Chef stuff works the way you think it does. The time to make sure that all your Chef ducks are in a row is when you’re sitting around bored, not when you’re single-handedly rebuilding your infrastructure in a different data center in a race against time before a hurricane hits the city currently housing your infrastructure. It may or may not make it run any faster, and events totally out of your control may strike you down, but at least you’ll feel a little less stressed while pulling everything together.
All of these small shop best practices boil down to “discipline”. They’re practices that would be good for any Chef user to follow, of course, but for those lonely souls out there who are the entirety of the ops team by themselves it’s doubly important. It kind of sucks, but if you’re by yourself you need to keep yourself disciplined to get the most out of Chef and not take the easy way out because it’s quicker at that moment. It will pay off later though when you really need it, and you don’t have to worry about installing that one package on all those servers manually when you have to rebuild all your webheads because the data center you had everything in went away.
comments powered by Disqus