HN user

cparedes

119 karma

I'm a UNIX systems administrator in the Seattle area.

http://redbluemagenta.com is my website.

Want to contact me? cp[at]redbluemagenta[dot]com.

Posts7
Comments67
View on HN

Thanks BTW for understanding - that's my intention here, which is to call out bad companies for doing egregious sorts of things (along with me being pretty frustrated in general, which could've been in poor form but whatever)

There was never an interview where you meet with Canonical execs. There was an interview panel where I met with higher ups and whatnot, but not at the executive level. That was a completely, 100% hidden step in the process depending on where they think you lay in the hierarchy (probably decided after the interview - the job I was interviewing for wasn't originally written for a senior level hire.)

"Tool" here.

I don't blame them for not having a paycheck, I blame them for handling this incredibly terribly. If I had known there was even an exec-level review of my application, I would've acted otherwise. I don't burn bridges often, but given the amount of bullshit that exists in the tech industry, I'm okay with calling it out and risk burning bridges than to keep mum and allow shitty practices to continue.

Everything was assured - I turned down other offers either due to some bad timing or seemingly worse fit, related to waiting for this particular position to get back to me. Of course, no one should ever turn down other jobs unless you have an official offer letter in hand, but even then, an employer can rescind the offer even if you sign the paperwork.

Er, sure it does:

* No web UI. This may be a blocker for business folks who want to update a blog. This is a blocker for folks who are using devices like iPad's to update blogs (which is a huge reason why I moved away from things like Octopress and moved toward things like Ghost or Wordpress.) This may be a blocker for folks who don't always have the tools on whatever machine they're using at the time in order to generate the site.

* No imposed structure. It's easier to impose a web UI form that locks folks into certain workflows than it is to just give them a git repository with all sorts of stuff strung about, with no apparent structure other than the FS structure.

IMO, Amazon gets 'DevOps' right. It's mostly just called 'ownership' over in Amazon. (source: I used to work in Amazon as a systems engineer)

You still have specializations - SDE's, systems engineers, DBA's, etc. However, if you write code and it ends up in production, you are responsible for the proper function of that code in production. As a friend of mine put it in terms of developers who don't want to be on-call: 'what, you don't trust the quality of your code?'

DevOps is simply a nicer way of just saying, "own your damn code." The corollary to this is that the organization must help you in getting to that state where you can effectively own your code - this means collaboration (so that you build maintainable systems) and building tools that enable fairly frictionless code ownership.

Security is about mitigating risk, not about eliminating it.

Keep up with CVE's, don't provide a wide attack area (so lock down interfaces to your machine and don't expose much to the world), and keep blast radii as small as possible (so even if your machine does get owned, you can possibly restrict it so it doesn't automatically mean they gain access to other systems in your network.)

Oh, and model the threats to your network/application. Make sure you're securing against the right threat. As an example, anti-malware is wholly ineffective against social engineering - maybe it's more productive to train employees and make sure that each employee doesn't have total access to all privileged systems.

FWIW, if you do need ELB support right away (and keep in mind that you'll have to look at it from each server's point of view), you can probably use https://github.com/opscode-cookbooks/aws (and use the elastic_lb resource), and hook it into your own cookbook for setting up whatever ELB's you want (then set up your custom cookbook repo for your stack.)

(I'm trying to say that this platform is incredibly flexible, and you can reuse what's already out there. If you need support for X, Y, or Z, then you can likely write in support with Chef.)

Sort of, but not exactly. It's not API compatible with Chef server, and I believe you don't get the nice search syntax for grabbing node information across your fleet that match some kind of criteria. But you still do get info on your other machines, and you do get nice orchestration across your fleet.

You can use TCP/UDP/GRE for your traceroute packets with the -P switch:

traceroute -P udp <hostname>

This is super useful if you're probing a network that's hostile to ICMP packets - it's a lot more reliable than ICMP in many scenarios, from my experience.

The reasons that other folks have mentioned (static files, nginx is built for lots of HTTP requests), as well as basic load balancing between several 'worker processes'. Sometimes you'll want to have several app server processes running with several ports open, and have nginx load balance between all of those processes (then load balance again at a higher tier between all of the different nginx processes.)

Kick off Puppet/Chef runs? It's not too hard to automate these tasks, either (or even put a fancy interface around it, though of course this takes a bit more time.)

It's also a bit insulting when you say that we 'in fact [do it] by hand', because sysadmins, as a whole, strive to not do that. Fine, I can see myself logging into a box and typing 'chef-client' or 'puppet' by hand - but this is too much work to be done by hand. That's why we automate. I don't know why you have this idea that we simply want to make things harder just for job security.

I can't know for sure, but it's at least corroborated with quite a bit of research: https://www.google.com/search?q=usenix+configuration+managem...

I'm a fan of both of those systems, because it's easier to express system state with them (and clearer, too), versus using shell scripts or what have you. I'm definitely not advocating using it as a way to deploy software, but it's definitely possible to do so (I'd likely build system packages, throw them in an internal repo, and use yum/apt to ensure software packages are up to date, or at a certain version.)

It's not the only solution, though. I've worked at shops where there have been exceptional software deployment pipelines that are absolutely not tied to a configuration management system. I'm also not a fan of using Chef or Puppet for doing a git/svn repository checkout of software.

Again, I want to reiterate my point: it's still not about the tools, but about identifying the problem you need to solve and discussing possible solutions to the problem. There's no right answer.

Author here.

NT? (edit: you mean non-technical? I'm definitely not non-technical - I work as a systems engineer by trade.)

And, yes, you don't need to use Chef or Puppet to solve the software deployment problem. You use it to ensure system state, so you know you can throw software on top of it without any problems. (this can go on into a debate about golden images vs. configuration management, but I think this is a different discussion.)

That's probably true as well, though... I've been in shops without Chef or Puppet, and it's really not a big deal for me, either. The thing that stood out was the justification for not choosing Chef or Puppet, not really the fact that they weren't using it.

Author here. They do, though for a certain set of problems, you can get away with using things like Chef for coordinating lots of information between a lot of machines.

My point, however, was that tools should never be the focal point of solving a problem - if you recognize the problem first as a distributed locking problem, or as a configuration problem, then you can start deriving a solution (and maybe that solution might have Zookeeper in it, maybe you'll write a new piece of software, maybe you'll use Chef, etc.)

So, tell me - what would this tool offer over, say, RunDeck or mcollective? Why only use SSH keys? (What if I'm using LDAP across all of my machines?)

I'm not trying to knock your product - I really do hope you get Kickstarter funding! But I just wanted to say that, to me, as a systems engineer, I only see a pretty interface, but not much value add over the other two products mentioned above

I also feel that comparing Commando.io to Puppet, Chef, or even Capistrano is disingenuous - Puppet and Chef are configuration management systems that are meant to keep your systems 'in policy', and Capistrano is meant for repeated tasks over SSH (mainly for deploying software, but it can be used for other things too.)

I think the problem isn't inherently the class content (in 'soft' subjects), but how decent the professor is in letting students get by with weak arguments in their papers. I think there's some correlation between how much they actually care about the class, and how tough the class will end up being.

I took an English class in college that was probably one of my toughest classes in college (yes, about as tough as my math classes) - the professor and the TA's tore apart each paper ruthlessly, and judged them at a quality slightly below academic papers. I wrote a paper that was well formatted and had decent grammar in it, but it was summarily torn apart and was given a C on the paper, because my arguments in the paper sucked.

Also, I'd say that Philosophy is also a pretty damn tough subject in the 'soft subjects' of college. You're expected to be nearly as technical as a mathematician in your arguments, except you're writing essays/articles in prose, rather than with terse explanations and symbols. Philosophy took about as much time for me to study for as my math classes.

Seattle is crazy awesome for startups, but man, SLU has changed a lot since I last worked there a couple of years ago at a research lab (and I think it changed for the worse - way more expensive food, not much variety.)

Pioneer Square is very low key, has cheap eats, and has lots of startups.

Obvious troll is obvious.

As I've said in the past, the consensus is simply "use a CM system," especially if you have to maintain these machines for a while (or if you have 50+ machines to take care of.) It's much better than dealing with various shell scripts or whatever else.

Puppet 2.7 Released 15 years ago

Over at Seattle Biomed, we keep everything in the same branch (for now) - all of our nodes are defined in nodes.pp, and we have modules that install software/configuration files as needed. We include modules in the node definition that should comprise the service we want to run on the server.

Puppet 2.7 Released 15 years ago

Chef has decent documentation, but the last time I checked, it was a lot less polished than Puppet's documentation. With that said, the folks in #chef on irc.freenode.net are quite helpful, and it's possible that the documentation has improved since the release of 0.10.

Puppet 2.7 Released 15 years ago

I see where you're coming from - however, the great thing about CM tools is that they abstract a lot of the weird differences between UNIX's and Linux distributions. Defining a user in Puppet or Chef will create that user the same way, whether it's in RHEL, Solaris, FreeBSD, etc.

Puppet is also more lightweight than Chef in terms of the amount of dependencies it requires, given an already existing installation of Ruby (in fact, I think there's only one dependency: facter.) The only problem is that it comes with a lot less available language features out of the box than Chef, mainly because Puppet doesn't have a way to store and serve configurations of other machines out of the box without something backing the data store, like MySQL or CouchDB.

Puppet 2.7 Released 15 years ago

The consensus on CM tool choice is simply, "pick one that's comfortable for you and your team," and, "it doesn't matter which one you use, just be sure you're using configuration management." It's definitely better than dealing with shell scripts, since a lot of things you write in your CM language gets abstracted automatically for whatever operating system you're using (this is the biggest takeaway - you really don't want to rewrite shell scripts just because you've decided to add a different Linux distro or UNIX in your network.)

I've used both Puppet and Chef extensively, and they both do the same thing, they just go about it differently. (shameless plug of a blog post I wrote that compared the two: http://redbluemagenta.com/2011/05/21/puppet-vs-chef)

Also, it's almost always about how you approach configuration management - thankfully, with Puppet and Chef, you don't have to manage everything on a system in your CM right away. If you have to manage SSH authorized_key entries and it's getting annoying to do it by hand, then write a Puppet module / Chef cookbook to manage it for you. Then go on to manage /etc/hosts entries, then to deploying packages... you'll hit a point where you can reasonably rebuild a server from scratch without thinking about it.

There's other CM systems out there too: bcfg2, cfengine, etc.

I don't see anything wrong with specialization at all - it's just that college usually appeals to this notion of "well-roundedness", and even if it doesn't, I'd say that it might be great to try taking that weird underwater basket weaving class for shits and giggles, or an English lit class (outside of your requirements, IMO - you'll get a better experience in the classroom when you don't have a ton of people in the class trying to fill basic requirements), or whatever else.

I've personally taken a couple of film and English classes when I didn't need the credits - it doesn't help me one bit in my career, but at least it had helped me think of things in a different way than what I'm used to.