HN user

ksmith14

36 karma

[ my public key: https://keybase.io/kylesm; my proof: https://keybase.io/kylesm/sigs/Pa_WqH2yKXzZ-qKpivxRyhIvfLj2z1ZJ9Ip0TtcgDUE ]

The postings here are my own and do not represent those of my employer, Fitbit.

Posts0
Comments13
View on HN
No posts found.

I was actually using Pushover before I switched to OpsGenie. I typically have my phone set to do not disturb overnight and Pushover didn't have a "critical alerts" option to punch through - OpsGenie did. I may revisit it since I can dredge up the integration code from my Git repo.

Google staffs SRE teams as either 8 in one location/TZ or two geographically distributed teams of 6 -- often some pairwise combination of U.S., Europe, and Australia to accommodate reasonable on-call shifts.

The on-call compensation varies depending on what tier of service they're offering. Tier 1 (5 minute response time) is 2/3 of your effectively hourly pay for on-call time outside of local business hours and 1/3 for tier 2 (30 min response time). Or time off in lieu.

I have a house with 3 levels and when I moved in, 2 of those levels had analog thermostats. I'd be watching TV in the basement with the heat set to 68, go upstairs to bed, and realize I'd left the heat up so I'd trudge back down stairs, change the temperature, and then go back to bed. I ended up replacing everything with Nest thermostats within a few months so I could adjust them from my phone if necessary. Then Alexa came along and I could just lay in bed and tell it to do things, which was even more convenient.

I also have kids that are terrible at turning things off, so I've stuck one smart bulb in each of their rooms and quietly conditioned them to prefer that light to some of the switch-activated lighting so I can just centrally switch them all off once they're gone or when leaving the house without trekking around.

Having lived at worked in Poughkeepsie at IBM for several years found it strange to see it on HN as well. I've since moved to Massachusetts but something my wife and I really miss about the Hudson Valley are all of the historic sites along the river (Locust Grove, FDR's house, Vanderbilt mansion, Mills mansion, Olana, etc.) that were often free and were great places to just go sit and visit, as you say.

It's good you learned that lesson about having someone else announce the server, but isn't always that simple. With things like Apache Aurora it would often announce jobs as soon as the process started. With something like Finagle, it tends to do it as soon as the component is initialized, instead of waiting until the server is fully initialized and ready to handle requests.

Something that I implemented at my last job and others rediscovered as part of my current job is to implement an "administratively up/down" API as part of the control plane and only have the server announced if it was "up." Decoupling the announcement from process start/initialization complete allowed us to roll out new versions of software in a disabled fashion and then "flip the switch" (red/black deployments). It also enabled us to take individual instances out of service without killing them, enabling developers to debug issues/anomalies more easily.

Load shedding/backpressure/rate limiting at various layers is also extremely helpful, whether at the load balancer/API gateway or at individual servers. That has saved our bacon numerous times.

This brief excerpt really does not do the Manager Tools' feedback model justice. All of the background has been stripped from it in order to fit in a pithy blog post and without that context it does sound jarring or shallow.

Over the past 3.5 years I've used the Manager Tools "management trinity" (including their feedback model) with my directs and found it works extremely well. If you've got a professional relationship with a direct and have briefed them on changes before enacting them, it doesn't come as a surprise and it's not seen/received as a "drive by."

I encourage folks to listen to a few of the episodes on feedback before judging it: https://www.manager-tools.com/2005/07/giving-effective-feedb...

In addition to Managers Tools they have a second podcast, Career Tools, which is geared towards any working professional. I recommend that one as well.

I worked for VMware from 2007-2014 on things other than the core virtualization products.

I think you’re overestimating the situation. There is a group that did have a fair number of CS PhDs to develop and maintain the core piece — the virtual machine monitor (VMM) but then there’s a large number of plain old everyday SWEs developing the bulk of vSphere and the desktop virtualization products.

I would expect VMW’s policy to apply to just about everyone. A few old timers that have reached the principal engineer or fellow levels might be able to avoid it but that’s a few dozen out of 20,000+

This is exactly where I've found it can be helpful as well.

At a prior job we created extremely large, complicated CloudFormation templates in JSON that could have been made much simpler with Jsonnet and the ability to do comprehension and some other tricks to keep it DRY. A coworker ended up writing a bunch of Python scripts to templatize the CF templates (ugh).

At my current job we ended up using Jsonnet as a configuration language for Apache Aurora jobs [1], which was also helpful when trying to make it easy for others to pick it up and create their own jobs from base templates.

That being said, it can be frustrating to get going with and for some cases I do find myself having to trace through things very carefully in order to to figure out exactly what's going on.

[1] https://github.com/benley/aurora-jsonnet

As a VMware employee from 2007 to (January) 2014 the only time I recall any fear from leadership or product teams about HyperV was in 2008 or so when it was about to come out and then for a little while after it first came out. We knew we had a several year lead over Microsoft and wanted to make sure we stayed ahead of them feature-wise. After 2009 I don't even recall people bringing up Microsoft or HyperV in conversation; we were more focused on competing with cloud platforms.