PagerDuty: You Have The Power

http://blog.pagerduty.com/2012/09/you-have-the-power/

by mamatta • 14 years ago

38 20 14 years ago

blog.pagerduty.com

We're lucky in that we have so many technically astute users, who've already built cool stuff in spite of us. I'm excited to see what they'll do now.

lanstein• 14 years ago

At Loggly, we integrated our alerting app with PagerDuty, and open-sourced the whole thing if anyone is interested:

https://github.com/loggly/alertbirds-community-edition

DanielRibeiro2• 14 years ago

TL;DR New Pagerduty API docs at: http://developer.pagerduty.com/

olidb2• 14 years ago

Datadog hearts PagerDuty. We use it every day. And sometimes nights too...

And here's its awesome API in action in Datadog -http://www.datadoghq.com/2012/07/pagerduty/

bradgessler2• 14 years ago

Is PagerDuty worth it for $18 per month? For a 5 person team its $1080 for an entire year to do something that a small script + Google Calendar could take care of. Maybe I'm missing something? I'd love to hear a story about how a feature of PagerDuty saved somebody's butt.

dredmorbius• 14 years ago

Do the math.

What's your downtime cost?

What's your implementation (and testing) cost of your home-brew alternative?

How are you going to document your homebrew alternative?

Odds are good that your DevOps hire will be familiar with Pagerduty (or something like it, though probably worse).

PD works, is straightforward, has plugins to other alerting tools (Nagios and New Relic off the top of my head), has multiple contact methods (SMS, MMS, eMail, phone).

Pagerduty is out there. It can't be bargained with. It can't be reasoned with. It doesn't feel pity, or remorse, or fear. And it absolutely will not stop, ever, until you are dead. Or your site comes back up.

Just a happy customer (despite their core competency being waking me up at 4am -- but given that that of itself is a momentous task, that they're good at it says something).

jedberg• 14 years ago

> Is PagerDuty worth it for $18 per month?

Yes. We use them at Netflix.

The real value that they add is they take care of the edge cases you haven't thought of yet. :)

moe• 14 years ago

Sadly they do an embarrassingly poor job at it.

The text of their notification e-mails is ridiculously bad ("You have 1 triggered Incident", not customizable). The mails don't even contain direct links to the incidents, much less links to ack/resolve them...

There are no notifications about status updates at all, not even an e-mail. People still call each other all the time to synchronize about "Are you on it or should I take it?" and "Is it resolved yet?".

etc.

I'm still baffled how they managed to ruin such a trivial product so thoroughly.

Needless to say we went back to a simple shell-script with TSV schedule (and that amusingly works better in most regards).

bradgessler2• 14 years ago

Could you elaborate on one of these edge cases? A specific anecdote would be great.

rdl2• 14 years ago

I'd pay $50/mo/user for monitoring and alerting which was:

1) quite reliable (although not as reliable as my production app)

2) totally uncorrelated from failures in my own infrastructure

3) tuneable to eliminate false positives, get good confirmation from users that they've taken issues. An iOS app is the obvious solution here, smarter than IVR + SMS.

Even better:

4) tied into my ticketing system

5) tied into my config management/ops system, so I can let people have simple app/web tools for "roll back! roll back!", etc. At 3am if you get paged, it is nice to have good tools for that kind of thing.

I haven't found anything which is #1-3 without roll-your-own, and #4 and #5 I've never even built internally (but may this time).

If PagerDuty really is #1-3, then it is a bargain at $18/mo.

berkay3• 14 years ago

This pretty much sums up why we've built OpsGenie ( http://www.opsgenie.com )

- a reliable notification system with multiple notification channels (iPhone/Android push, email, SMS, voice, etc)

- integrated with IT management systems (monitoring, ticketing, etc.)

- can provide full context with the alert like performance charts, change history, etc., and not just "something went wrong"

- enables recipients to initiate investigative/corrective actions, communicate with others, etc.

We've build iPhone/Android apps to take advantage of push notifications as well as ability to present rich information and configurable user actions.

I'd love to get your feedback and collaborate to develop it further to meet type of requirements you mention above.

Disclaimer: I'm one of the founders of OpsGenie.

patio11• 14 years ago

Is PagerDuty worth it for $18 per month?

If you ever find yourself asking this question, consider the follow-up: "Is my business sought by PagerDuty?" If the answer to your question isn't screamingly obvious the answer to my question is.

n.b. This is OK. In a world with X customers and Y firms there are X x Y possible commercial relationships and the overwhelming majority of them will never happen. That's the nature of the beast.

DanielRibeiro2• 14 years ago

Note that Pagerduty also has a $9 per month plan (up to 3 users), aiming at startups...

jauer• 14 years ago

Yes. Every penny ($90/mo). We've been using them at $SMALL_ISP since September 2010. The scheduling, reporting, and flexible alerting has been well worth it to us.

Before PagerDuty we were using Nagios with a modem dialing out to pagers. Later we went to Nagios with email to SMS gateways. We had "problems" with email to SMS delivery and no way to tell if it was a bad employee or flaky cellco gateway. We also had problems with scheduling on call rotation & overrides for vacation.

The escalation/accountability & reporting let us weed out a bad employee that was ignoring pages. We suspected it but phone call delivery with confirmation in Pagerduty ensured there weren't any plausible excuses.

Most importantly it makes scheduling oncall rotations painless.

Yes, you can do all this in plain old Nagios with enough time or hackery. We reached a point where we couldn't waste time on that kind of hackery (changing alert schedules & escalation every time someone went on vacation, google calendar to nagios config scripts) and needed it to just work.

These APIs are just icing on top. I have two projects that will benefit immediately: 1. Our customer service manager has been asking for a easy way to tell who is on call and we didn't want to give her a pager duty login. Now I can make a simple "who is on call?" dashboard so the call center knows who to call if there's a problem monitoring missed. 2. With the Integration Triggers we can open Jira tickets for incidents. This will streamline workflow.

kapitalx• 14 years ago

You should checkout the following blog post as a possible solution to item 1 on your list. You can setup a phone number that will page the on-call person.

http://blog.pagerduty.com/2012/08/how-to-provide-24x7-phone-...

shaggy• 14 years ago

PagerDuty is a fantastic service. The flexibility in scheduling on-call rotations (they added a custom schedule for my team at my last company), the support for integrating with lots of different monitoring platforms and the generally fantastic support make it well worth the monthly cost.

Raphomet• 14 years ago

Chiming in - PagerDuty has saved many butts at Airbnb, too. We love to hate getting pages from you guys.

PagerDuty: You Have The Power

Related Stories