emcooke

Twilio has a service-oriented architecture internally. We leverage a variety of application stacks for different services.

Several people have asked for additional details. We just posted a quick follow-on:

[UPDATE] A central theme of the recent AWS issues has been the Amazon Elastic Block Storage (EBS) service. We use EBS at Twilio but only for non-critical and non-latency sensitive tasks. We've been a slow adopter of EBS for core parts of our persistence infrastructure because it doesn't satisfy the "unit-of-failure is a single host principle." If EBS were to experience a problem, all dependent service could also experience failures. Instead, we've focuses on utilizing the ephemeral disks present on each EC2 host for persistence. If an ephemeral disk fails, that failure is scoped to that host. We are planning a follow-on post describing how we doing RAID0 stripping across ephemeral disks to improve I/O performance.

Our main infrastructure is deployed on AWS but we have capacity at several cloud providers for load-balancing, redundancy, etc.

Yup, good idea. We set up an ngnix proxy to cache the page while the blog hosting provider fixes their server.

Ironically, this highlights one of the main issues we discuss in the post!

The Twilio Engineering blog is hosted off an external Wordpress site with a single IP that's forwarded from ngnix load balancer pool. Since the load balancers assume that the external service can fail, they won't tied resources blocking access to other parts of the site.

Hope you enjoy the post :)

-Evan Twilio.com

We just enabled caching on the ngnix proxy to the external site hosting our Wordpress install for the engineering blog. Hopefully that should help performance.

-Evan Twilio.com

Hi Anthony, Twilio service status available via our public status page http://status.twilio.com/ We communicated degraded international SMS service to customers on August 16.

You bring up a good point that the information might not be readily discoverable. We'll work to make the status page more findable and to extend the API (http://status.twilio.com/documentation/rest) with features such as RSS to let customer subscribe up-to-the-minute status information.

For those wondering about the technical aspects of this decision take a look at this thread: http://getsatisfaction.com/twilio/topics/international_sms-1...

When we launched the Twilio SMS Beta we tried hard to support sending SMS messages to both US and International destinations. When there were problems, we worked with our customers to collect forensic data on hundreds of carriers worldwide and pass it to our carriers partners to debug.

At Twilio we are dedicated to working with top quality carriers and technology. After months of working to fix problems, were not able to deliver the reliable International SMS service our customers have come to expect.

We apologize for any problems this has caused for our customer and we'll work to bring back International SMS service after were able to deliver on the quality we do the rest of Twilio services.

Cheers, -Evan

CTO and Co-Founder

In a 100% ideal scenario an organization would only need developers as every possible failure/problem would be covered and handled by automation.

Obviously the real world is different. New code is deployed that has bugs, there are unpredicted events, there are complex failure scenarios that are difficult to automate etc.

A DevOps engineer may write automation software in conjunction with developers to automate the operational aspects of business logic. Thus, it's a partnership between the DevOps engineer whose metrics are driven by availability/reliability/scalability/security and the developer who is trying to attain some business objective.

Can't someone whose role embodies a certain culture and philosophy not have a title that fits that goal? How would you define DevOps?

Thanks for the pointer to OpSec term. DevOpsSec... I sense a tshirt in the making?

Correct. Stashboard is simply a lightweight frontend display for your API/service status. It has a GUI and REST API that allow you to update status information. Using the API, one could wire Stashboard into Nagios or any other alerting system.

You might check out http://venmo.com/ They have a dead simple SMS payments platform.

-evan (@twilio)

We provided an brief overview of how Twilio uses AWS here http://www.slideshare.net/twilio/twilio-voice-applications-w...

Props to Dave. All the flowery language, pictures, and hyperbole generate great discussion and traffic.

Dave is one of the most savvy investors and marketers around. When we were first pitching Twilio no one seemed to understand "marketing to developers" but he got it while half asleep from an all-nighter.

If you curious about Dave's geek cred, scope his linkedin profile: * Investment Program Director at fbFund Incubator * Investor at Bit.ly * Investor at DailyBurn * Advisor at Life360 * Startup Investor (FF Angel Fund, fbFund Incubator) at Founders Fund * Advisor, Investor at Twilio * Organizer & Founder at Startup2Startup * Advisor at 750 Industries * Advisor, Angel Investor at SlideShare * Advisor, Angel Investor at TradeVibes * Advisor, Angel Investor at EventVue * Advisor, Angel Investor at Oortle * Angel Investor at RichRelevance * Advisor, Angel Investor at KISSmetrics * Advisor, Angel Investor, Boardmember at TeachStreet.com * Advisor, Consultant, Angel Investor at Mint Software * Advisor, Angel Investor at Mashery * Advisor at Kiva.org * Co-Founder, Boardmember at Silicon Valley Microfinance Network * Advisor, Angel Investor at Simply Hired * Angel Investor at Canopy Financial * Startup Advisor / Angel Investor at 500 Hats LLC

And, if you head hasn't exploded from the colors in his blog check out his presentation style... in. your. face. http://www.slideshare.net/dmc500hats/startup-metrics-for-pir...

maybe i am too stupid but bookmarks are just too complicated for me... they are always on the wrong computer, organizing them is a pain, and i just don't trust them to a 3rd party to keep them. google always seems easier :)

because i found myself typing "hacker news" into google instead of typing all 20 characters into the url bar.