The “Run Less Software” philosophy

https://blog.intercom.com/run-less-software/

by yrsamrith • 8 years ago

210 96 8 years ago

blog.intercom.com

mooreds4• 8 years ago

Great set of principles with some good examples. It's worth noting that leveraging third party providers is not fairy dust--choosing that path brings with it constraints and you may have to spend time working around those constraints.

I wish he'd talked more about the four armies and how they played into the choices Intercom has made.

pg_bot• 8 years ago

Does/Did Intercom use MySQL, PostreSQL and MongoDB at the same time?!

mariokostelac2• 8 years ago

No :). Started with Postgres long time ago, switched to MySQL. Added MongoDB into the mix. Removing MongoDB from the mix.

Traubenfuchs• 8 years ago

Do you have a "Why MongoDB didn't work out for us." writeup?

Gorgor• 8 years ago

What was the reason for switching from Postgres to MySQL?

jmspring2• 8 years ago

When I read "Outsource undifferentiated heavy lifting", I was skeptical, but outsourcing infrastructure and components to established companies makes sense (especially the list mentioned). Startups should not outsource to random body shops unless they either have experience doing such or have specific targeted/quantifiable/definable needs that a specialist can help with. Outsourcing, in general, sucks more time and money than the benefit it brings - specifically speaking to job/body shops.

wpietri• 8 years ago

Yes, agreed. For startups, writing software should rarely fall in to the "undifferentiated heavy lifting" category; it should mostly be in "enduring competitive advantage", either seeking it or creating it.

For very early stage startups, I think outsourcing software development can sometimes have merit. If the main point of the software is answering the question, "Do we have a real business here?" then if there isn't a technical cofounder I think outsourcing can be worthwhile as long as you assume that all the outsourced code will be thrown away.

But as soon as the founders are sure it's a real business, I think at that point you have to toss out the oursourced stuff and start building the real code base. Then you're reasonably sure that some of the code you're writing is going to be delivering customer value over the long haul, so it's worth investing in sustainability.

akditer• 8 years ago

And you think thousands of companies didn't do their math before outsourcing.

GarvielLoken• 8 years ago

Is that a joke?

jmspring2• 8 years ago

I've seen management push for outsourcing simply on the basis of having worked with a particular company/group before when there was no need/sense. It's about knowing the trade offs and making the right decision for the company as a whole not due to personal preference.

And many companies used to make outsourcing decisions based off of simple head count cost and didn't factor in the extra time/hours/delays due to integration/poor work/other factors.

hw• 8 years ago

Not every company is like Intercom where they have enough funding to spend on 'outsourcing undifferentiated heavy lifting'. The only competitive advantage they have is that they are well funded and they have budget for marketing.

There are other software companies out there, smaller and leaner than Intercom that are eating their lunch by offering similar or a better product at a lower cost. Those companies aren't 'outsourcing undifferentiated heavy lifting' but rather have a lean and talented team that's able to do the heavy lifting themselves without sacrificing on product development.

chynkm• 8 years ago

Good read and info

Sukotto• 8 years ago

My readability bookmarklet doesn't remove the vertigo-inducing animated background. So I'm going to have to pass on reading this one.

Will someone please post a summary?

askvictor• 8 years ago

Annoying though it is, it's only on the title; the article text is readable.

chrismorgan• 8 years ago

And the <hr> elements between sections.

I also find that type of animation really annoying, even if it didn’t cause problems due to its high resource demand.

As a rule of thumb, I’d say animations should only be used for short-term things. Eternal animations will make a substantial fraction of people’s devices heat up, have fans speed up, &c. I think it is probably most.

And in the case of this article, animations don’t even fit with the art direction which is otherwise delightful.

hn-52• 8 years ago

If you're using Firefox, reader view works well (the note paper icon towards the right side of the url bar).

uniformlyrandom• 8 years ago

My adblocker blocks the whole domain, so I am also waiting for someone to post a summary.

andyfleming3• 8 years ago

It's because Intercom is sometimes classified as marketing software. Unfortunately their blog is hosted on their domain which is blacklisted by most adblockers.

dredmorbius• 8 years ago

The console browser w3m does fine. Others should work similarly well.

jtth• 8 years ago

Kinda wish my browser would follow your advice; the blog post is over 10 megs.

hueving• 8 years ago

Keep in mind this is an article extolling the virtues of outsourcing critical components of your business by... a company that you outsource critical customer interaction components to.

codinghorror• 8 years ago

that's right and instead you should outsource them to us

TeMPOraL• 8 years ago

Oh my God, not just any company, the company responsible for those annoying chat popups on websites?

curun1r2• 8 years ago

To be fair, if you want to annoy visitors to your site and make them hate you, you should probably choose to outsource that functionality to Intercom. Annoying your visitors may be dumb, but rolling your own chat widget would be both dumb and time consuming.

blauditore2• 8 years ago

Ah, I already wondered why my ad blocker has a rule for that very domain and tries to prevent loading the article. Now it makes sense.

avip2• 8 years ago

Just to set things straight - you can use intercom without the popup. And on a more poetic level, a product can be annoying and useful simultaneously. Like a washing machine.

weego• 8 years ago

I love these things. SaaS companies who aren't explicitly clear about what their product does or it's limitations with pricing that is all contact us for a quote then BING "how can we help you today?". I have no idea because you don't provide a single point of reference for any of the things I need to evaluate before I care if you exist.

tobr• 8 years ago

I opened the link and only skimmed it, but I left the tab open in the background since it seemed like an interesting read to return to later.

Then they started to blink the tab title.

It's one thing that you do irritating stuff to get my attention when I'm actively on the website, but when I'm doing other things? This is right up there with popunder ads, blocking right click, and autoplaying sounds.

Instant close.

notyourday4• 8 years ago

My adblocker looked at me funny when i tried to click on a link.

dfabulich• 8 years ago

I don't see why AWS is alone in "Tier 1" on a list that claims to prioritize standard technology and outsourcing undifferentiated heavy lifting.

The post gives an example of switching from self-hosted MongoDB onto AWS DynamoDB. But AWS DynamoDB is proprietary technology, not standard. Amazon doesn't even claim that DynamoDB is standard; they argue that DynamoDB is the best in the world, worth the premium in price. It's not at all "undifferentiated heavy lifting."

It sounds like they made the right decision with DynamoDB, but they did it by violating the pillars of their own philosophy, adopting a differentiated proprietary solution for their scalability problems.

curun1r2• 8 years ago

It's the "self-hosting" part that's the "heavy lifting" for MongoDB. They could've also chosen a hosted Mongo service, and that would've worked too. The idea is to spend time focusing on your own product and do as little as possible outside of that core competency. Setting up, optimizing, monitoring, upgrading and everything else that goes into running your own database, however standard, falls outside of that core. The more you can do to avoid doing work that isn't directly building features and experience for your customers, the more you'll be successful.

By undifferentiated, they mean outsource anything that doesn't differentiate their own product offering. Build things that give you a competitive advantage over other products in your space. Outsource everything else because wasting work on it won't help you get customers. Of course DynamoDB is differentiated for Amazon, but it's undifferentiated for Intercom...it's just a persistence strategy that customers should never know about.

scarface74• 8 years ago

DynamoDB is not even a good replacement for Mongo. DynamoDB only lets you index two (?) fields and everything else is a full table scan.

At least go with a hosted Mongo solution.

mooreds4• 8 years ago

Actually out can have multiple indices, but they are eventually consistent:

https://docs.aws.amazon.com/amazondynamodb/latest/developerg...

ciaranlee4• 8 years ago

It’s certainly nowhere near as full featured. But in our case it is entirely appropriate. Over time we simplified our use of MongoDB to the stage where it’s effectively a k/v store. Swapping that simple pattern over to DynamoDB was easy, saves us a ton of money, and means we get to shed a ton of operational responsibility.

Also, you are incorrect about only being able to index two fields in DynamoDB. Though we luckily don’t need anything other than the primary key.

rich_archbold6• 8 years ago

Thanks for taking the time to read our blog post and for sharing your thoughts!

For us, the concept of “standard” technology is not about whether it’s proprietary or open source, more that we have a “standard way of doing things. So in this case it was moving toward saying, “hey, if you’re an engineer at Intercom, and need a scalable key-value store for the thing that you’re building, we think you should lean toward using DynamoDB, rather than any of the other technology options, because that’s the one that we’re committed to making a first developer experience for internally for the long term”.

The reason for having AWS alone as a Tier 1 option is similar. We’re already very heavily invested in making developing on top of AWS a great developer experience at Intercom, and we’re committed to doing so for the long term. Therefore we think that it makes sense for us to advocate for always exploring relevant offerings that put out.

I recognize that I could have elaborated on the philosophy a lot more, there’s always a tension between writing too much and too little. And the philosophy is just that, not dogma and not right or wrong, just trying to give insight into how we think.

Thanks again for the feedback! Rich.

dajonker2• 8 years ago

Somehow the recommendations feel like you are making them to the reader, and in that regards it doesn't feel right to just recommend AWS. Other public clouds provide equivalent or sometimes even better services.

andyfleming3• 8 years ago

"We quickly discounted this as it didn’t fit with our three pillars of Run Less Software and decided to consolidate on MySQL, specifically on AWS MySQL. As we were able to benefit from the pace of AWS innovation (and adopt AWS Aurora when it came along) we got 5x throughput at a 30% cost reduction."

It's worth noting that Aurora is compatible with PostgreSQL (as of Oct 2017) [1].

1. https://aws.amazon.com/blogs/aws/now-available-amazon-aurora...

venuur• 8 years ago

The indented quote renders as a code block and isn’t readable on mobile. I like your reference, so it might be nice to change to formatting for easier reading on mobile.

andyfleming3• 8 years ago

Just updated it. I thought there might be a way to apply blockquote styling, but the traditional markdown syntax didn't work.

jve• 8 years ago

uBlock has blocked this page from loading due to this filter: ||intercom.com^ Peter Lowe’s Ad and tracking server list

eeereerews• 8 years ago

You've missed out on a charming popup bubble offering you a... customer retention start kit, was it?

andreareina3• 8 years ago

Anybody know why the domain is on one of uBlock Origin's blocklists[1]? The styling is annoying as all hell but it doesn't seem to be an ad server.

[1] https://pgl.yoyo.org/adservers/serverlist.php?hostformat=hos...

TAForObvReasons• 8 years ago

Ghostery has it marked as a "Customer Interaction" tracker, which apparently

Includes chat, email messaging, customer support, and other interaction tools.

https://ghostery.zendesk.com/hc/en-us/articles/115000740394-...

The summary page has some privacy details:

Data Collected:
Anonymous (Browser Information, Cookie Data , Date/Time, Page Views )
Pseudonymous (IP Address (EU PII))

https://apps.ghostery.com/en/apps/intercom

andreareina3• 8 years ago

Yeah apparently they provide chat popups, didn't realize since reader view killed it. Thanks.

blauditore2• 8 years ago

This company provides those contact chat popups (one of which you can see on that page too), so that's likely the reason.

andreareina3• 8 years ago

Thanks. Reader view killed the chat popup, so I didn't notice anything untoward.

Nullabillity• 8 years ago

Because uBlock° is an annoying-crap blocker, not just an ad blocker.

_raul• 8 years ago

Related reading: "Ephemeralization", by Adam Wiggins (Heroku co-founder) https://adam.herokuapp.com/past/2011/4/7/ephemeralization/

avip2• 8 years ago

I feel a strange dissonance between the fantastic write-up, with which I 100% agree, and the ever so annoying intercom product... oh that horrible popup. What a terribly effective product though.

antirez• 8 years ago

Arguing simplification and then using DynamoDB, that is, opaque software handled by somebody else, does not make any sense or useful argument IMHO. It's like saying, we don't want to handle all this stuff, let somebody else handle it so that we can just be users. This is fine if is good for your company, but kinda negates the initial premise: you have a simplification problem if you own your infra, otherwise what's the point? Somebody else is handling all that for a premium price, I bet you have less software to handle.

ciaranlee4• 8 years ago

I’m humbled that you read our post! We love your work, and heavily use Redis at Intercom. Thank you :)

The DynamoDB move is certainly a tradeoff, but for us it really makes sense. Moving to it will allow us to:

- scale out this storage layer trivially easily

- shed a ton of operational responsibility

- save lots of money

There are certainly downsides to moving to a proprietary database run by a single proivider, but we’re happy to make that tradeoff.

Also, we’re absolutely not recommending that people move from MongoDB to DynamoDB as some general rule. It just made sense for us at this time.

pvsukale3• 8 years ago

And that is why I use rails and JQuery for my side projects. It is boring but I can pull it off on my own. If I run into some problems there are thousand already who have had same problem.

mooreds4• 8 years ago

Yes! Boring stable tech is sometimes the right answer. I've heard you should either be pushing business boundaries or tech boundaries every time you solve a problem. But pushing both is too risky.

vbsteven2• 8 years ago

The article is mostly about outsourcing "heavy lifting" like database hosting to cloud providers and non-critical software to SaaS providers. I agree with the article when it comes to database hosting/infrastructure but I would like to see more discussion about outsourcing to SaaS.

Outsourcing to SaaS means trusting your (and your customers) data to lots of different companies. And in the case of SaaS those companies could be small software companies or even one-man operations. This is not very compatible with upcoming data privacy regulations like GDPR. And I suspect that more and more companies are going to be looking at a model where they purchase/license/outsource non-core software but host it themselves in their own datacenter or cloud accounts.

The solution I am working towards in my current project is to build the software from day 1 with dual-mode operations in mind. Easy to host as SaaS but with the option of running Standalone on customer premises with the same codebase. Just different configuration/modules.

Cthulhu_2• 8 years ago

Well since this article is from Intercom, I can share that we outsourced our support / chat / things to Intercom, instead of (what was also proposed at the time) building our own.

mooreds4• 8 years ago

A company for which I worked also leveraged intercom. I found it quite useful. Especially in the way it let non technical team members set up rules based customer interaction around onboarding and feature discovery.

See more of my thoughts here:

http://www.mooreds.com/wordpress/archives/2872

rushabh• 8 years ago

A great opportunity to move to open source. It will help you not only have more control on your assets in case of privacy and GDPR, but also avoid vendor lock-in.

notyourday4• 8 years ago

I find it hysterically funny that companies think that operating their own mysql, postgresql, redis or mongodb is "a difficult task". It is like a home owner deciding that knowing how to operate their own toilet is a difficult task that must be outsourced.

Oh and before someone says "we outsource fixing our toilets to plumbers" - in this case we are not event talking about replacing a flap, we are talking about being able to use a plunger.

ape4• 8 years ago

You're outsourcing sewage treatment when you run your own toilet. But, yah, I agree running a database on a computer is pretty easy.

vbsteven2• 8 years ago

Setting up a single postgresql is not that difficult indeed. But in a production scenario with decent uptime guarantees you are going to at least need master-slave replication, make sure they fail over when the master goes down, have backups so you can do point-in-time recovery, rotate and ship the logs somewhere, apply security patches, and more.

Sure, I can do all that by myself but that takes time, time that is better spent serving my clients instead of setting up yet another replicated db. It takes me a few minutes to setup an Amazon RDS instance with all that enabled out of the box.

ciaranlee4• 8 years ago

Operating such services in a fairly static production environment is indeed simple, though in many cases I suspect it’s wasted effort that could be better spent elsewhere.

In our (Intercom) case these tasks are not simple.

We are lucky to have had exponential growth in most metrics (including database throughput) for the entire 7 years of our existence. This is a harsh environment for datastores! A 99.9% SLA makes it even harder.

We are also nowhere near finished building out our vision for the product. We prefer to focus on that instead of operating data stores, which we can pay someone else to do.

tetha• 8 years ago

Mh, my current company does this, and it's a real pain and a massive innovation showstopper. It's nice as long as you have a single application and a single database. That's easy to do on premise.

Then you need search, so you either build the search yourself, or build on solr/elasticsearch or anything else building on lucene. Then you need a cache, and suddenly you need to worry about cluster setups, choose a cache implementation. Then you end up with a use case which would be easily solved with a persistent queue, or neo4j or whatever, but your on-premise solutions block you.

It sounds simple to have n different backend solutions, and to offer a binary choice - everything built-in and everything not-built in. But usually you end up with about n! * #on-premise different setups and they all end up with their own classes of bugs and difficulties to setup.

zerogvt• 8 years ago

Nice piece with lots of takeaways but I think that they tend to lock themselves in AWS. This certainly adheres to the Run Less SW paradigm but they might suffer a vendor lock later on.

rich_archbold6• 8 years ago

We've deliberately chosen to accept vendor lock-in with AWS. I probably should have made that explicit in the blog post.

We believe that the value we get from going all-in on AWS is greater that the potential negotiating power you may get by staying higher level and having the illusion of low cost switching to GCP or Azure.

This also accepts the risk that if AWS goes down, we go down.

I think both of those are acceptable risks/tradeoffs at this point.

mbid• 8 years ago

"Here, read my ramblings about the "Run Less Software" philosophy on my blog that loads 10MB of data for what could be a plain html + basic css page, among which you'll find 800,000 characters of javascript code to execute. And that's not even counting 3rd parties, isn't that great?"

This is obviously marketing for the author's company. What do the four armies have to do with the point the autor's trying to make? "Hey, you should run as little software as possible. But first let's analyze the professional IT landscape with an analogy that does not make anything clearer but allows us to embed fancy pictures and videos."

javajosh• 8 years ago

Yes, the 4-armies analogy was a remarkable non-sequitur. I can't help but think he introduced it to talk about the hidden perils of "standard solutions" and "vendor lock-in", but then abandoned this idea when his conclusive paragraph brags about solving a problem with AWS Aurora (a cloud RDMS) and AWS DynamoDB (a cloud mongo-ish).

This trade-off between (apparent) simplicity and vendor lock-in is a fundamental force in the industry, and is worth talking more plainly about. (Another big area where this affects people is authentication - it's easier (for users) to use Google- and Facebook-backed OAuth, but what are the trade-offs involved? Does this increase my vulnerability in any way?)

I liked the idea behind this article, but it's definitely a missed opportunity. And yeah, as something of a web-minimalist myself the ridiculous page-weight was a rather ironic touch.

ausjke• 8 years ago

exactly what I felt, 'run less software' page seems is 'running a huge chunk of stuff' itself, how ironic, not to mention it's tediously lengthy, gosh.

SaturateDK• 8 years ago

1.6MB for one image of a map, that alone is a huge factor in the size. Unfortunately it's the state of many websites.

shadowtree• 8 years ago

Bit of a shame as Intercom has been doing Content Marketing right for a long time. Older stuff by Des Traynor was pretty good though (and entertaining, his talks about Product Strategy are hilarious). Might have switched to a staff writer, hence the genericism (i hope they don't let engineering write articles on their own).

I guess that is the peril of content marketing in a narrow field - eventually you run out of fresh takes but still need to attract eyeballs. Hence the Buzzfeed-style headlines.

jkaljundi• 8 years ago

Initially Intercom was all about being great in a niche (on-site chat messaging), doing one thing well, focusing. Then growth came, demands for expansion, and so their mantra switched to all-in-one suites. So did their content marketing.

wuliwong• 8 years ago

If all that bloat is generated because they are using third party frameworks or plugins and all they really are doing is writing the copy of the post itself then it still basically fits with the narrative.

wink• 8 years ago

I think it was one of the most nice-looking blog posts I've seen in a while.

Yes, I know - wrong medium for presentation over content.

devxpy• 8 years ago

Exactly

rich_archbold6• 8 years ago

Hi! I’m Rich, the blog post author.

Thanks for the candid feedback :-)

This blog post is somewhat a written summarization of a 25 -> 30 minute conference tech talk that I gave at SRECON-EU last year. It was a real struggle to figure out what to leave in and what to leave out. I didn’t want to arrive at a blog post that takes 30 mins to read :-) So, I can see how the four armies and battlefield analogy might come across as disjointed or unnecessary. There is definitely some context missing from the blog post that was in the talk.

When I was putting together the original talk, the “four armies and battlefield” framing was something I felt was useful to help illustrate how competitive I feel the modern software business landscape is and thus how important it is that your company has the right engineering strategy to help them “win their own battle” / succeed in your chosen market.

Anyway, thanks again for the feedback, hopefully the above provides a little more insight on why I used the four armies framing.

Rich.

z3t4• 8 years ago

In early stage you should outsource as much as possible but as you grow and get competition - in order to lower margins you have to incorporate full stack expertise and control the supply chains. However contraditional to the software solo-entrepreneur who's time is free, which allows almost zero operational cost vs crazy burn rates of some startups.

kingkongjaffa• 8 years ago

Everything about this post and the company behind it, is a contradiction of any proposed "minimalism in software" philosophy.

Personally I have found growth and simplification by replacing the complex with the simple, not necessarily less.

But certainly less software that does many things.

For example, I might replace one monolith app with many command line tools that each do a single job well.

Embracing markdown vs myriad of closed note taking tools has been a boon.

dschuetz• 8 years ago

Opening that link my ad blocker is going crazy. I guess the website surely runs less software, huh?

jaequery• 8 years ago

This philosophy fits well with a new open source framework I've been working on that is focused purely on simplicity.

https://github.com/jaequery/jasis

The goal is to have a framework that's so easy to use that even non-programmers can understand it. I'd really love to hear your thoughts, especially anyone that is new to programming. @jaequery

asdf1234tx• 8 years ago

That article was a long winded way to say "Serve an underserved market efficiently, and execute sooner, better than competitors."

You don't say!!!!

ciaranlee4• 8 years ago

That is a very concise summary! I still think the examples and detail are useful to explain how we do that at Intercom, rather than simply issuing a statement :)

halfnibble• 8 years ago

This statement about software talent appears to be in direct contradiction with software development history:

"But as technology advances and provides us with more advanced software components it devalues and commoditizes what we’re working on and at worst makes us redundant."

The “Run Less Software” philosophy

Related Stories