HN user

thinkbohemian

1,024 karma

Professional hacker for http://heroku.com former gowalla.com

Find me on the twitters @schneems

Posts48
Comments341
View on HN
www.schneems.com 11y ago

Recruiter Sniping

thinkbohemian
7pts0
www.schneems.com 11y ago

36% smaller Rails memory footprint through Benchmarking

thinkbohemian
3pts0
www.schneems.com 12y ago

Design Driven Tests: This is how I Roll

thinkbohemian
2pts0
blog.heroku.com 12y ago

Hacking Hack on Heroku

thinkbohemian
29pts3
www.schneems.com 12y ago

Why We Should (Absolutely Never) Build Software Like We Build Houses

thinkbohemian
2pts0
schneems.com 12y ago

Explain Shell Commands from your Shell

thinkbohemian
1pts0
blog.heroku.com 13y ago

Ruby 2.0.0 Now Default on All New Ruby Applications

thinkbohemian
17pts1
www.kickstarter.com 13y ago

Wirelessly connect all the things with Sapphire

thinkbohemian
2pts0
www.craigkerstiens.com 13y ago

Why I Blog

thinkbohemian
38pts16
schneems.com 13y ago

Heroku Engineer Engaged to Ruby

thinkbohemian
2pts0
gist.github.com 13y ago

Specify Connection Pool Size in Active Record

thinkbohemian
2pts0
www.codetriage.com 13y ago

Help Contribute to Python's Flask with Code Triage

thinkbohemian
3pts0
ruby.onales.com 13y ago

Ruby on Ales Streamed Live

thinkbohemian
1pts0
www.wired.com 13y ago

Why Coders Should Try Quilting and Origami

thinkbohemian
5pts2
rubysource.com 13y ago

Anatomy of an Exploit: An In-depth Look at the Rails YAML Vulnerability

thinkbohemian
50pts24
postgres.heroku.com 13y ago

Fork Your Application's Data (Not Just Your Code)

thinkbohemian
74pts35
amydoesntlai.tumblr.com 13y ago

Forget Recruiters: Hire Junior Devs

thinkbohemian
15pts4
schneems.com 13y ago

Would you like a Mobile App with That?

thinkbohemian
6pts0
blog.heroku.com 13y ago

Heroku Sunsets the Aspen Stack

thinkbohemian
19pts17
blog.heroku.com 13y ago

Sunsetting and Deprecation at Heroku

thinkbohemian
6pts0
hone.heroku.com 13y ago

Resque 1.22.0 Signal Handling Changes

thinkbohemian
1pts0
schneems.com 14y ago

ZOMG My Passwords are Insecure, now What? How to Change Auth Backends

thinkbohemian
3pts1
github.com 14y ago

View your Rails Routes without Rake using Sextant

thinkbohemian
1pts0
schneems.com 14y ago

Legacy Concerns in Rails

thinkbohemian
3pts0
devcenter.heroku.com 14y ago

Building an iOS Photo-sharing and Geolocation Mobile Client and API

thinkbohemian
22pts2
schneems.com 14y ago

Super Charge your Rails App with Rack Cache and Memcache

thinkbohemian
2pts0
schneems.com 14y ago

Quit Complaining About a Talent Shortage and Interview Better

thinkbohemian
3pts1
schneems.com 14y ago

Stack Overflow Needs Mentors

thinkbohemian
2pts0
engineering.gowalla.com 14y ago

So You Want to Hire an Intern?

thinkbohemian
7pts2
engineering.gowalla.com 14y ago

Gowalla Launches Engineering Site

thinkbohemian
7pts1

This program offers interesting features to a business but not s much for the traveler. Anyone who is spending 25+ nights a year in a hotel is going to be sure to be getting points for those stays. Points == status and status == more convenience when traveling. Beyond earning free nights, there's expedited check-in, private lounges with free food, gift baskets when you check in, etc. The caveat is that to get status most (or all) of your points need to be with the same company. This means most hardcore business travelers are very loyal to <program x>. This travel program doesn't have any kind of loyalty rewards which is a non-starter.

Other downsides of airbnb: booking isn't instantaneous as with a hotel (wait hours or days for confirmation from owner), checkin has to be pre-arranged and requires you get in touch with the owner (i.e. no 3am red-eye check-ins), each place you stay is slightly different amenities versus hotel chains which all have the same brand of pre-wrapped toothbrush when you forget yours. These might seem trivial but when you travel a bunch it's the little things that hurt the most.

Story: I once had to wait for 2 hours to check into an airbnb in SF since the owner was in Africa and his mother forgot to leave his key where he said it would be.

At Georgia Tech we got a very small server space which was used in a few CS classes, it would have been large enough to run hubot. Your university might have a similar setup.

If you've got a static files you could still serve it using Heroku free. Put cloudflare or another CDN in front of it and boom you've got 24 hours of fast responses. Though if it's a static site why do you need Heroku? Kind of like using a cannon to kill a mosquito.

New Heroku Pricing 11 years ago

The first step was removing varnish

This was removed for architectural reasons. It brought a lot of downtime to the platform and didn't help speed all that much. It's better and faster to cache this type of content with a CDN anyway.

If you want to get into:

  * Discussing technical issues with developers.
  * Learning about new technologies.
Check out http://www.codetriage.com. It will also help give you the tools you'll need to:
  * Explain new technology to customers.
  * Simplify technical issues for customers.

Pick a few open source tools that your company uses like http://www.codetriage.com/rails/rails and start reading issues. Eventually you'll learn enough to start helping out.

I bought this game a few years ago. I learned quite a few things about vim. Unfortunately the puzzles in the later levels aren't very clear. I spent hours trying to figure out what on earth they wanted me to do. To me this was counter productive and got less and less fun. I ended up leaving the last few levels unfinished as I felt I got very minimal returns for time invested.

If you're interested in picking up vim, I love it for the first 5 or so levels. After that, not so much. (I currently still use sublime text but do occasionally use vim on servers)

Austin is amazing! If you're in town stop by an Austin on Rails, great community. There's always friendly folks, companies hiring, and ususally some food.

I think there are a few issues. The first is funding. If you are an engineering manager and trying to ship product, you don't want a 25% faster Ruby VM in 10 years, you want someone to benchmark and optimize existing code in existing products 25% today. I agree that it's a good pay off, cost versus benefit...however the benefit is spread around hundreds and thousands of companies. Google gains marketshare for chrome when V8 gets faster, there's very few companies that get significant advantage explicitly from making Ruby faster for everyone. If you're paying to make Ruby faster, it's also getting faster for your competitors too (if they use Ruby).

There are a bunch of core contributors that are very active. Several of their companies allow them to spend time contributing (most of them are from Japan). Another example is Aaron, he is on Ruby core and his company allows him some time to contribute to open source, so this is sponsorship in a way. I while a company stands to benefit from sponsoring a full time developer, they benefit just as much if someone else sponsors a full time developer. Right now there's not enough companies with either a business incentive, altruism, or interest. Perhaps there are companies out there interested in sponsoring full time devs who just don't know how (and to your point cannot find them) but I think that would be the minority case.

We sponsor Matz, Nobu, and Koichi. Of them Koichi works pretty much full time on speed. He implemented partial generational GC in Ruby 2.1 and incremental GC in 2.2. He also implemented faster keyword arguments https://bugs.ruby-lang.org/issues/10440.

There's actually been a bunch of great performance increases in the past few years in addition to the GC. Optimized method cache invalidation by the late James Golick, frozen string pool for hash keys by tmm1, using vfork instead of fork, etc, copy on write GC. There's also new features, like Ruby's ability to GC symbols that let developers use symbols in more places and spend less time converting back and forth between strings.

So yes, money is a factor. Having employees work on it full time helps. Despite only having 3 full time employees Ruby has made some pretty impressive improvements recently.

This is really really cool. Thanks for posting. I'm especially interested in memory usage right now. I wrote a way to generate load and measure memory impact on your app: https://github.com/schneems/derailed_benchmarks

Comparing the two, it looks like this app only uses heap size for memory and doesn't take into account memory that isn't being used but hasn't been freed. https://www.dropbox.com/s/679sd83gqlkligk/Screenshot%202015-...

From `ps` i'm seeing about ~ 200mb RSS usage where rbkit is only showing ~50mb. I wonder what we could do to increase the accuracy here without having to shell out (very slow).

Also as a note if the gem install fails, on a mac I needed to run this:

    $ brew install msgpack
    $ brew install zeromq
Then it worked fine. Thanks again! Projects like this get me really excited for a future with faster applications and better informed developers.

I've learned to do this, but for me it wasn't obvious at first. At the end of the day if it's my PR, it's my job to provide the supporting evidence as to why that code should be merged and not some other code. If you respond "no you benchmark it" and they never respond, the ticket can sometimes get stuck in limbo.

This happens so frequently to me. I'll Spend an hour running benchmarks and then post a PR. Someone chimes in and says "method x is faster" with nothing to back it up. I have to benchmark again, and then my way is still faster.

can't Heroku/GitHub/37 signals or someone sponsor a week of dev time to automate a system

Heroku does sponsor Matz, Nobu, and Koichi. We hired them and give them a full time salary to work on Ruby.We don't tell them what to work on (i.e. we don't dictate what features or projects get shipped in what versions) it's more like corporate sponsorship. GitHub hired tmm1. Beyond that CRuby has a host of other non-sponsored contributors that contribute code and doc patches as well as set up tooling. You don't need corporate sponsorship for that, you need a passion and some time. You, in fact, could be the very person that sets up this automated system.

I find the attitude of '<company x> should sponsor <thing y>' a bit misguided. While I agree that companies who profit from a OSS should give forwards to OSS, individual contributors are ultimately they only way progress gets made. If you or someone reading this works for a company making money off of Ruby do that thing! Tell your boss you can't deploy on friday afternoon and have to fix a bug in the Ruby codebase. Ship the project on company time, and then boom...your company just sponsored that thing. It's like magic!

Anywhoo, yes finding out what things are blocking the MRI team from adopting a tool/technology and working around them can be extremely valuable. For example Matz has agreed to move the codebase development to github. This would make contributions a bit easier, however there is a blocker. There are a ton of SVN bots and tooling written around the current workflow. Right now the Ruby core team wants to spend time focusing on pumping out C code to fix bugs, improve performance, and progress the language forwards. They don't want to go off into the woods on tooling. These types of projects could be hugely impactful to the team and don't require C knowledge so most Ruby devs may be able to help. I encourage you and others to reach out to the core team to ask them what tooling projects they need and how we can help them. Understanding why someone isn't using a tool/technology is a really good start.

tldr; Good idea, let's talk to the core team about ways to help with tooling.

Ruby 2.1.5 Released 12 years ago

2.0.0 means no memory errors

2.1 has a totally new GC setup (generational GC). It is guaranteed to use more memory (expect 3~5%), but be MUCH MUCH MUCH faster. Typically you can trade off running one less Puma worker to decrease memory usage, and the increase in speed may make up for it. Alternatively you can bump up to 2x dynos, and cut the number of dynos you have in half. If you haven't tried 2.1.3 or above do yourself a favor and revisit. The 2.1.3 patch means memory grows much more slowly.

Also if you didn't see my post on memory profiling: http://www.schneems.com/2014/11/07/i-ram-what-i-ram.html (It got over 40 upvotes on reddit, and like 3 on HN, again...HN what is going on? I've had much less substantial posts make it front page). That post shows you how to remove ~30% starting RAM (by bumping mail version to 2.6.3+) and shows you can profile and remove libraries that are using a ton of memory in your app.

You can also use something like puma_worker_killer (or unicorn_worker_killer) to tame your memory growth if it only balloons over 512mb after a while, but ultimately those two solutions are bandaids.

Ruby 2.0 isn't going to be supported forever and the generational GC is definitely here to stay. Maybe try the 2.2.0 preview (it's on Heroku), it has 3 generations (total) which means much less retained memory and you still get speed performance improvements.

I see most of the bigger production apps using 2.1+ fwiw.

Ruby 2.1.5 Released 12 years ago

If you're still running a Ruby 2.1.0, 2.1.1, or 2.1.2 (i.e. less than 2.1.3) you're missing out on some killer memory optimizations. I saw huge savings in my production app when i bumped from 2.1.2 to 2.1.3.

Also 2.1.5 is available on Heroku: https://devcenter.heroku.com/changelog

Think HN is loosing its touch. It's been on Reddit for over 24 hours: http://www.reddit.com/r/ruby/comments/2m6p6n/ruby_215_is_rel.... Anywhoo, glad for the publicity, please upgrade your Rubies.

I dislike the proprietary nature of Heroku platform

Most everything we do, we open source. For example I help maintain the Heroku Ruby Buildpack, and it is fully open source. This is the technology that allows us to detect and install dependencies and precompile your assets. We open source a ton of other things such as our CLI, our logging aggregation infrastructure, our continuous Postgres backup service (backs up postgres to S3). We also invest heavily in Open source, we support PostgreSQL development, and we hired the 3 most prominent Ruby Core developers so they can work on the language full time.

Under the hood, we use LXC a standard linux container. We generally don't talk about this level much because it's orthogonal to the running of your application. Most people don't know what system libraries come pre-installed on our servers, or the versions of the operating systems, or what virtualization/containment systems we use. They know their app runs, and stays running.

I'm happy you're comparing and contrasting. I think docker is a great technology like LXC and it's very useful. I wanted to chime in on the "proprietary" comment because if you feel our service ever "locks you in" in some way, it's likely a bug that needs to be fixed. I consider my work 100% open source time.

I would say depending on what you're looking at they're complimentary. NR can give you some really in-depth timing information such as how much time is spent in the DB versus garbage collection on a certain request. The metrics here are very high level, response time, memory usage, CPU load, etc. New Relic doesn't have the ability to be as precise with its system resource measurements inside of the Dyno so they're both very useful.

Fix the versioning 12 years ago

1) Semver is hard. I've got publicly used libraries with millions of downloads and sometimes people use it in ways I don't expect and this results in breaking backwards compatibility. Most large projects aren't, or can't be true semver. Take for example Rails. This is a great discussion over an area over why breaking semver may be better than keeping it in one case: https://github.com/rails/rails/issues/16497

2) Breaking semver sucks, and I go hulk rage mad when my shit breaks.

Generally if it's a large enough security incident and the fix is in gem land, we send out emails. At the end of the day, you're still responsible for security of your own code (and libraries), but we try to help.

From firsthand experience I would have thought their designer hiring strategy would have read something like this:

Buy a company of 30, fire most of them and use the severance pay as leverage to force the designers into staying with Facebook. Wait a year or so until all those designers you strong armed into joining your company quit, then repeat.

We've had huge buy-in from Ruby as it's pretty difficult to set up and deploy a Ruby stack. Where as with PHP it's basically supported out of the box by pretty much everyone.

It's hard to sell Heroku which is why we have the free trial dynos. I find most people either love us see us as a huge value add, or don't understand why anyone would use us.

We're not simply a small abstraction layer, instead we support deployment at the application layer. Which again basically sounds like meaningless fluff unless you've already bought into the product. If I had to sum it up, i would say we're a company that provides a collection of small, sharp, extremely well integrated tools that seek to make your production development experience more pleasant.

For Ruby you get the app setup which is huge, but then you also get process monitoring out of the box so if your app goes down due to hardware failure we get paged at 3am instead of you. It's extremely easy to do things like have multiple staging servers, and I really like our CLI.

Instead of trying Heroku, I would recommend starting off trying out Heroku Postgres and using it through whatever server you deploy to right now. Get the credentials from the CLI.

On a production DB you get forks, and followers. We also have point in time recovery which is pretty fancy. Hacker destroy your DB? Forgot to schedule a backup? Rollback to 1 second before the deletions, and done. If you like Heroku Postgres, then I would recommend trying the dynos, and if not: agree to disagree.

For PHP the sell is management, stability, and consistency. If you're on a shared host your performance is very inconsistent as well as your uptime. If you're on a VPS you have pretty decent performance but now you have to manage everything manually, and even then there's that uptime thing we're not 5 nine's but hopefully above most: https://status.heroku.com/uptime.

I honestly don't think we're right for every application, every developer, and every scenario. When we are though I hope to provide the best experience possible.

No worries. I start a lot of writing with a "hook" first sentence and try to end with a memorable sentence. I then diagram the work into bullet points and try to make sure all key takeaways are obvious, note I didn't actually work on the _final_ draft of this doc. I was pleasantly surprised to find they left my first sentence in there.

Some hooks are better than others. I'm an engineer before a writer. It sucks that we have to market tech things, but the sad truth is most announcements that have no "hype" get no hype. I do a bunch of open source work, and some projects I promote and they do well, others I don't promote and they don't do as well.

Generally at Heroku I try to only put one or two flashy sentences in. The rest should be direct substance delivery, though that's not always the case. Thanks for taking the time to click the link, read, and comment!