AI, Ashby Engineering, and the future

https://www.ashbyhq.com/blog/engineering/ai-ashby-engineering-and-the-future

by fredley • 2 months ago

62 55 2 months ago

As someone who has used Ashby pretty regularly over the last several years, I suspect I’m not alone in wishing they’d let the AI make the technical and product decisions.

It’s a product that feels like it’s designed and built by the C or D team. The slowness and bloat of Rippling with wild, fever dream interaction models and workflows that frustrate and confound.

(And enjoy fucking emailing them if you need anything at all: almost literally no self service paths exist in the app for account upgrades, adding on SAML etc. And I do mean that you have to email them—last I checked it’s a mailto: link, not even a form.)

abhikp4• 2 months ago

Ashby Co-founder & VPE here, happy to chat if you have time to share feedback, abhik AT ashbyhq ᐧ com.

We had to build a lot of product in a short period of time to be a viable option to replace incumbent ATSes who had fought off many startups (and it couldn't be buggy or unreliable). The startups that focused on simplicity and usability just never made it. So, we focused on building something flexible and customizable first, with usability being something we wanted to make excellent second. You probably felt that over the past few years, and I'm sorry you did. We've been working hard to improve it, especially in the past year (e.g., you can self-serve account downgrades and some product upgrades now!).

Still more work to do, but thanks for putting up with some of our deficiencies :)

cursuve• 2 months ago

Now, this is the right way to respond to negative feedback! Kudos!

nxrabl• 2 months ago

We have a blip in March / April every year; these cyclical patterns aren’t relevant to explain here.

The 'blip' post-AI is up ~30% for the months in question. There simply isn't enough data here to prove or disprove the thesis that "customer issues remain broadly stable" - you could equally argue something more along the lines of "AI engineering does not increase issues under ideal conditions but amplifies issues under external pressure."

applfanboysbgon• 2 months ago

"Not relevant to explain this stat" in an article entirely about using those stats to justify their AI policy is brazen, I have to give credit.

One other point of note is that the chart only goes back to January 2025, and only dilineates "majority AI" from "not majority". If the usage rate in January 2025 was at 45%, "issue rate remained roughly stable (with a giant asterisk) when moving from 45% to 51%" is not exactly a compelling story. There's no narrative I trust less than one created by cherrypicking data to lend itself an aura of faux objectivity.

miltonlost2• 2 months ago

A “blip” that we have to take their word on, though the relative jump from the 1 prior year wasn’t as high as this one. Bad stats juking

colinhowe4• 2 months ago

Author here. This is valid feedback, thank you. This post was originally an internal post. Internally, I have credibility as "the guy that does his homework". That doesn't carry over to the broader internet!

I think you're right that the data presented doesn't prove or disprove the thesis that "customer issues remain broadly stable". I think it lends credibility to the idea that "AI didn't lead to an obvious and immediate slopfest of bugs".

As for the bumps, we've noticed them annually and we haven't yet found a solid reason why. There’s no correlation with customer growth, PRs shipped, or features shipped. It's likely a combination of various seasonal effects in our own hiring and customer onboarding. What isn't in this article: we've more than doubled our customer base in this time period, our customers are more mature and raise bugs more readily, we've got ~60% more engineers, and our product surface area has grown a lot too. I'm yet to land on a satisfying metric that normalizes bug rates in a changing environment.

agentultra• 2 months ago

I’d recently been applying for plenty of jobs that were hosted on Ashby. They had a tiny link to a form where you could opt out of having your resume processed by their AI system. Only it never worked. Submitting the form displayed a spinner and did nothing. Not sure if that was intentional.

I’m starting to come around to the belief that vibe coding isn’t engineering. Not saying Ashby is vibe coding. The post certainly says they’re super serious about making sure everyone understands every line of change. I wonder how they manage that in practice.

There are few empirical studies on the effects of informal code review, what most of us do on GitHub and Codeberg, on error rates. It’s not much. But a little side line mentioned that the effect we do have on error rates disappears the more code you read per hour.

So you used to have the 80/20 rule. 20% of your dev team is doing 80% of the work. Now we have a tool that enables the 80% of the team that manages to squeak by without doing a whole lot to… write a whole ton of code that the 20% now have to review… only if they read too much it’s not going to do a whole lot.

So I dunno. Seems like that combined with the come wave of price hikes we might see orgs talking about rolling back their AI usage later this year.

square_usual2• 2 months ago

I don't understand why people look at bugs and go "oh it must be AI". Did humans never write bugs? I've seen plenty of platforms where bugs like that were the norm, bad software engineering has always existed.

SpicyLemonZest• 2 months ago

Before it reached universal usage, I repeatedly saw teams I work with adopt AI heavily and then immediately start shipping more and worse bugs. It's probably not eating up all of their productivity gains but it's eating a lot. Just today I saw someone merge a PR whose description was clear that it would cause an outage if deployed, and I really don't see any explanation other than the humans who were theoretically in the loop not reading it.

saadn92• 2 months ago

Bugs did exist, but bugs that come from AI generated code can be easily avoided if a good enough engineer designed it. Also, the rate at which AI generates code means that there's A LOT more bugs now than there used to be.

cute_boi• 2 months ago

These job portals are getting out of hand. These days, forget AI scans, recruiters are asking me for SSN and all sorts of personal information. I have been refusing to provide it so far. But many people do provide it, which puts people like me at a disadvantage.

colinhowe4• 2 months ago

Author here! That opt out issue is definitely not intentional, I just gave it a quick try and it worked for me in a few different browsers. Feel free to ping me on colin AT ashbyhq.com with any additional details and I'll see if we can get a reproduction.

We only hire engineering managers that are deeply technical. This gives us more ability to work closely with engineers and make sure they're using new tools in the ways we want. It's not perfect. Sometimes, it goes wrong. But, that's why we continuously add tests and other tools to help us spot mistakes.

darkwater2• 2 months ago

Can anyone tell me why Ashby is so en vogue in SV and also the rest of the world? My company recently switched to it and as an EM I feel... disappointed? What's so special about it to have yet another hiring system with VC money?

dominotw• 2 months ago

where did it come out of all of sudden. they must have a killer sales dept

Ben-G• 2 months ago

Benji Co-Founder & CEO here. Curious which areas of the product you've touched on that you're disappointed with? We have ample things we want to improve - but the reason folks are switching to us is that we're well ahead of competition on most dimensions. I also was an EM on the prior generation of products which let us to build Ashby. But it's a surprisingly broad product and so takes a long time to perfect.

darkwater2• 2 months ago

First of all, this is my opinion as a hiring manager and "user" of the system. I'm not in HR so maybe there is some secret sauce there that appeals them - I still haven't discussed with them their rationale.

To answer your question: I mainly used Lever before and it was more or less OK, just like Ashby is OK. So, I'm not saying it's a bad product, it's probably a good product. I just don't see it "so much better" to justify a switch from Lever from my point of view. Both are "good enough" from an hiring manager perspective (or at least, my perspective).

eudamoniac2• 2 months ago

I last applied for jobs in 2022 and never saw it. This year it's the majority of the jobs using it. I don't know where they came from but they seem to be eating everyone's lunch.

Terretta3• 2 months ago

Because Greenhouse

oytis• 2 months ago

Just by the headline: are they doing layoffs?

UPD: surprisingly not

abhikp4• 2 months ago

We're doubling the Engineering team :).

conartist6• 2 months ago

It just sounds to me like there's probably a 20/80 split between your "new rockstars" who are using AI to make huge messes that others will have to clean up and your "malicious compliance" types who basically still actually do the work but run up the company credit card to put a patina of AI on their normal work so that they won't get fired.

But, like, this is a model you can embrace without AI. Have some people make messes, have others clean them up.

miltonlost2• 2 months ago

Our thesis is that the cost of producing code is heading towards zero. AI isn’t coming for our jobs, it’s coming for the mechanical parts of them: syntax, glue code, and the tip-taps of keystrokes. The parts that are less interesting, less challenging

The cost is heading to 0???? Do they not pay for Claude, Anthropic, OpenAI? Have they not seen the sudden price increases? Hosted locally LLMs still have electrical and server costs.

4ffee• 2 months ago

When people write stupid stuff like that, can you trust anything they say?

I believe not. It shows they dont truly understand the nuance of what they are talking about.

eudamoniac2• 2 months ago

Ah yes, local CPU cycles, famously never getting cheaper.

brettgriffin4• 2 months ago

Our thesis is that the cost of producing code is heading towards zero

This (correct) thesis should illicit an interesting question about the future of SaaS markets: What happens to the SaaS markets when the cost of code approaches zero?

Coincidentally, the company that authored this post is a perfect case study.

Few people truly grasp how hard bringing a software product to market is. The feature density required to even begin selling your product often requires more human capital than early stage investors are willing to underwrite. Some founders have the background to command those terms, but most have to wedge into a market with some very small insight.

I was at a dinner with the CEO of Greenhouse in 2021 and vividly recall him explaining just how deep the feature set of a new ATS needs to be to even enter the consideration set of a buyer.

One serendipitous way companies could enter these markets is by being founded in one market, and pivoting into another. Ashby didn't start as an ATS. They were originally targeting a mid-market ERP (a hot thesis at the time). Ashby is a prime example of a company that likely could not have entered their current market because of the sheer engineering resources required to break through (and kudos to them for doing it!)

But as the cost of code approaches zero, the deterrents of entering the market also drop to zero. The next Ashby could just pursue ATS from day one. In a sense, this is the entire story of humanity and especially software (consider the cost of building an ATS in 2012 when Lever was founded vs the cost Ashby faced in 2019).

But the slope of this change is unlike anything before it.

ATS, like many other markets, has a handful of big players and a long tail of smaller offerings. No matter how many long tail providers there are, the economics of entering a market suppressed that number compared to what can and will exist tomorrow. The cost of building an ATS a decade ago was literally orders of magnitude more than it is today.

In 2016 I heard a A16Z partner describe a future where every software market would have offerings in virtually every little niche one could imagine. Years before the LLM renaissance, this sounded insane. How could the market afford to build and sustain each offering?

I don't think companies are going to start building their ATS in house, but I do believe that cost of producing code is approaching to zero, and that means hundreds or even thousands of offerings will exist in every shape, way and form we can imagine.

jpalawaga• 2 months ago

I suspect the ceo of greenhouse wasn't saying 'and there will never be another new ats again.' that's a ridiculous thing to say, as evidenced by the fact that greenhouse and lever both cropped up.

the reality actually is, you DO need depth in order to close lucrative enterprise contracts.

but startups will use anything. you use early money/traction to fund the deeper features. that's saas/startup 101.

anyhow, saas just means the second "s" matters more. maybe software gets cheaper (that's actually an unproven hypothesis), but service encompasses many dimensions. the most lucrative contracts won't be eaten by fly-by-the-night operations almost by definition. any startup that knows the words 'vendor risk' will tell you.

brettgriffin4• 2 months ago

He didn't say there wouldn't be competitors. He was actually acknowledging the competitors exist, but he was expressing how none of them are in the consideration set of Greenhouse's buyers because they lacked an enormous number of features that Greenhouse had. My point is, at least behind closed doors, he and most CEOs would admit that their features are an eroding defense as the cost of code approaches zero.

abhikp4• 2 months ago

Co-Founder of Ashby here, just want to correct the record, that we were not targeting a mid-market ERP. We really wanted to build better hiring software. While our first customer was on our analytics-only product, it was built on a platform intended to be an ATS. We did build some abstractions/building blocks that are not hiring-software-specific (e.g., a workflow engine) because we recognized they allowed us to build rich features faster.

brettgriffin4• 2 months ago

Apologies. When I met you in circa spring 2019 you described Ashby to me as mid-market ERP, which I had quite a few thoughts on given the amount of ERP work I had done. Sorry for droning on about ERPs to you, then.

fontain2• 2 months ago

“What happens to the SaaS markets when the cost of code approaches zero?” … “Few people truly grasp how hard bringing a software product to market is.”

Are these related? The difficulty of bringing a software product to market has never been that it takes time to produce code. We’ve had the concept of MVPs for decades and pre-AI companies have raised hundreds of millions off of no code prototypes.

YC has always preached that you launch. Launch today. Launch as soon as is humanly possible. And learn. And iterate. You get something into the market as soon as possible. Spending years building the perfect software has never been the strategy pursued by technology startups, and not because of the time it takes, but because you need users to know what to build.

I think we could even go as far as to counter your argument. We know that constraints are powerful. Constraints force us to focus. Imagine you decide to build an ATS. You spend a bunch of money on Claude Code to generate hundreds of features, every little idea you can think of, you include it. You launch with the most complete ATS on the market. Ashby can’t hold a candle to the depth of your software. Do you have a better chance of success than if you had launched with a much smaller surface area, experimented, found the right area of focus, and iterated according to user behavior?

I encounter a lot of vibe coded software products that people are launching and I am not precious about code, a good product is a good product whether it was hand crafted over a decade by a thousand well paid programmers or generated in an afternoon by Claude Code from a laypersons prompt. I can name one single vibe coded product I pay for that is genuinely good software. All that the AI age has done is show that most of us absolutely suck at building products, and that even with free code, what we produce is garbage.

“How could the market afford to build and sustain each offering?”

I think you’re conflating build and sustain here. Nothing fundamental about business has changed. Sustaining a business isn’t about being able to generate code. Even in a world where code is free, how could the market sustain 10,000 Ashbys? Where are the customers coming from?

As a concrete example: how many customers does Jira have despite Linear being better software in every single way? Would generating 1,000 more Linears change Jira’s market position? Or Microsoft Teams, Teams is awful, there are so many better options. Will generating 1,000 more better options change Microsoft Teams’s position?

brettgriffin4• 2 months ago

The P&L of every software company has an R&D expense (R) in a total cost (T). We can debate what R is, and R might be approaching 0, but it is not 0.

This does not mean that R is on the only cost (T-R grows over the life of the company). It does not mean that, even if R were 0, you could launch a product. But R is a real cost.

albingroen• 2 months ago

Yeah expect the experience of using Ashby has just been declining. So that’s that.

samstokes2• 2 months ago

Most of the comments so far are responding to the first few paragraphs of this article. On reading further, I thought this was actually an unusually balanced take on how to use LLMs in a software org.

I can't help but cringe at the "cost of code is now zero" meme that this article repeats because in my experience the biggest cost of code was always the activities around the code - planning, communicating, reviewing, validating. This author, despite repeating the meme, seems to agree. Their emphasis on writing PR descriptions and specs for humans rhymes with my experience and it sounds like a nicer way to work than chasing some dark factory fever dream.

I also thought the "Two Modes of Working" section was useful. People get wildly different results from coding agents depending on how they use them, but I've not seen a lot of actual guidance on when to do X vs Y.

Personally, I've been using what the author calls "sidekick mode" since last October - before the supposed "AI got good now" threshold - and agree it's a more useful default for an engineer than "delegate mode".

tayo423• 2 months ago

Writings code has been viewed as an expensive part of development for a while. That's why the coding interview script the industry has adopted requires you to to not write any code until you suffenciently planned the whole problem.

habinero3• 2 months ago

No? You do that to make sure everyone understands and agrees on what to build.

Writing code is the easy part. Figuring out what your new feature or product should do is the hard part.

itissid• 2 months ago

I've heard there is a way to know if your resume is getting blackholed by a AI resume screening process — sort of like a test to fix it up. Any one knows how that works?

0xbadcafebee• 2 months ago

This still centers humans around the incredibly flawed "software engineering" paradigm we've been slogging through for decades. The fact is that we're not engineers. We're craftspeople slapping together buildings out of mud and stone like it's the middle ages.

"We must think more" - unless your industry wakes the hell up and standardizes its work. In that case you don't have to think more, because you have reliable standards that remove the need to think. I hope I'm not dead before the industry finally grows up.

vcryan• 2 months ago

The irony is that you can also use AI coding to just build your own Ashby since building Ashby is so easy to do with AI (apparently).

weakfish• 2 months ago

Why read article with human brain if not written by human brain?

pullshark91• 2 months ago

This irritated me too. It's unclear how we are to know that author has put at least some thought into this article.

annoyingcyclist• 2 months ago

To me, if "you are responsible for what you write" is to mean anything in an org, it needs to come with teeth. If you are demonstrably _not_ responsible for what you write, that needs to be recognized and treated as underperformance. I've seen a couple orgs who had some flavor of "you are responsible for what you write" guidance, and they mostly fail to follow through on it. People who ignore that guidance and ship a lot of low quality stuff (code, design docs, PRs) survive perf cycles and tend to get shout outs from management for moving quickly. People who take it to heart will tend to look slower by comparison (their work may be better, but often not in ways that are legible or compelling to management), and may be less favorably viewed when it comes to raises, promos, and so on.

I have no reason to doubt the sincerity of the authors of that post (and enjoyed reading it), I just hope they have a good sense of how they're recognizing responsible and irresponsible use and taking that into account in a perf process.

colinhowe4• 2 months ago

Author here. 100% agree with your sentiment. When we do performance reviews of engineers we take the time to comb over at least some of the PRs the engineer has written. We flag any patterns around code quality (regardless of whether it came from AI or not). This is on top of any other ongoing feedback we're giving.

E.g. in a recent review I wrote I picked up on an engineer not splitting up React components aggressively enough and over-relying on deeply nested ternaries. In that same review I also commended the engineer for being particularly excellent at explaining complex bugs to our support people and called out one example of a nasty race condition.

Terretta3• 2 months ago

When we do performance reviews of engineers we take the time to comb over at least some of the PRs the engineer has written.

Wait, whoever is reviewing the engineer isn't already familiar through exposure to – if not participation in – flow of work?

AI, Ashby Engineering, and the future

Related Stories