It took 12 weeks to ship an MVP I thought would take 3

https://boxci.dev/blog/why-it-took-12-weeks-to-ship-an-mvp-I-thought-would-take-3

by davnicwil • 7 years ago

282 217 7 years ago

I can’t accurately estimate non-trivial software. I’ve never met anybody who can accurately estimate non-trivial software. I’ve never seen anybody claim to be able to accurately estimate non-trivial software. Yet for some reason people still insist that estimating non-trivial software is not only possible, but trivial.

drspacemonkey• 7 years ago

Yet for some reason people still insist that estimating non-trivial software is not only possible, but trivial.

It's always maddening to get into debates about this stuff with project managers who believe this nonsense. I know the Fibonacci point scale is supposed to address this by giving PMs something, but I've only ever seen it turn into time-estimates-by-proxy.

These days, I only ever give estimates in terms of time scale. A task will take "hours", "days", "weeks", or "months". As long as it's within my power, my team will only ever estimate on that scale.

wikiman• 7 years ago

This 100% this. I run my teams via "Kanban" (I don't know how kanban it actually is, but that's what I call it).

The only question I ask around prioritization is "Is it worth doing?" If the answer is yes, I ask the stakeholder if they care how long it takes. If they do, I ask for a range and give a best estimate if it's achievable. If the answer is no, I ask why they care how long it takes if it's the most important thing to do.

My engineers know that they're expected to do that one project to completion, and we loop in other stakeholders (marketing, sales, QA, etc) progressively as we approach completion. Admittedly I work at a company of 20, but it works remarkably well, despite the absence of a "schedule"

cocoa19• 7 years ago

Most maddening thing I've seen is PM translating agile points to man-hours because that's what upper management requests.

Same people: you must complete assignments in the estimated time, or else, be prepared to work overtime including weekends.

It drives me bonkers.

xky• 7 years ago

Similar but I’ll ask for a range which has a built in confidence level. A lot of developers are very reluctant to give estimates for various reasons but if the reason is that they don’t know where to start, I’ll do a thought experiment with them.

I ask first will the task take 20 years to finish and they usually laugh and say of course not. Then I ask what about 20 seconds and they laugh again. We keep going on both sides of the boundary until they stop laughing and it starts sounding more reasonable.

Then I have something to work with like 3 to 6 months which gives me more options. I can accept the risk or I can take other steps to reduce it or break it into more manageable bites. It’s not perfect but it’s worked pretty well.

danpalmer• 7 years ago

The trick is to break it down until it’s all trivial. That process takes time itself, but you can estimate that much more easily. I’ve recently found I end up giving an estimate of, say, 2 days to get a solid estimate, and then come away from that with 2-5 weeks of tasks that are no more than a day each. Estimates of a day are pretty accurate (for me).

Another approach I had some success with was estimating the 80% likelihood case, I.e. I’m 80% sure I can finish this in the time I’m estimating. Less accurate, but much more predictably on schedule, which is more useful for many stakeholders.

A last one is giving rough estimates and being clear about how rough they are and how much time it will take to get more confidence. I was handed a 50 page PDF of API docs for a service and asked what the estimate was for integration. After 15 minutes I said “1-4 weeks”, if you want more accuracy then I’ll need a few days. The answer was “no problem, we wouldn’t consider it unless it was < 1 week”.

These examples are all on the weeks to low number of months scale, but the same applies further up. Having a good understanding of the codebase and domain are very useful.

Estimating isn’t easy, but treat it like any skill, it can be improved. It requires some flexibility from product managers, but open communication goes a really long way and results in better decisions overall, less wasted effort.

henryfjordan• 7 years ago

You cannot, by definition, break down a non-trivial project into trivial parts without having at least 1 non-trivial part in there somewhere

salty_biscuits• 7 years ago

Think about how variance propagates through that chain of work though,especially for tasks that are contingent on other tasks...

bradknowles• 7 years ago

Breaking it down until it's all trivial is the most non-trivial part of the exercise.

If everyone could do that trivially, then there would never be any problems.

olooney• 7 years ago

This is better than not breaking it down, but it's still not very good. McConnell advocates this approach strongly in his book on software estimation[1], and I've seen it work to some extent. It works for reasonably repeatable projects that are very similar to earlier projects you have experience with. But when we get into seriously non-trivial projects that are more "R" than "D"... well, it just isn't enough.

The problem is that on a non-trivial project, these breakdowns are guesses which carry considerable uncertainty. And the less you know, the more the unknown-unknowns get you. If you write a detailed breakdown of a major piece of software before writing a line of code, at the end of the project you will look back and laugh and laugh at your own innocence.

As just one random top-of-mind example, consider Carmack writing Doom's rendering engine[2]. He tried several approaches that didn't work before striking the right balance with pre-computed BSPs. Some of the things he tried were fundamentally good ideas and would later be used for different games - but didn't work on the hardware of the times. How do you estimate something like that? "I'm going to spend a month on my first approach. That could be one month or two. If that doesn't work, (50% chance), I'll read research papers and textbooks (1 week - 3 months) until I find a promising technique. Then I try the most promising of these; since this is an approach I haven't even read about yet, it could be one week to implement, or six months. And there's some chance that will fail to and I'll have to do it again." The final estimate is anywhere from one month to a year. You just can't know.

[1] https://www.amazon.com/Software-Estimation-Demystifying-Deve...

[2] https://twobithistory.org/2019/11/06/doom-bsp.html

femto• 7 years ago

I'd extend this by aiming to have each trivial step be a useful result that can be immediately released to market. If you can't make every step useful, try to get as many as possible useful with a minimum number of steps between useful results. Put a hard estimate on the first few trivial steps, but recognise that it's largely guesswork beyond that. If the trivial product can be released and start earning money then that reduces the pressure for accurate longer term estimates, as the earnings can fund the uncertainty. Recognise that this is an unachievable ideal, but it is at least worth striving for since even an imperfect attempt will deliver some benefit.

icebraining2• 7 years ago

If you can break them down to such a lower granularity, doesn't that imply the software is trivial? To take an extreme example, how do you break down the software developed by Waymo?

darkerside3• 7 years ago

The biggest problem I've found with this approach is that it drives the prioritization process towards smaller and smaller pieces of work, because those can be accurately broken down and estimated. This means larger, but proportionally much more valuable, pieces of work do not get picked up.

bcrosby95• 7 years ago

Some things can't really be broken down into it's trivialities unless you already know the answer.

E.g. I worked on a warehouse app before. None of us ever had before.

If you have no experience with any of this, how do you break down picking: SKU design (ours weren't just random numbers), barcode printers, and barcode scanners to be used in your software project? That part alone took us about 7 weeks. Including a complete SKU redesign because our initial test hardware worked better than some of the stuff we got later.

Nevermind all the hardware integrations, which was also new to us: multiple printers (barcode label, shipping label, pick list, packing slip), barcode scanner, a scale, all seamlessly integrated into the warehouse application.

This also reminds me, one of the services we decided to use had a serious (for us) bug that we didn't find out about until late in the game. They didn't fix it for 3 months and it blocked us for a while. So yeah, you could be very well end up integrating more than once. You can't really break that down into trivial parts.

Kaotique2• 7 years ago

I've estimated dozens of projects over the years and in my experience the more you split it up the more the total estimate is going to be. You can even use this a tool to get more budget/time allocated to your project. Break it up as much as you can and no task is ever going to be smaller than 1 hours, but in reality you can do 10 of those tasks per day because if you bunch those 10 tasks together people will estimate 5 hours of work. But now you have suddenly got 10 hours to do the same work!

edoo• 7 years ago

I do pretty well at it. My method is to identify all the components of a system and individually assign how many days I think I could do it in if I was really motivated. Then multiply by about 2-3 depending on your experience implementing previously similar modules. Then multiply by 2. I usually end up with a conservative time estimate where some things take longer and some shorter but overall it generally works. On this project that approach would have been... i can do it 3 weeks, give myself 6, and double it = 12. People love it when you come in at or under estimated time and really don't like it when you are late. Give yourself a buffer.

twothamendment• 7 years ago

That is pretty close to how I do it and I'm not the only one who thinks I'm good at it. This only works on a stack and team that I'm familiar with. It works with projects that are days, weeks or a year long.

If you can't break it down you don't understand it and if you don't understand it you aren't ready to estimate it.

r41nbowdash• 7 years ago

i ran analysis on our codebase, most of the files had 40% rewrites, with some outliers being 90% rewritten. the file sizes followed linear distribution, usually from 100 to 2000 lines of code. when the 90% rewrite hit a larger files, it accounted for almost 50% of total effort put into development. what i got from this, for a single component i take the ideal estimate then multiple it x2, and do the same x2 project wide. there's additional friction when integrating outside components, but i'm rarely blamed for it, so i don't care that much to put it in estimates.

now the hard part is selling the idea to the management, like "here are our ideal estimates, lets multiply it by 4". when doing freelance work i usually bill for the ideal estimated time multiplied by 2, the rest being my risk, on the other hand any changes in the requirements, or miscommunication is a risk of the client.

davnicwil18• 7 years ago

OP here - it's just (somehow, repeatedly) surprising the magnitude to which you can be wrong, even when it's just you and there's no pressure from anyone else to get it done quickly and it's really just an honest guess.

I see this over and over again with people I know bootstrapping projects - the over optimism I think is just part of being a builder. Sometimes in startup circles there seems to be this meme of how you should 'throw together a prototype of the tech in a week' being the standard way to get going with an idea. In practice though, seems so impossible that you could get anything even useful shaped built in that amount of time. For me when I get down to work on something, even just plumbing together the vague structure of an app takes longer than a week full time.

iainmerrick• 7 years ago

Have you tried to track data for multiple projects? For the specific case you wrote up, is the lesson simply “multiply my estimate by 4 next time”?

a_wild_dandan• 7 years ago

I think this is due to people adhering to a bad implicit analogy: "Estimating software construction projects must be like estimating other types of construction projects, e.g. bridges or cars." The problem is that most software projects don't have a myriad near-identical existing solutions from which to source a project's duration. We aren't making a building with the exact same floor plan, using the exact same crew size with the exact same skills, in the exact same regions, using...You get the point. It's just a bad comparison.

Most software projects are more like R&D than construction. For the adventurous analogy, software is like hiking an unknown trail. You might generally know where you're going, but you can't foresee the river, detours, and feral hogs you'll have to battle to reach the end.

You simply can't estimate software the way you can more tangible products. You can do Scrum planning poker and pretend your're not playing Numberwang, if that makes you feel more in control.

wvenable2• 7 years ago

Construction projects have detailed blueprints used to construct the estimate. Developing the blueprints is much more like developing software and probably has similar issues with estimating the time it takes, effort, etc. But nobody ever compares that. There is no construction phase in software (except compiling); the entire project is the planning stage.

chrisweekly4• 7 years ago

I'm reminded of the book "The Hard Thing About Hard Things"... and even more, of this also-classic post: https://www.quora.com/Why-are-software-development-task-esti...

verinus• 7 years ago

I even think that the construction analogy does not hold merrit: I have never heard of a really big project that was a) on time b) on budget.

I think those projects might be best compared to the software projects we think of here...

pas• 7 years ago

Estimation needs a baseline. Predicting the time needed for a greenfield (from scratch) project without some experience working on that project with a fixed team ... will lead to misery.

seph-reed• 7 years ago

It's magic, but I'm actually pretty good at it. Pretty much every time I'll say the number, my boss will be unhappy and get me to say a lesser number, then it'll be what I predicted. I have a method, but I'm pretty sure it's just magic.

eyegor2• 7 years ago

I find I can "accurately" estimate it by estimating the time for each major feature, and then tripling that number. Typically I ship ahead of schedule, even with glitch ups. Of course if you're in uncharted territory you should spend more time making sure your funding is secured regardless of ship date. Nothing worse than having the rug pulled out beneath you one day.

drawkbox• 7 years ago

There is also too much given to the importance of hitting the date over hitting the product/market polish, value and fit needed for success. People forget a project was late if it succeeds, they never forget a project on time that failed because it was rushed, buggy and not polished production.

Discipline and time estimates are important, but not more important than a product setup for success. One that is easy to sell and market because it has value, is polished and a good experience.

Game development notoriously is late, and as long as the developers have time to finish it right people forget. Everyone knows when a game company and the business/marketing/financial side pushes out an incomplete or buggy product, it can permanently harm perception.

Valve Time is something that is really common game development where engineers/developers/designers/product people are still in charge and making good solid product. [1] Technology, innovation and especially game development needs time to make it fun, a good experience and a solid product.

Creativity can't always be rushed, there must be "open" and "closed" modes of development, not crunched in "closed" only modes. John Cleese has an excellent talk on managing creativity and I recommend all people I work with watch it, very valuable insight. [2]

[1] https://developer.valvesoftware.com/wiki/Valve_Time

[2] https://www.youtube.com/watch?v=Pb5oIIPO62g

jammygit• 7 years ago

How do you do estimates for contract jobs then? Clients are very uncomfortable when you say “between X and 3X weeks”

meh2frdf2• 7 years ago

Often it’s the assumptions that go into an estimate that are critical, and defining them along with the estimate. Often it’s problem and delays at the client side that delays things or makes them harder.

buzzkillington2• 7 years ago

I can. Do your estimation normally. If you've done it a dozen times multiply by sqrt(pi), if you've done it a couple of times multiply by pi, if you've never done it before multiply by pi^2.

I'm not off by more than 20%.

jacobolus2• 7 years ago

Your heuristic that someone who has done a dozen previous software estimates will not take more than 177% of their current estimate to finish a “non-trivial software” project seems wildly optimistic to me.

Consider that many software projects led by experienced software engineers go so far over time that they are never finished.

fastball• 7 years ago

How can you have an accurate estimate when you're multiplying it by 10? A small error in your original estimate gets multiplied by 10. If I'm off by 10 days in the original estimate, that means I'm off by 100 using your method.

Also, I think by definition, a non-trivial project obviously hasn't been done by you many times before. Doing something you've done a dozen times is trivial in this sense.

dboreham• 7 years ago

Deluded people are everywhere.

mathgladiator7• 7 years ago

I do it well, but only after I completely understand the problem. For instance, I wrote out a series of design documents, then gathered buy in for management to start hiring. I told them that this will take 2 years to complete, but we can start realizing value in the first six months. Every timeframe was hit without crunch, and everything landed roughly on schedule.

The problem is that most people I've met dont really understand what they are actually doing and/or have the leadership to grow people to achieve.

It is not trivial in the least and requires a great deal of lead time to even understand the problem(s). This field is at its infancy, and I dont yet know if my insights can be replicated yet.

I'm thinking of writing a book where I illustrate thought exercises and career challenges that help people grow from junior to senior principal, but it is proving difficult.

jdc• 7 years ago

How long does it take to understand the problem and write design documents?

generatorguy• 7 years ago

Isn’t The hard part understanding the problem and writing the design documents? That is the task which takes the unknown amount of time. Once the roadmap is in place and people are assigned to build the things that have been designed the problems that occur are much more concrete. Unless there is some oversight or omission in the design, change in the requirements. Etc.

How long was the discovery and design document phase on the 2 year project?

chrisweekly4• 7 years ago

A 2-year horizon -- even for massive, strategically sound projects -- is a non-starter, simply untenable in virtually every situation I've encountered in my 21-yr career in software dev and mgmt and consulting. Not saying you're wrong to think and advise in those terms, just that the opportunity to do so is exceedingly rare. Big public companies tend to be overly constrained by quarterly myopia, and startups usually don't have runway to attempt to plan that far out. Personally, I once endured a project being shelved after 15+ months of solid, productive effort. I've also been the catalyst, at least twice, for adoption of paradigm shifts that ultimately spanned ~2yr timeframes (one was performance as a first-class citizen, and another was incorporating RWD (responsive web design) into a large company's MO. These successful cross-cutting / interdepartmental changes took time, but I don't think either of them could have happened if they'd ever been pitched up front as multi-year projects. Maybe you've just been luckier finding far-sighted decision-makers.

(If I seem bitter, I'm not. Just sharing my lived experience.)

tomelders• 7 years ago

Developers are optimists.

mc3• 7 years ago

I feel that to get accurate estimates you need the same people using the same tech stacks producing roughly the same solutions week in and week out. I imagine something like making slightly interactive marketing sites using Ruby on Rails every time, Active record every time, etc. These estimates will be perfect.

On the other hand, for a complex SaaS back end and front end doing something unique, a complex system to which you are being asked to add something groundbreaking, where you need to pay off various tech debt as you go in order to get the job done, where you are negotiating the requirements as you go along, and finding dead ends and impossible things that you can't do that you thought were simple and vice versa - I think it is hard to estimate. I think in that case estimation is a waste of time and it should be thought of like a "value train". If there is a genuine deadline (rocket launch) then best to manage it well with appropriate large buffers and dependency management. I don't see this being done anywhere, it is usually the business trying to haggle down the engineers on their estimates (even though the engineers are paid a salary so ???).

markburns• 7 years ago

The issue with this is that if you’re doing cookie cutter websites. At some point it’s going to be more efficient to abstract out the shared functionality and build a framework of some sort. You’ll become more efficient but you’ll either lose predictability or you’ll just be padding estimates to whatever you were estimating before. Not a bad position to be in but you’ll always be somewhere between predictable and inefficient or efficient and unpredictable.

paggle2• 7 years ago

Many people have the ability to estimate non trivial software, given the number of software products which ship on a schedule and don’t slip. Things like features announced at big conferences. Granted, the companies that do this can add people to projects as necessary, so they can work with months rather than man-months.

paggle2• 7 years ago

The much-derided “execs” who demand time estimates don’t do so out of spite. They do so because their costs are measured in man-months, and a business needs to know how much something is going to cost to decide whether it’s worth building. Also, they may have partner companies who they need to coordinate timelines with.

I’d guess that freelance developers who are paid per project, and thus have to own their P&L, are very good at estimating.

jdlshore• 7 years ago

I don't know about freelancers, but typical fixed-bid RFP responses are lowballed in order to get the work, then they make their profit on the inevitable change requests. So accurate estimates aren't as important as you might think.

navinsylvester• 7 years ago

Its hard to estimate without breaking it down to smaller tasks and this process is laborious. This process assumes we are tackling stuff in which we already have expertise.

The rule of thumb i use - anything which takes more than 3 days needs to be broken down further into tasks. If any task takes more than 3 days during execution then it needs to be broken down to more tasks.

teddyuk• 7 years ago

I was taught it takes about a quarter of the time to estimate work so if you think it will take 1 day you should have spent 1/4 day estimating. If you think 4 weeks then 1 week estimating.

Whenever I’m asked to estimate I tell them that I’ll take a week to estimate a 4 week piece of work they almost always say don’t bother - then when the made up estimates are wrong we can discuss the fact that they didn’t bother estimating.

It is a funny thing but works every time (in corps).

username90• 7 years ago

It is possible that a project is non-trivial to you but trivial to someone else, then that someone else could properly estimate it. Also most software projects are trivial so I don't see why estimations would be impossible, it is not like your typical react app is going to push boundaries on research topics. They might push boundaries to mediocre developers though, in which case it is a non-trivial project for them.

I think the real reason estimations are hard is that basically every developer is working on projects too hard for them. Like, if you are so good at your current job that you can reliably churn out good solutions with ease then instead of just producing you would move on to do harder things for better pay.

Aeolun2• 7 years ago

Estimating non-trivial software is easy. Making those estimates any kind of accurate is the problem.

I was talking with my project manager today, and he was asking me if we could finish on time if we got twice as much time as I estimated. My reply was that we’ve tried that 4 times now, and we’ve never reached the point where we actually finish within the time estimated.

So increasing the time would just mean we fail later, not that we suddenly succeed. I’m firmly starting to believe that work expands to fill the time allocated to it.

thrownaway954• 7 years ago

so basically all the common reasons why every deadline is missed. nothing new here.

redroot• 7 years ago

Hofstadter's Law: It always takes longer than you expect, even when you take into account Hofstadter's Law.

18monthsin• 7 years ago

Im 18 months into a project i thought would take 6. Why? Co founder bailed and i took some shortcuts early on to try and secure some clients that fell through, those shortcuts led to a substantial rewrite.

mgbmtl• 7 years ago

A colleague from a previous job once told me: "when a developer gives you an estimate, always double and add one of that unit" (estimate = 2n+1). He told me this after a while, because he would often nag me to clarify if I meant 7h or 1 day.

i.e. if you estimate 7h, then calculate 2x7+1 = 15h.

If you estimate 1d, then calculate 2x1+1 = 3d = 21h.

It's a silly joke, but it also gives insight about how we perceive time and the elasticity of estimates.

And of course, instead of multiplying by 2, you multiply by a time factor that depends on the risk factors involved (which are also guesstimates, but for me 'x2' would mean a rather low risk factor).

rhizome• 7 years ago

That sounds like a relative of a rule of thumb I read once upon a time: projects slip by the units they're estimated in. N weeks slips by weeks, days and days, months, years, etc.

mtlynch3• 7 years ago

Interesting writeup! Thanks for sharing.

Seems easy right - it's a CLI - just document every option and you're done in half a day. That's what I thought too.
But then you realise that each individual option is its own thing that requires thought to explain, and crucially, interacts with all other options. It's not enough to explain it in isolation. You need to tie everything together. You need examples. You realise from doing all this that there's a simpler way to name or combine options, you go back and make that change and have to update all the examples, explanations, etc.

You question whether your MVP at that stage was viable, but when I read this, I wondered whether it was actually minimum. Is it possible that you added too many options and your MVP would have been easier to ship if it had been more limited in scope?

The product did not look like something anyone would want to pay for

One pitfall to look out for is that you probably can't accurately predict what customers pay for until you talk to a real customer. Maybe things like aesthetics that seem important wouldn't actually be important to a customer if your product solves their problem. A designer gave you feedback, and they no doubt have valid expertise, but they're also not your customer.

I had the experience of shipping too late last year:

https://mtlynch.io/shipping-too-late/

tl;dr of my takeaways:

* Even if you've been warned about startup pitfalls a million times, sometimes you just have to make the same mistake to learn it.

* The goal of the MVP should be to get something in the customer's hands ASAP so you can hear their feedback. If you focus on polish and design, you might be polishing something that nobody wants.

* Question whether your "must haves" are actually just ways of delaying scary conversations with customers.

davnicwil18• 7 years ago

Thanks! And thanks for the link, I'll give that a read.

On the 'get it in front of a customer' thing - I agree and struggled with this decision a bit after I'd made it as a result. Probably the best way to validate you're building the wrong thing, or it's not ready yet, is hearing 'no' and hopefully why.

Super interesting factor in this case though is that basically I am the customer - it's really a product I'm building because I want it to exist. I asked myself the question, would I honestly pay for this right now? The answer was no. So at least for customer number one, me, I did kind of have that answer. Maybe cheating a bit though :-)

gspetr• 7 years ago

Yep, the good old "build it and they'll come" trap for the naive engineer that I've stepped into myself. Thankfully, having read Rob Walling's book now I know that marketing comes before code: https://news.ycombinator.com/user?id=rwalling

It's a bit dated now, but most of the advice is still relevant to solo engineers looking to turn solo founders.

mtlynch3• 7 years ago

I like that book too! I wish I had read it before shipping that first MVP.

My notes: https://mtlynch.io/book-reports/start-small-stay-small/

notjustanymike• 7 years ago

I've gotten pretty close with two 4 months projects this year (2 weeks off in both cases). It took two weeks of planning, research, and understanding roadblocks to get to that point.

It's doable, really, and it feels an awful lot like waterfall when you're in the moment. The most important steps were:

1. Write it down 2. Don't keep it to yourself

Everyone else is smarter than me and provided immeasurably good feedback.

davnicwil18• 7 years ago

2 is really helpful, 100% agree.

I shared my deadlines and progress on this build via the blog and on twitter under @zero_startup and I would say doing that probably pushed me to ship the MVP at least 50% faster.

davnicwil18• 7 years ago

OP here - this is a sidebar but the HN hug was making the page load really slowly (~10s for me) even though the blog part of the site is cached. Anyway, because I'm using kubernetes all it took to fix it was bumping the nodes in my cluster, 4x the replicas, kubectl apply and it's snappy again! All done in about a minute. What an awesome tool kubernetes is.

gnrl• 7 years ago

Try to add autoscaling. You shouldn't be doing it manual :-)

davnicwil18• 7 years ago

Yeah, you're totally right and I actually will now! Infra wasn't ready at all for the HN hug. I think I'd had at most 20 simultaneous connections prior to this!

joeblau• 7 years ago

The first time my site ended up on HN, I set up Cloudflare because this community blew my site off the internet pretty quickly.

skrebbel• 7 years ago

Please take this comment in good faith because I'm genuinely curious: why do you need a kube managed cluster of N nodes (an now 4x N) to host a static blog? I wonder what works goes on under the hood that this needs such scale.

davnicwil18• 7 years ago

The blog is just attached to the product site https://boxci.dev, thats the reason for using kube. Should be separately hosted but this was just simpler.

The static blog pages though are all cached via nginx so I was surprised I needed to increase the nodes & relicas. Short answer is I don't know, and will need to investigate, but I suspect it's because the kubernetes node instances themselves are fairly low powered, there's a low cpu limit on nginx, and perhaps nginx is doing a lot of work serving the js bundle to so many simultaneous users, which really should be hosted on a CDN (probably also the blog).

oblio2• 7 years ago

The real short answer is: he doesn't, he could just use Github Pages or similar :-)

I'm guessing he likes to tinker.

m0zg• 7 years ago

That's why, when people ask me to estimate work, I tell them that if they need a hard deadline things will (counterintuitively) take much longer, and try to convince them that hard deadlines are bullshit. If you want a hard deadline, I have no choice but to allocate a huge amount of buffer time, at least twice the "aggressive" estimate so that if shit doesn't go as planned (which is what usually happens) I don't miss the deadline. And I have to plan for the worst possible kind of excrement hitting the fan, which is not realistic. And then, in spite of that, people are still under more stress, because procrastination being in human nature, hard shit gets done last, and there's more in it that can go wrong.

Consider instead shipping stuff incrementally, feature by feature, when features are done, without any hard deadlines. Both the speed of delivery and quality will be improved compared to hard deadline driven development.

Plans are worthless. Planning is indispensable.

mikepalmer• 7 years ago

"What's the idea for Box CI? A CI service that does everything for you, except for running the builds."

Marketing advice: define "CI" near the top of the first page. The word "Integration" is not on the page either. I am familiar with Continuous Integration but not the abbreviation CI.

Adding to the confusion, the font on (most of) the home page is sans serif, so CI cee-eye looks like Cl cee-ell.

Moreover, you mention CLI on the page... though I did know what that is!

Good luck with it.

mynameisvlad• 7 years ago

CI is a pretty widely understood industry abbreviation at this point.

Adding to the confusion, the font on (most of) the home page is sans serif

I mean, except the logo at the top of every page.

davnicwil18• 7 years ago

Thanks for this. Yeah you're right - it's that thing of when you are so focused on something for ages, lots of things about it seem obvious to you that aren't necessarily at all to other people coming in cold. This is useful feedback.

eyegor2• 7 years ago

Sometimes it takes a conversation with someone else unfamiliar with your work to really question your fundamental principles

This is one of my favorite tools for development. I always sketch out an idea (no coding) and then try to describe it to someone with zero CS background before diving into an mvp. The times when I didn't follow this process - well - just as op described while trying to document the cli, I basically wrote the damn thing 2-3x over. So instead of a couple hours of chatting with other people, I had to spend days/weeks running in circles around my own work. Please, if you have juniors or if you are a junior, try to drive this lesson in.

davnicwil18• 7 years ago

Whilst working on Box CI, I've had a few conversations that have totally change my trajectory and way of thinking about the product. It's really the best thing you can do to advance things. Sitting writing code is only ever linear - you make progress one hour at a time, very slowly. A 15 minute conversation however, especially with someone you haven't talked to about it yet, can save you days of unnecessary work on something useless or, better yet, get you to an idea that might have taken weeks to come to on your own.

proverbialbunny2• 7 years ago

Fun fact, what you're describing it's called dechunking. https://en.wikipedia.org/wiki/Chunking_(psychology) And ofc, rubber duck debugging too. ^_^

m12k• 7 years ago

I think most programmers tend to do their estimates in this wonderful parallel dimension where the code base they will be working on is pristine, with well-designed, documented and thoroughly tested code that they get to work on without interruptions or meetings of any kind.

davnicwil18• 7 years ago

The thing is in this case I'm building this thing from scratch on my own, so literally without interruptions or meetings of any kind, and all the code, architecture, devops, everything is exactly how I want it, and the estimate is still wildly wrong :-)

andrewstuart2• 7 years ago

Developer to self: "It'll take me about a week"

Developer to development lead: "It'll take about two weeks"

Development lead to project manager: "It'll take about 4 weeks"

Project manager to self: "It'll take double that plus 2 weeks"

Project manager to management: "It'll take 12 weeks"

Management to client: "It'll take 8 weeks"

Actual time taken: 16 weeks.

riantogo• 7 years ago

Client: Wow, this is the first project that took only 2x time. We usually anticipate 4x.

andrewstuart2• 7 years ago

> We usually anticipate 4x.

All software estimation boils down to the long established scientific methodology:

(2 X what the last person said) optionally plus 2 weeks

proverbialbunny2• 7 years ago

Isn't it common for sales people to intentionally give a longer time frame than management specifies? I believe they try to buffer 20%+ rounding, unless there is an exact public release date. Though, I'm not in sales so I could be completely wrong.

In the consulting world there is the rule of three, where you multiply everything by three, not just how long it will take.

TeMPOraL7• 7 years ago

IME, sales people usually change the estimates to much lower than those reported to them, leading to no end of frustration and overtime on the engineering side.

msteffen• 7 years ago

I once had a manager who insisted on time estimates, but used them in a way that I actually found valuable. You'd write out all the tasks involved in a project add them all up, and then say "this will take T"

Then he'd say, "okay, what would it take to do what you just said in T/2?" Then you'd cut stuff until you got to T/2. The project then, very often, would take me the original T.

This seemed to work a lot better than just doubling the initial estimate to 2T.

techsin101• 7 years ago

In UI design you do this exercise where you put everything in 4 categories. Must have, should have, could have and won't have. Take your idea and jam it into these for while being 100% honest. It does the similar thing as you mentioned. But truly like the idea

mettamage• 7 years ago

MoSCoW, I learned this studying CS

hinkley• 7 years ago

A lot of devs have trouble decomposing work into small enough chunks. This guy had a way to force that, at least a little.

robert_tweed• 7 years ago

I'm stealing this with the caveat that it could put pressure on a developer to work out requirements, which isn't always what you want. If used correctly, to remove "gold plating" from an MVP, this seems like a great technique.

Another estimation trick that I've found to be very effective over the years is to start with the same sort of task breakdown and finger-in-the-air estimates. Next, start to make a mental list of all the things that could possibly go wrong with each of those tasks. Add those to the list. Most developers with some experience don't add these initially because they don't consider them to be "tasks", but can come up with a huge list surprisingly easily when prompted. Now factor in all the extra time for dealing with those unexpected-but-will-definitely-happen issues.

The original estimate is your best-case, since almost all developers tend to estimate optimistically. The new one is the worst-case.

A good rule of thumb is that the worst-case is 16x the best case, so if you got less than that, it's a good indication you didn't think of enough things that can go wrong. If it does follow the typical curve, 4x the best-case estimate will be a good expected-case (75% chance of success).

What this manager did right was prioritising the tasks, because estimates are just the starting point. The next thing is to know what absolutely must be done for MVP and what is nice-to-have.

If you have 3 weeks to launch an MVP then that MVP needs to be achievable in about 4 days. Otherwise you are setting unrealistic expectations. If you actually finish it in 4 days (unlikely) then well done. You can start adding polish in order of priority, starting with a load test in production to find out as many "unknown unknowns" as possible.

afarrell• 7 years ago

If you look at that conversation, you'll see that the value comes not from the number T but from the list of tasks which can then be cut.

celticninja• 7 years ago

Army place we point stories, Fibonacci series. Works to help identify a thing over an 8 usually needs to be broken down further. 5 and 8 are pairing stories,1 and 2 for when you have a half day available (depending on the seniority of the dev) or for a Friday afternoon if you finish another ticket.

transitivebs• 7 years ago

This is one of the core reasons why we've built https://saasify.sh, an extremely simple solution to launch FaaS-based businesses and start validating product / market fit in hours instead of months.

I agree with the author in terms of estimates and their assessment of the process, but the point I'd like to make is that the vast majority of those 12 weeks it took them to launch the MVP weren't spent on core, differentiating functionality, but rather on boilerplate that's common to every SaaS product.

Important but non-core features like billing, accounts, documentation, and a marketing site end up causing many projects like this to fail before they even get launched.

If you can launch MVPs like this significantly quicker albeit in a more constrained yet platform-agnostic way (e.g., using serverless functions as an abstraction), it becomes really powerful for makers like the author of this article to focus on core value and ship quickly instead of wasting time on all of this other stuff.

jrumbut• 7 years ago

I really wish the estimation process would get turned around. Let stakeholders decide when they need it, and engineers are responsible for delivering the best version they can on that date. As the project unfolds, stay in close communication and make timely decisions about what tradeoffs are acceptable.

My estimates get accurate once I know where we are on the spectrum from quick and dirty to pushing the boundaries of what's possible. There's a 3 month version and a 3 year version of a lot of ideas.

photonios• 7 years ago

You might find the way Basecamp works interesting: https://m.signalvnoise.com/how-we-structure-our-work-and-tea...

Each project is six weeks. You deliver the best possible version within those six weeks.

cpeterso• 7 years ago

That's the idea driving agile product development: defining and estimating big projects is hard, so let's do many small projects (sprints). If the stakeholders have a hard deadline, you can ship whatever you have then because your product is supposed to be in a shippable state, though possibly with reduced functionality, at the end of each sprint.

raveenb2• 7 years ago

My personal experience is that traditional software estimation techniques don’t work when done for a MVP situation. Yes it’s frustrating like hell to not know how long before your money, patience and interest runs out. In a non MVP situation some estimation models work, some 40% of time. But managers make it look like they work all the time. Cutting scope to hit deadline doesn’t count.

raveenb2• 7 years ago

Also the Viable part of MVP is subjective. As you build the MVP, your idea of what is viable changes. This needs to be accounted for. The minimum product has to be viable for a paying customer. If the MVP is not a paid product the you can get away with less polish, not so with paid products.

davnicwil18• 7 years ago

Exactly what happened with Box CI :-)

The viable I had in mind was 'it works' - later realised that actually viable was 'I think there is a chance I can ask people to pay for this' - turned out that second part around doubled the already overrun build time!

edoceo3• 7 years ago

Aty place we don't estimate time at all. It's nearly impossible. But value of the fix or feature is loads easier to estimate and measure. Bug $X is worth $10000, dedicated $5000 to fix it. Check progress. Dev says almost done? One more cycle. Still seems far? Punt.

I'm always confounded when managers want a time estimate but cannot provide a value estimate.

The goal is the same: fix important/urgent bugs, develop product and keep good close velocity.

MrStonedOne• 7 years ago

What is an MVP?

grzm• 7 years ago

Minimum viable product.

https://en.wikipedia.org/wiki/Minimum_viable_product

hitpointdrew• 7 years ago

This was my question. I know Hacker News caters to techies, but there are a LOT of acronyms out there. I see MVP and I think "Most Valuable Player", which clearly doesn't fit in this context.

mikekchar• 7 years ago

As my colleague says: The time where you know the least about a project is when you start. This is also the time when your estimates are likely to be the worst.

When can you say with fairly good certainty how much longer it's likely to take? In my experience it's about 1/3 the way through the project. I'll give you my reasoning behind this number.

There is a model of software defect discovery called Littlewood's model. It basically says that the rate of software defect discovery is a random variable. As you discover more defects in the code, the number of defects that are left to discover diminish. Assuming you put in a constant effort to discover defects, your rate of discovery decreases. You can estimate the number of defects left in the software, by looking at the decrease in discovery rates over time. There are lots of scholarly articles on Littlewood's model, so I won't go into more detail than that. There are newer models too, but I always found Littewood's "good enough" for my purposes.

It occurred to me that requirement discovery might follow a similar curve. As time goes on and you understand more and more about what you need, the number of things to discover decreases. If you work on your project in a constant manner, the rate at which you discover new requirements will decrease over time. I took data for a number of project and the discovery rate curves were very similar to defect discovery rate curves.

The key is to look at the rate of change of discovery. Once it is tailing off, you can estimate the speed of the drop and get a good idea of how much work you have left to do. Each curve is different depending on a number of factors, but by the time you are about 1/3 the way through the curve, you have enough information to estimate the parameters.

Note: Littlewood's model doesn't actually hit zero, but you can set a threshold that is "close enough to zero". Where you place it will change the 1/3 figure, but I hope what I'm saying is understandable.

kstenerud• 7 years ago

The thing about estimating is that you can't factor in the things you don't know:

- The tool you planned to use has a bug/defect that blocks you.

- The people who said they could give you some information you need, can't.

- You misunderstood or were misled as to the capabilities of a tool you need.

- The parts you need don't actually fit together as planned.

- The documentation you were relying on is wrong.

- A delivery you are relying on won't actually be ready on time.

- A process that's worked every time has an unhandled edge case that you're going to trigger in this project.

- A regression in a software package will block you because you can't use the previous version for other reasons.

The more moving parts there are, the more likely it is that one of the interface points between them will block or delay you in one of these ways, potentially for weeks, maybe even months.

So for any nontrivial project, you should expect a 2x-8x delivery inflation over your "safely padded" estimate. You have a 75% chance that you'll deliver within 2x of your padded estimate if you've made a good estimation, sliding to worse as the number of moving parts increases.

defined• 7 years ago

This is great and tallies with my experience.

phito• 7 years ago

You can add personal problems that can affect focus to the list

ibudiallo• 7 years ago

It's hard enough to estimate accurately when you are building your own project with no one breathing down your neck. But be a contractor, where your estimate becomes your salary, and you'll find that you tend to give longer estimates.

I once gave a 3 days estimate for a client project. 2 days to make sure I build it right and a third for testing. It turned into a 7 weeks project: https://idiallo.com/blog/18000-dollars-static-web-page

brigandish• 7 years ago

Being asked "how long will it take?" is actually my greatest fear in tech work. Give me long enough and I'm (fairly) confident I can overcome any other problem, but estimating time, even for seemingly trivial things…

<gulp>

I'll probably give everyone an upvote here as a) it's nice to see I'm not alone, and b) there's some really good tips.

Not sure why every tech interview introduces a whiteboard with an algorithm problem when all they have to do is ask me "how long will this simple command line client take to build" to really see the difference between the way I attack a difficult problem!

gaoshan• 7 years ago

I started multiplying estimates by 3 and found it to be, in general, much closer to reality once all was said and done.

jordiee• 7 years ago

Man your telling me. I planned on spending 4 months on https://appdoctor.io and ended up launching it after 1 year and 2 months.

davnicwil18• 7 years ago

Have heard so many stories like this. It's unbelievable how much longer these things can take compared to what you think at the idea stage. How's it going now?

qaq• 7 years ago

I think it depends I work on a large enterprise product we are generally within 20% of the estimate. If some prominent team members were not aggressive in planing that figure would be smaller. Average age on the team is over 40 avg experience around 20 years.

mathgladiator7• 7 years ago

Experience makes a huge difference in planning simply due to know where all bodies are buried and where dead bodies will emerge.

trpc• 7 years ago

nice way for marketing. You could have posted a SHOW HN a thousand times and nobody would care, but this is creative way for a succesful SHOW HN

pgt2• 7 years ago

How did you manage to constrain the time overrun to only 4x?

As a mercenary engineer, I've had to answer the estimation question many times. Whenever I've given estimates, it has only ever shot me in the foot. Spolsky might have figured it out, but I haven't. So I try to avoid giving estimates as far as possible and instead focus on demonstrating velocity of a working system they can choose to stop funding at any time.

Some clients, especially from the construction industry, expect me to give them GANTT-chart style estimates down to the hour. Here is how I convey the software estimation problem to them:

If I write the same code twice, I done fucked up. So any code I write, is new code. If I can reuse code, I will, but that's not the code you're asking about. In your industry, you have built the same house many times before with the same team, on the same foundation. But you are employing me to design, architect and construct a new house that has never been built before. If it existed already, you would just buy it off the shelf instead of employing me.

So think of me as more of an architect than a labourer, and together with you as the client, we are embarking on a design that you're not quite sure you want yet, that won't collapse on a potentially unstable foundation.

davnicwil18• 7 years ago

This is a good analogy, and when you're bootstrapping a product somehow it's extended further because in addition to being the architect you're also taking on the role of town planner, deciding where it gets built and thinking about if it should be built at all (do people need it).

pgt2• 7 years ago

Town planner or city planner is a great analogy! The role conveys the uncertainty of future growth and anticipation of problems much better than an architect, who typically only does a once-off design. Thanks for that :).

endymi0n• 7 years ago

Having done hundreds of small to mid sized projects I take comfort in the fact estimating is a very distinct task that even the best of developers learn last.

I've got three tricks in my pocket that at least help me with this:

The first is that I go by the formula mentioned in "The Mythical Man-Month" — which is that the effort in larger projects is distributed as follows:

1/3 planning 1/6 coding 1/4 component test and early system test 1/4 system test, all components in hand

Developers tend to mainly estimate their coding time, which leads exactly to the effects mentioned. It also explains all the annoying situations with all the holdups once the project is "almost done", this is simply a systems test and will add another 25%.

The other is that I simply add 5-10% on top for each person involved into the project (dev, stakeholder, anything) for communication overhead. It usually holds true.

The third trick is meticuluous focus on lean delivery, which involves scoping down, early testing on as much as possible, hidden and pre-deadlines and setting tractable sub-milestones.

Unfortunately, all of these techniques don't accelerate the project, they just help you be more realistic. If you find something that accelerates complex projects, let me know ;-)

ido2• 7 years ago

If you find something that accelerates complex projects, let me know

Developer experience. I've been working as a developer for a living for 18 years & I thought I was pretty hot-shit 10 years ago already, but I can probably do more today in 4 days than I used to in 4 weeks (or 4 months a few years before that).

tarsinge• 7 years ago

Really insightful, thank you.

Also regarding: > If you find something that accelerates complex projects, let me know

When possible cut scope and split the project in smaller projects with real deployments (much like you said with "The third trick is meticuluous focus on lean delivery, which involves scoping down, early testing on as much as possible, hidden and pre-deadlines and setting tractable sub-milestones.").

In nearly all my projects I now propose something like this to clients: "Ok we understand your needs and requirements. What do you think about having a V1 with only these core features deployed in a few weeks (which ideally already provide business value), and after that we iteratively improve it?". Works great for smoothly throwing away all the nice to have.

Kaotique2• 7 years ago

Don't forget the sanity check at the end. If other people also add a lot safety hours then you will end up with an inflated estimation that is hard to sell.

triggercut• 7 years ago

>If you find something that accelerates complex projects, let me know ;-)

I know this was partly joking but there are two real ways:

Knowing your true critical path and proactively re-plan and manage the risk of those activities that sit on it to clear the way for it to run smoothly... that's of course assuming you planned and estimated it correctly in the first place.

If you have a real schedule that is properly resource and risk loaded, run it through a monte carlo sensitivity analysis and see if you can re-plan those highly sensitive activities or de-risk them.

mattmanser2• 7 years ago

For me Joel was spot on in my experience, and I disagree, you are always writing the same code again and again. Just tweaking it a bit to fit the requirements.

An easy example is the thousands of JSON APIs you can integrate these days, where the code to do that is almost identical, but not quite. Generally the only complications are in what stupid authentication and pagination method this API came up with.

But it happens everywhere in coding. Every system might need user management, but slightttttly different each time. Or sending emails but Bob already uses Sendgrid, where traditionally you've used mailgun.

Also the software industry changes all the time. That email code you wrote ten years ago using SMTP won't be reusable. The invoicing system you built for one client using PHP and jQuery 1.7 in 2012? Not appropriate to sell to a client today.

Basically you should have written things at least very similar to what you're estimating on. Then you need to find your "multiplier". The figure you need to increase your gut estimate by to get the real estimate.

Plus, there's a lot of legal complication about reusing code with different clients, often you should be rewriting purely to be safe for them and you from a copyright perspective.

laumars• 7 years ago

In my experience working in a broad range of fields in IT; web development is quite unlike any other type of development when it comes to code reuse and framework churn.

LoSboccacc• 7 years ago

probably it's because developers tend to estimate the technical part first and foremost. then comes prouctization, then comes integration.

if the estimate is consistently between 3x and 6x, that's what's likely missing.

ebiester• 7 years ago

I'm a strong proponent of learning formal and informal estimation methods - I'd start with Steve McConnell's book "Software Estimation: Demystifying the Black Art." For estimates that I would put money on, I will estimate in 4 hour blocks and get features down to the most granular level. If I don't understand it at that level, I don't understand it well enough to estimate yet.

saalweachter• 7 years ago

One of my favorite shows is Grand Designs.

Building a house is probably the most understood human activities; as you stated, you can take a standard design on a normal building lot and throw up a house in a very predictable amount of time. If you've done it enough, you can write down a schedule of when every piece of material and every trade needs to be on site.

Grand Designs always involves some combination of unusual design, unusual sites, and unusual techniques, and frequently inexperienced builders. People set out to accomplish something grand, and usually succeed, but it takes much longer and more money than expected, and especially more than constructing a conventional house on a conventional lot.

Now, besides echoing your point that many novel software projects are much more akin to a Grand Designs house than a cookie cutter ranch in a Midwestern suburb, in Grand Designs one concept that is proved true time and again is that once you are out of the ground -- once you have your foundation in, your gas and water and sewer and electric hookups -- the hard part is over and the job will proceed much more smoothly. It's hard to estimate before you get out of the ground, because the ground is filled with unknowns and unforseen problems. You might still misjudge how long it will take you to finish, particularly the finicky details that always take forever, but this is purely a failure of estimation; an experienced builder could predict the time to spackle, trim, paint, install flooring and built-ins accurately, you just didn't realize it. But under the ground, there are all sorts of problems with rocks and water tables and pipes and whatnot you can't predict.

Do we, as software engineers, have a similar inflection point in our software development? Do we ever get "out of the ground", to the point that most of the unknowns are known and we have a predictable (if still great and difficult) amount of work remaining?

antoniuschan99• 7 years ago

This is a great analogy on why it's so difficult to estimate: https://www.quora.com/Engineering-Management/Why-are-softwar...

Estimating should be fun, plus there are various ways, such as estimating as a team (majority wins), t-shirt sizes, etc

jillesvangurp• 7 years ago

Viability is a KPI and not a set of requirements for some product.

You can't plan for a product to be viable. You first build a product and then you verify whether it is viable. At best you can formulate a hypothesis about what set of features you think is good enough in terms of KPIs and then plan to build only that. However, you have to factor in the likelihood that your hypothesis is wrong and also that the process of building something should result in refining that hypothesis over time. If that doesn't happen, you are not learning and you are probably not really building a viable thing.

The fastest way to get to viability is to take baby steps: short sprints/iterations, ship often, re-assess where you are every step of the way. Do the most valuable/risky/uncertain things as early as you can so you can adjust course if your assumptions about their value turn out wrong. Most startups get this wrong and fail for this reason because by the time they figure out they are on the wrong track they've already wasted most of their seed funding on building pointless things.

The lean movement tends to focus on the M part too much which has a built in risk for products to be unexciting and ultimately non viable. It's great if you are copying somebody else's business model or building some kind of market place. It's not so great if you are trying to do something new.

Lean has a tendency to postpone value creation until you've built a lot of low value commonalities like a login system, user management or crap like that that every startup seems to spend ages on without getting it really right. If you hear the words MVP and Android IOS and web in one sentence that translates as we're building a lot of common functionality three times and all our experiments have 3x the cost. The chances of that being utterly unremarkable and non viable are huge.

The minimal thing would be to postpone that stuff until you have something worth logging into and worth having multiple implementations of on multiple platforms. Building a good mobile experience is a huge investment. Don't even think about it until you have something viable.

You are not proving the viability of a login system with your MVP nor are you proving the viability of a slick IOS experience. Your thing is viable if despite obvious UX and feature issues you still get positive KPIs. Once you get there, you can justify the expense of making it better. So spend as little time as you can on stuff like that instead of making it the top priority in your first iterations.

As long as you are doing minimal things you are not actually creating a lot of value. You are actually postponing value creation and viability. It's the hardest things that create the most value and that are the hardest to plan and the easiest to postpone when planning. Therefore I believe, Scrum is the wrong process for building something that has a high risk/reward balance and the right process for building something that has low risk/rewards. All the value is in the stories that everybody struggles to estimate. Scrum results in most resources getting sucked up by the least valuable stuff.

themmes• 7 years ago

Thanks for sharing, I just quit my sideproject after a year of coding and find posts like this helpful to learn from that experience.

Then I stopped, took a step back and looked at the product from the perspective of someone who would want to pay for it.

Though I think this is dangerous advise. You do not know what this perspective is like (unless maybe when your product is a “scratching your own itch”-type product) and therefor making design changes makes no sense. You will never know if users maybe even preferred your first “design”, or that there is a different blocker like landing-page, installation-instructions or anything really preventing them from starting using it.

davnicwil18• 7 years ago

Thanks for saying that as it's basically the reason I wrote the post. Really good to share these lessons with fellow bootstrappers as so many of them are general and applicable between projects, even whole businesses.

Totally agree with what you're saying here and let's see, I might write another post in a few months essentially saying this was a mistake and a waste of time, but yeah if you're interested in seeing a comparison, check out the twitter account for the blog @zero_startup and you can see the old screenshots and the newer ones.

robsun• 7 years ago

In my career I have estimated a lot of project. I tried different techniques and I find https://en.wikipedia.org/wiki/Program_evaluation_and_review_... is the most accurate one.

If I was asked to give some hints, I would say:

- in optimistic and pessimistic cases don't fear to use extreme values

- don't estimate on your own, take reasonable amount of developers with variety of experiences

- don't give exact value to client, give probabilities ("it is very likely that we will be deliver this in between X and Y man-hours/days)

All estimates I made using PERT were quite accurate (<5% time difference between estimated and real execution)

mitchtbaum• 7 years ago

Yes. I was about to link this too.

From my experiences, I've found that 3x my initial "estimate / goal / ego trip timeline" gives the most realistic picture. Sometimes I can hit my best; most often things come up. It took a little while to accept, mostly because I wanted to do my best always and be able to promise and fulfill it. But Reality hit back, and I had to adjust.

It's hard to see the network effects of each micro-interaction in long range projecting, but once you really break down how long each thing could take, how that would affect each other related part, and who/what else is involved at each step/layer, it clears up the fog.

https://en.wikipedia.org/wiki/Fog_of_war#Military

rkangel• 7 years ago

I agreed with everything right up until the "extra time for design" section. I spent last year building a startup and have thought a lot about MVPs. Coming from a background of building big expensive systems, it took a lot to shake the mindset of "do it right from the ground up". My cofounder did a great job at pushing me towards the absolute minimal solution to learn what we wanted to learn - I don't need to spin up a backend and write CSS, I can use Wix to mock it up and see how that goes, THEN build it properly.

The idea is to learn as much as possible (read The Lean Startup if you haven't). If you build too much in one go, you're going to have tied yourself to one vision of how your users will use the product and be more closed to learning from them what they actually need. Note that you were persuaded by a designer to make the design nicer, not a user, potential user or even product manager.

What was the problem with launching, and then deciding that design was the next priority and working on it while your product was running?

[disclaimer: I definitely haven't got this figured out as we shutdown the company and I'm back building Big Expensive Systems]

davnicwil18• 7 years ago

What was the problem with launching, and then deciding that design was the next priority and working on it while your product was running?

Great question, and one I asked myself when I was about 2 weeks into the redesign :-)

On reflection, basically no, you're right, there was no reason I couldn't have shipped and then improved the design in the meantime. The thing about it not being good enough for people to pay for was just my opinion - not tested with real customers.

Cederfjard• 7 years ago

Is there any well-regarded book, study, methodology or even individual who claims that the development time of non-trivial software can be accurately estimated?

maschinenz• 7 years ago

I would consider myself a fairly experienced developer (~ 10 years), but yet I have had the same experience on countless occasions. People always think of you as "being the expert" knowing everything, so I try to explain my situation like this:

"Think of me more like being a journalist. You want a good story and have a rough idea, so I will find the right people and interview them (domain experts, business people), try to figure out how to distill their knowledge into something that is both accurate, yet not too detailed and most importantly has to fulfill the needs of my readers/users (easy to read, yet super insightful, with some pretty images etc.). Wheter the article is about nuclear physics or siamese koalas is secondary, the process is more or less the same, yet I am neither a koala expert nor a nuclear scientist. You are the expert."

I also try to explain why I am unsure about estimating even relatively small tasks like this:

"You know restaurant (or another famous location) Y, right? We both know how to walk, I mean we have had like 30 years of experience in doing that, correct? So how many minutes does it take you to walk from here to restaurant Y?". The more people in the group, the more interesting it might get. Estimates usually differ by a factor of 2-4. People usually cannot even correctly estimate a trivial thing like taking a walk.

davnicwil18• 7 years ago

The walking example is really good, I also regularly underestimate how long it will take me to arrive to meet a friend walking, by public transport, whatever.

The reason it's an underestimate even when I know the distances and usual times is because I think about the ideal set of conditions (not missing train connections, clear streets for fast walking, being able to find the place we're meeting instantly on arrival rather than looking around for the entrance for 5 mins) and go with that. I never account for the possibility of missing a train connection by 15 seconds and then having to delay the journey by 30 mins. Even though I know, of course, there is a non trivial chance of that happening.

_pmf_• 7 years ago

You're pretty fast.

amirathi• 7 years ago

It's astonishing how many developers resign to the idea that "estimating software projects is impossible". It's impossible when there are either,

- Technical uncertainties (e.g. self driving cars)

- Human uncertainties (multiple different teams building single large software system)

- Scope uncertainties (we don't know what we are actually building until we get into the weeds)

Outside of these we should be able to make reasonably accurate (+/- 30%) estimates. I wrote a blog [1] about it if someone is interested but here are the main takeaways,

- Don't just estimate writing code but also include time required for testing, documentation, communication, setting up infra/deployment

- If certain part is hazy ("is there a reliable python library for speech to text") then research it enough to know the path ahead

- Breakdown the system into smaller units until you feel confident to estimate each piece

[1] https://blog.amirathi.com/2018/02/05/science-of-software-est...

marcosdumay3• 7 years ago

Scope uncertainty is always larger than the certain part. It's often nearly all of the scope.

Technical uncertainty is also more common than not, unless you are using a dying platform, it will change between the estimation and implementation.

Human uncertainties are an avoidable one, if you work alone. If you are in a team, they are also certain.

Or, to put it shortly, yes it's perfectly possible if you are doing a university project alone in an old platform without an active community.

giancarlostoro• 7 years ago

Yep. Me and my boss estimated rewriting what looks like a really simple Silverlight application in HTML5 in about 3 months time. He's far better at estimating than I am, and a year later we finally did it. Problem was we chose to use .NET Core cause it was more modern, and who knows if Microsoft will dump .NET Framework at some point (and honestly after .NET Core 3.0 I could finally see this being possible). So we upgraded to .NET Core.

Estimating requires you already understand the very thing you're estimating. This is why plumbers, and electricians give you reasonable estimates. Heck when we hired movers after buying our first home they estimated two hours, and were done just a few minutes before the 2 hour mark on the spot.

The issue with programming is, your tools change, your approach changes with new iterations (usually for the better!) and you notice new problems, or sometimes you do one thing in 5 minutes, and your second run through using a similar approach is broken because it requires an edge case you didn't anticipate code wise.

It took 12 weeks to ship an MVP I thought would take 3

Related Stories