Agile Scrum: Delivering Broken Software Since 1991

http://www.writemoretests.com/2011/09/agile-scrum-delivering-broken-software.html

by peteretep • 15 years ago

102 54 15 years ago

This just makes me feel like scrum isn't worthwhile. Here's how my work flow goes, as a developer: There's an issue tracker. A handful of issues are assigned to me. I know, by memory, which issue's important, because it's always the one people are yelling about the most. I write the code for the issue, test, commit, get feedback, close the issue. Done.

I could not remember all of this terminology. Stories? Sprints? Clearly this is something above and beyond what I ever do. How do people do it? Does it really deliver on its promises?

peteretep• 15 years ago

Author here...

I once started a contract at a company where I was working with a permie on this hugely popular site running off the God-awful blogging software. Everything was a fire that had to be put out RIGHT NOW. And there were two of us.

Coworker had earned himself a reputation for being unhelpful and unproductive. This was partly because he had sociopathically bad communication skills and hated people, but he actually worked hard, and productively. But everyone thought he was lazy and didn't get shit done.

Essentially his workflow was: work on a task. Someone comes and says "DROP EVERYTHING, YOU NEED TO ADD A WIDGET TO THE BLOG NOW". So he would start on that, and 3 hours later, someone would come to him and say DROP EVERYTHING. And three weeks later, he'd struggle to explain why he hadn't done much of it.

So we (well, I) split the work in to Stories. We estimated them, and stuck them up on the wall behind us. We found someone who was meant to be the Product Owner. Whenever anyone came to ask us anything, I'd intercept them from talking to the coworker, ask them for details on the task, estimate it, and put all this on a card. I'd then call over the Product Owner, and with the Requestor, we'd put it on the Story Board.

Suddenly items weren't /that/ important any more. With the opportunity cost suddenly marked out, many things found themselves inserted far below the current task being worked on. We'd burn down time against each task. We could tell you where we'd spent our time, and also, we could tell you what we'd achieved recently.

Essentially then, with a very small amount of overhead (like learning these terms ;-)) we suddenly had a system for prioritization and accountability. I did very little programming in the six weeks I was there - the productivity gains we saw - both real and perceived - made it look like there were 5 of me working fulltime.

MostAwesomeDude3• 15 years ago

Cool story! This is interesting.

daemin2• 15 years ago

In this case it seems to be a tool forthe developers to manage the expectations of the 'other' non-development side of the business.

wccrawford• 15 years ago

It sounds like you're in a 'bugfix' position, and we found it hard to integrate that into Scrum, so I'm not surprised it's confusing for you. In the end, we ended up just saying that bugfixes were more important and slicing off some time for how much we expected, and using the rest of the time for Scrum.

Once you get into Scrum, it isn't hard to remember the terms. A 'story' is just a feature to implement. It's called that because it's supposed to be seen from the eye of the customer, rather than the developer.

A Sprint is a timeframe for completing a bunch of stories. Usually 2 weeks, but some find shorter sprints are better, and others need longer ones.

As for the question: Yes, it was WAY better than waterfall for us. It forced everyone to get organized. Management could no longer be ignorant about what they were doing to the developers and testers on a daily basis. Instead, they were forced to actually put things on a timeline and decide what was more important, instead of just getting upset when everything didn't magically happen.

As a bonus, though, they gained the ability to estimate when things would be implemented, and changes to the system were rolled out every couple weeks, instead of going months between releases, and having a ton of bugs when it finally did launch.

irahul• 15 years ago

> It sounds like you're in a 'bugfix' position,

You might work differently, but that's not bugfix mode. Everything is an issue in an issue tracker is a development model followed by many projects. I don't think OP is in bugfix mode.

> Once you get into Scrum, it isn't hard to remember the terms.

So not very different from not having terms at all? Which is what OP is following.

> A 'story' is just a feature to implement.

And since features/tasks it interchangeable, I would rather not introduce another word.

> Yes, it was WAY better than waterfall for us.

Of course, but there are methodologies other than scrum and waterfall - like the one OP mentions. I don't see how scrum terminology improves over the model OP is talking about:

1. Having a list of tasks to work on.

2. Fuzzily committing on the tasks which will be done in a given cycle(5 days, 10 days...)

3. Repeating.

> Instead, they were forced to actually put things on a timeline and decide what was more important, instead of just getting upset when everything didn't magically happen.

How is scrum doing anything different? I have a list of tasks, I can only commit to a certain ones which I am going to deliver this week, management can choose which ones. Isn't that the same with having a list of issues on an issue tracker?

> they gained the ability to estimate when things would be implemented

Your team must be super efficient. I have been doing this committing for the week for last 4 years, and I can recall only 2 or 3 weeks where I haven't lapsed. Whatever I tell you is my best guestimate, nothing more.

> instead of going months between releases, and having a ton of bugs when it finally did launch.

You can release early without having all this extra terminology kludge.

MostAwesomeDude3• 15 years ago

I am in what might be accurately called a "student" position. I work not more than 4h/day, vary wildly in capabilities, and occasionally need training on the spot.

We don't have customers. We have the sysadmins, who want features, and ourselves, who want our names to be associated with quality software. Maybe it helps that we're an open-source-only shop.

Stories seem like they might not work well with what we do. For example, a bug I was given in January was "implement WebSockets in this proxy so that we can have WebSockets in the backend." Estimated time: 2 months. It's now September and we are just now considering the relevant code battle-ready and getting it out the door. Turns out that the bug didn't really address any of the design decisions, so those have all been slowly reevaluated over the spring, and I ended up rewriting an entire library in July.

This sounds interesting, though. Thanks for the story.

AndrewDucker• 15 years ago

Scrum is more useful when there are a bunch of people at work producing new functionality, and therefore a need to talk to each other on a regular basis, produce code that doesn't walk all over each other, and get the functionality (or "stories") in that the users want.

SeamusBrady• 15 years ago

Here here. I have worked in places where they had a bog standard waterfall development process but used agile buzz-words like 'story'. The result? Can you imagine a cross between a giraffe and an car sized amoeba? Yikes.

dools• 15 years ago

From my (limited) experience, scrum such as this is really geared towards dealing with the interface between the technical team and a bureaucratic environment in which they need to operate, although even on our small (2 - 3 man team depending on the project) we use elements of this methodology. It's just that we don't have anyone to report to, per se.

So really I think this applies to larger software teams, or small software teams (ie. 1 man technical departments) surrounded by non-technical people to whom the development process is traditionally a frustratingly black box.

rpwilcox• 15 years ago

This is why I don't think the product owner should have the sole power over the backlog.

In addition to business stories that need to be meant there are at least two other areas whose stories are as important: development and testing. Maybe ops too, depending on your structure.

With a product owner that doesn't listen to any area beyond business, of course stories from those other two departments are discardable crap. There needs to be conversation from the departments involved. Sure, if priorities change in mid-sprint that's one thing, but there are other voices that need to be heard beyond business, when planning the story selection.

Likewise, sole product owners might think, "Oh, if we have these programmers work 16 hour days, instead of 8 hour days, we can get twice as much work done, and make the deadline!". Again, other voices (voices of reason) need to be heard in this situation too.

shareme• 15 years ago

Scrum is for those firms who do not want to hire a real product manager and empower one after they are hired.

kreilly• 15 years ago

Care to elaborate?

arctangent• 15 years ago

Scrum is a great idea and I use elements of it to manage how I organise my work. If I've blocked out a month to work on one piece of software then I will sit down with the relevant team and we'll agree together a "prioritised product backlog" consisting of a combination of new functionality and bugfixes.

However, what Scrum doesn't seem to address is how to deal with the bureacracy in a large organisation, especially where you don't have all the resources you need to meet demand from different teams who want you to do work for them.

It's been a fairly exceptional month, but I've been forced to switch projects 3 times recently owing to differences in opinion (at board level) about which piece of work is more important. This has obviously been less than ideal from an engineering perspective and has left me with just a couple of weeks to develop a fairly important web application.

I'm currently thinking about how I could extend the Scrum model to account for multiple projects but I'd appreciate any insights people here on HN have to offer...

DanielBMarkham• 15 years ago

Work 100% on one thing at a time and decrease your cycle time (length of sprint) to allow for rapid shifts.

If you have to go to a 3-day cycle time -- as long as you are 100% dedicated -- do that.

This, of course, assumes you've understood the agile/scrum concepts enough that you can scale them down to a few minutes each. If you're still having 30-minute standups and 4-hour sprint planning sessions, my advice is not going to work.

Here's the thing: no matter how stressed out and context-switching the work environment is, you can only work on one thing. So can you predict what you are going to deliver and then deliver it? If you can't -- if you can't even predict day-to-day what you are going to do -- then you're well and truly hosed. Scrum or agile isn't going to fix that. (BTW, there's another answer if you are truly in an environment without any structure at all. In that case, I'd look seriously at some lightweight kanban)

api• 15 years ago

My experience with Scrum is that it has a tendency, especially if implemented "textbook" with absolutely no flexibility, to completely de-emphasize any kind of architecture, forward thought, elegance, or big-picture thinking about the code.

The result tends to be a giant snotball of hacks. In my personal experience the biggest casualty is efficiency, followed by maintainability and scalability.

The fact is that good programmers write good code, and bad programmers write bad code. There is no management "methodology" that can make bad programmers write good code. There are, however, methodologies that make good programmers write bad code. Scrum at least can be one of these.

Wolf_Larsen• 15 years ago

Well said.

Aloisius• 15 years ago

The textbook way of running Scrum is to have a retrospective at the end of every sprint where you find faults with the process and tunes it for your specific environment. If you skip over that part or worse, refuse to adjust the Scrum process because it breaks some rules, then You're Doing It Wrong.

bad_user• 15 years ago

Ah, the standard excuse for failed Scrum implementations -- you're doing it wrong.

fabjan• 15 years ago

There is nothing textbook about Scrum without flexibility.

alexkearns4• 15 years ago

You're spot on. Scrum (and indeed agile development in general) is good a getting a group of average developers to create an average product.

It is also good at getting a group of great developers to create an average product.

rickmb2• 15 years ago

Scrum doesn't de-emphasize those things, bad programmers do.

I'm just following your logic here, which I agree with up to the point where Scrum has a negative influence.

I don't see how Scrum can encourage good programmers to write bad code, since it gives them all the room they need to write good code, and lets them decide how much time and effort they put into architecture etcetera.

Bad programmers however tend to get exposed by Scrum, since without anybody telling them they have to consider architecture, they will produce said giant snotball of hacks.

romaniv• 15 years ago

Scrum favors incremental/additive changes, which leads to inconsistencies. Often inconsistencies are too large to simply fit with some other task, and a separate task just for reconciling everything never gets prioritized. Viola.

mrgoldenbrown• 15 years ago

I don't have a textbook in front of me, but I doubt textbook Scrum says "be inflexible"

buff-a5• 15 years ago

There are number of classic failure modes for Scrum. One of them is to not have a Product Backlog at all, or to only plan it out a few weeks.

Since you claim that in your experience, Scrum de-emphasizes forward thought and big-picture thinking, I can only conclude that you don't do this, and have decided to blame Scrum for this fact.

I suggest that you identify whatever it is that is preventing your team from creating a full product backlog.

Also, my YAGNI detector is going off big time.

grncdr• 15 years ago

I think this previous comment by michaelfeathers supports what you're saying:

"The fact of the matter is that too many people think that projects can be run through the interface of stories and feature lists without paying attention to the quality of the software underneath. And, when you don't pay attention to it, it suffers. This, really, is _Joel's Law of Leaky Abstractions_ applied to process. Business wants to see features, and if that abstraction is their only view of the project, they will be blindsided by creeping quality issues. It's nearly inevitable."

Quality issues aren't always architectural issues, but architectural issues (especially lack of any planned architecture) always cause major quality issues.

http://news.ycombinator.com/item?id=2978806

fauigerzigerk• 15 years ago

What on earth does that have to do with Scrum? Pressure to make compromises on code quality is about as universal an issue as you will ever find one.

narag• 15 years ago

It seems a case of "bait and switch". You've been attracted to a methodology as defined by experts, but get an aproximation by some random PHB. The result is similar to Mr. Bean trying to apply ninja techniques as seen on a Steven Seagal movie.

Sadly, for many of us, it's the only Scrum that we've seen, so expect more hate. Maybe it is a problem with Scrum, after all. What good is a methodology that gets so often misimplemented?

buff-a5• 15 years ago

What good is a methodology that gets so often misimplemented?

Scrum is simple. Its not misimplemented because its complicated. Its misimplemented because organizations are fundamentally not good at developing software. Scrum just puts it in your face. You don't even need to actually write any software to know how and why you are failing. Just attempt to do Scrum, and see what excuses you make to do not do any given part of it.

lojack• 15 years ago

I agree that this problem exists, but I also feel that there is a better solution than whats listed. Stories should be given points for their complexity and not for the time it takes. Additionally stories should only be business oriented, so things like automated testing don't get their own stories.

Since automated testing does take time your team velocity is what's affected, and no one has control over team velocity once a sprint has started. Quality issues that recur affect the calculated velocity and one off quality things (like setting up continuous integration) should add a buffer to the velocity. This should be discussed in sprint planning so transparency is maintained.

rubenbos• 15 years ago

Really interesting article.

I can imagine there are a lot of companies that don't sell Scrum right. For example: teams are still ordered to fulfill a fixed scope in a fixed amount of time. As a team you can't fight that. It's not the fault of the team.

But from your article I understand 'Low Quality Code' and 'technical debt' are always the fault of Organizers. I don't agree on that.

A team estimates (led by their scrum master) how long they will be working on a user story. It's their responsibility to take everything account that's needed. They know how much time has to be spent to get the code right.

The Product Owner just gets the estimates from the sprint planning and knows how far the team can get. The team said they could do a particular user story in 5 points right? If that's 5 points with crap code as a result... well, maybe it was a too optimistic estimate with no space for good code.

neilk• 15 years ago

At least in my understanding of Agile/Scrum, there shouldn't be any cards representing code quality. There are only features that make sense to both the customer and the developer. The time to write tests and so on should be built into cost estimation of the feature.

However, as 'api' says, this tends to result in a mucilaginous mess. There really is a certain amount of work that you have to do on a project which is just about the code itself. I haven't seen any Agile method capture that in practice. A manager who listens to the developers will often suspend Agile in order to get refactorings done.

daemin2• 15 years ago

I don't think there should ever be a scheduling item for 'code quality' (and related activities) in any software development planning/management cycle. If software developers want to be treated as proper engineers at some point in time, we should start implementing quality as a default, not as an afterthought.

Therefore it stands to reason that 'code quality' should always be baked in when estimating software tasks, same goes for any other related and required development activity.

alexkearns4• 15 years ago

Even if scrum works perfectly, it does not empower developers. In fact, from what I can see, the whole purpose of scrum is to disempower developers while giving them the illusion they are empowered.

As the author reveals, under scrum, it is not the developers who decide what new features to add to the software. It is the product owners.

Who decides the priority of the new features under scrum? Not the developers but the product owners.

At least under waterfall, developers could get their teeth into a good chunk of interesting work. They could come up with some fancy architecture or something. But under scrum, all work is broken up into bite-sized chunks of work that are rarely challenging and often highly repetitive.

Under scrum, the developer's time and ingenuity are not valued. If a product owner out of the blue decides to ditch a new feature or radically change one, wasting the work a developer has done creating it, he does so without a moment's concern about the developer's wasted time and effort.

In short all that scrum offers developers is a small amount of control over their workload. Compare that with developers of the past, who would conceive software and build it themselves the way they wanted, and you can see that, far from empowering developers, scrum has reduced them to little more than assembly line robots, at the beck and call of their masters.

I am not saying that we should go back to the past when developers were in charge or everything but to say that scrum empowers developers when it clearly does not needs to be challenged.

bobbles• 15 years ago

This is the exact opposite of what SCRUM hopes to achieve. If a product owner decides to bin a feature, it's because it has been weighed against the options available and decided that this feature is no longer valuable to the company.

'Wasting' the developer time as you have stated it is a misnomer, as I would say developing a product you know wont be used by a customer is wasting time.

Within SCRUM, a developer should have full control of how software is developed, if a product owner is trying to dictate the way in which a product is developed, they are stepping too far and the ScrumMaster needs to get them to back off.

The way for a product owner to determine how a product is developed is to have a clear set of requirements in place that are testable and achieveable.

alexkearns4• 15 years ago

Everything you say seems to back up my point about scrum not empowering developers.

"If a product owner decides to bin a feature, it's because it has been weighed against the options available and decided that this feature is no longer valuable to the company."

And therefore the sweat a developer has put into creating this feature is not valued. Scrum pretty much ensures that developers will at some point create stuff that will never see the light of day. Hardly empowering.

"'Wasting' the developer time as you have stated it is a misnomer, as I would say developing a product you know wont be used by a customer is wasting time."

I would not necessarily disagree with you but it is hardly empowering for a developer.

Scrum is designed to empower the business, not the developer.

unobfuscate3• 15 years ago

The key here is to separate how we things done from how objectives are defined. Using Waterfall instead of Scrum does not inherently change who gets final say on what will be accomplished. If the project manager or project owner want to have complete control of what gets done, will little to no input, your process won't change this.

Scrum is about visibility, accountability, and continuous product evolution. It's about controlling what gets done, and when we do it, not who defines the goals.

It's your companies loss if they don't take the input of both the developers and customers when deciding what to do next.

buff-a5• 15 years ago

Some people say that this should never happen – once the Sprint is planned, it’s sacred, and that any changes to it need full team buy-in – any significant number of changes need a complete replan of the Sprint. I’ve not seen this so far in practice, but it sounds intriguing.

Some people being, in this case, the people who defined Scrum. If you allow a product owner to walk in and add stories to a sprint, and the team does not have the choice to "reset the sprint", then You Are Not Doing Scrum.

The Organizers are rarely empowered to change any of those thing, so they change the one thing they can, but shouldn’t: the build quality.

Again, if the Organizers are not empowered to change scope, deadline, etc, then They Are Not Doing Scrum.

So Agile Scrum doesn't deliver broken software. Fairy tale fantasists who believe they can ignore the central tenets of any process and get the results deliver broken software. Essentially: "People unable to implement a simple algorithm deliver broken software".

jroseattle• 15 years ago

From my own experience, where Scrum has broken down is the promise of, after the sprint, you have something you can "release".

Of course, the term "release" can mean many things to many people. If you're building something substantial, an initial sprint or two may be dedicated to simply getting some overall framework in place. It just depends on what you call "release".

In the cases where I've seen failure, "release" could mean only one thing -- a shipping product outside the organization to customers. In spite of declarations from the dev team that no product going to customers would be delivered. And, at the end of a sprint, with no product prepared to go to customers, product management decided scrum was a failure. I'm not making this up.

TheCapn• 15 years ago

Not sure why, but the site is blocked at my work. Can I get a pastebin of the content or something similar please?

Zimahl• 15 years ago

I've done a fair amount of Scrum and have seen it succeed and fail. Scrum works and is great, but like most tools it works only for specific types of jobs.

I wouldn't use Scrum in a situation where the final product is somewhat defined and is expected at a certain date. Any situation where you ask yourself 'why aren't we using waterfall here?' is one where Scrum will inevitably fail.

Scrum is typically great for startups (and sometimes established companies) that are building something that is more of an idea than a product and time is irrelevant due to being able to push new features out later if they aren't done. It helps if your customer doesn't expect polish and is fine with incremental improvements.

To use the Minecraft analogy, Scrum would be a great process for Mojang. The target is moving, the features the developers thought were significant yesterday have found to be of no concern to the users, and thus they can drop something and focus a new feature. The next version has what's done, not necessarily what was expected. Waterfall wouldn't work in this situation at all.

Agile Scrum: Delivering Broken Software Since 1991

Related Stories