cabalspecter

This is for me the dissonance between agile methodologies and reality. Agile methodologies assume having a flexible scope and time due to the project unknowns (technical, business, social, etc.). But usually budgeting in the real world is required to follow: I promise to deliver X scope that’s going to cost Y dollars in Z time.

I don't think the gist of successfully developing software has changed that much since the early 90s, at least in consultancy: You work in iterations where the team and client develop, refine and prioritize the features of the system until the system is good enough to be considered finished. There is an initial vision and strategy that needs to be developed in the first iterations. The UP implementations describe ways on how to do this, though the agile community has come up with more creative frameworks: design sprint, product vision box, customer service map, etc. The strategy is reviewed each iteration with the re-priorization of the features. For this it's important to have revisions on the documents created on the first iterations.

The challenge as I see it is that clients usually don't buy into this and they believe that developing software is like making sandwiches or turning buttons on and off when in fact is a discovery process of abstract things. They then demand a fixed budget, fixed time and fixed milestones for a fixed scope ("You are the expert, you should know how much time this takes."). To sell the project the salespeople from software companies deliver this, those who don't are considered incompetent. In this conditions agile methodologies are prone to fail but not for the methodology, but because most of the time those sold promises are done with incomplete information or tailored to be optimistic to sell the project. Metrics from agile methodologies show from the beginning that the project won't meet the expectations and then chaos ensues: they stop collaborating with the client to hide the reality of the project, "managers" pressure the team to do more in each iteration, they stop doing the disciplines altogether, iteration and daily reviews are treated as long meetings where people commiting errors are shamed and scolded and frictions start to emerge between all persons involved. This is where I believe the whole notion of "agile sucks" comes from.

Too bad people didn't bother reading the Unified Process book, which clearly specifies the use of iterative and incremental development and the creation of just enough UML diagrams to communicate and document a system design. It even argues that working software should be used to describe an architecture in the Elaboration phase rather than a 10000 words document. It can actually be considered as one of the first agile methodologies. RUP and other flavors of UP just added tools and guides aimed to facilitate the implementation of UP, but it was misunderstood that every template and section must be filled out instead of actually thinking what to use depending on the project. Unfortunately the whole thing was ultimately bastardized by the non-technical template zombies that thought software development is like building a skyscraper.

I believe the efficiency got "worse" because it revealed the harsh truth earlier which the Gantt chart hides... until the very end.