In Praise of Top Down Programming

https://www.i-programmer.info/professional-programmer/i-programmer/3864-in-praise-of-top-down-.html

by kiyanwang • 3 years ago

95 67 3 years ago

tabtab6• 3 years ago

"Stepwise refinement" is an intuitive and powerful concept, in part because it helps you grok and debug in a "fractal" kind of way, level by level. But it does have limits at a larger scale. (Event-driven architecture is often better for big.)

It's something Functional Programming either lacks or doesn't do as smoothly. The intermediate results and state are highly useful for debugging, and Functional doesn't "like" intermediate state. Yes, it can be emulated, but the emulation is rarely good as the real thing. For one, the emulated state lacks useful variable and token names, because it's machine-generated.

Ma8ee3• 3 years ago

Functional doesn’t like state. That’s the whole point. Functions that can be run in isolation with well defined input is much easier to debug.

tabtab6• 3 years ago

In theory. Practice has proven messier to many. Maybe some have a "functional mind", but they shouldn't extrapolate their head to others. Functional has been around for 60+ years. If it were the magical productivity and Lego-modularity golden hammer, it would have mainstreamed already. (Long LINQ is also hard to debug for many of us.)

ParetoOptimal2• 3 years ago

The intermediate results and state are highly useful for debugging, and Functional doesn't "like" intermediate state.

s/foldr/scanl

Only sort of kidding. Can you give aconcrete example of some task you'd like intermediate state around?

Also, do you know about the validate monad?

tabtab6• 3 years ago

       x = foo01();
       y = foo02(x);
       z = foo03(x, y);

Contrast with:

       foo03(foo01(), foo02(foo01()) );

x, y, and z were named by humans, giving them domain meaning, whereas a debugger would give the second one no name or machine-made names for intermediate state. (Here, z, y, and z have no meaning because it's a foo-bar example, but in practice they'd have better names.)

The first is also often easier to read than the second, at least to my eyes. I know devs who can read the second quickly, but I don't think it's the norm. The LINQ "dot style" can be a little easier to read, but has similar problems.

tyingq• 3 years ago

" Object-oriented design isn't top-down, even when it pretends to be. The decomposition provided by objects is a model of the real world, which is after all where the idea originated"

I've seen plenty of code in "OO" languages that is just the sort of top-down imperative function type thing being described in the article. That is, people using OO solely to encapsulate state and variable scope, using methods more like functions, versus trying to model the real world as objects. I suppose though, that's not "OOD" then?

gloryjulio• 3 years ago

The most usual patterns I have seen r using interfaces instead of inheritances and tagged union, and that's pretty much about it

sdze2• 3 years ago

Isn't true OOP about Messaging and not about modeling objects of the real world?

PhilipRoman• 3 years ago

Like many other religions, OOP has split into several competing factions, each with a slightly different interpretation. To complicate the situation even more, in many areas of the world OOP has been mixed thoroughly with the local pre-OOP beliefs from the dark ages.

lightbendover3• 3 years ago

I don't think I would trust any system that focuses purely on modelling the real world.

hinkley4• 3 years ago

We often have to model legal constructs. Short of working for organized crime there is really no way around it.

rvr_• 3 years ago

For the "play chess" example a state machine is the perfect place to begin modeling. Games, as a rule, are event-driven and are commonly coupled to a event-loop that triggers the transitions of the state machine. I think top-down decomposition has it's place, but the examples used to describe it should be given a little more thought.

mkehrt• 3 years ago

I think the advantage of chess for something other than top down programming is that you're pretty much 100% what the API you want for the chess state machine is before you start. But how common is this?

guhcampos• 3 years ago

I have the feeling that TDD-oriented people like me tend to do a lot of top-down, at least mentally. I'll frequently write the general behavior (in my case either the API or runtime params and inputs) then work my way down to make it work.

Somewhere in the middle, however, we all need to go down the layers to either store something in a database or figure out how to work on a collection of items. This turns the process bottom-up to some extend.

Then the very last phase is going back and forth until both solutions meet in the middle.

hinkley4• 3 years ago

It's a constant problem with trying to talk about process. It's much easier to see the struggles of others, and not your own. I've had a couple bosses who probably could write a book on management, but to a man, their thesis about what makes them successful and which bits are necessary always misses some critical piece, usually lamented in private amongst the senior people.

One of these, talking about his own mentor, asked me once, rhetorically, "is he successful because of his theories, or because he uses the same people on every project?" Wish that some of my previous bosses were that aware. They didn't always see the detailed work their senior people were doing to keep the wheels on. You always assume they see that about you when it comes to who they chose to report, but to hear them enumerate the virtues of the team, and omit such things, that stings.

There are certain fictions we engage in different strategies for solving problems as well, and there's always a little Socratic method in the middle where we pretend like we haven't seen the solution at the end, and go through a farce of arriving at it step by step. That may reduce the importance of sudden epiphanies, but it does not remove it.

Izkata2• 3 years ago

at least mentally.

Even without TDD, that's kind-of how I go: I'll have the top-down general structure in mind, but not in the code, and write the actual code bottom-up. That way I encounter problems with the implementation details sooner, and can more readily adjust the overall structure without feeling bogged down by what's already written.

In the broader structure though it's more like a through-line: One nice input/output, written in that bottom-up-with-top-down-in-mind way, all the way through a use case, then expand sideways with the other inputs/outputs built on the same lower-level functionality.

gashmol• 3 years ago

Pure Top down only works for small problems or problems you already know how to solve. Its main issue is that you don't know whether the solution is correct until you reach the bottom.

Structured Design, which is a quasi top down method, works better in real life. Its main issue is that it didn't survive the OOP hype.

nine_k2• 3 years ago

"Everything should be built top-down, except for the first time" (from Perlisms, 1970s or so).

Approaches to neatly building a well-understood thing are radically different from approaches to building / tweaking / rebuilding a new, so far untried thing.

Software engineering is most valuable around the latter kind of thing; the former kind of thing is usually already built and open-sourced.

hinkley4• 3 years ago

How often have you gotten to build things for the second time?

The dilemma there is that most of us only get to build things for the second time if we are working on a copy-cat product from a competitor, and those are not the sorts of projects we glorify. Everyone wants to work on new ideas, not copy-cats.

7thaccount• 3 years ago

Perl had a "write one to throw away" philosophy that you should write a quick and dirty prototype to understand the business problem and then you could design the full thing with an actual understanding for how things work. Stuff was still complex back then, but probably a lot simpler. I doubt it's possible for a project past a certain size.

nine_k2• 3 years ago

A lot of contract and some corporate work is about building things n-th time. A web form like many other web forms, a CI pipeline like many other CI pipelines, etc. Nothing fancy; 99% like last time elsewhere, just reshuffled to match the local conditions.

It's not exactly the same thing, but the amount of well-known stuff in it is pretty large. Here's where frameworks like Rails, or Django, or Next.js shine: they have much of the expected, n-th time code factored out, you need to glue predefined parts in predefined ways, and get an expected result.

BiteCode_dev7• 3 years ago

I usually champion the "let's start with the UI" state of mind in every company I go.

This has several benefits:

- it forces to confront reality of the problem and field

- you have to talk to people, and understand their needs

- you almost get your backend public API out of it for free

After that, you don't really need to do full top down, you can split in subsystems, and works on small parts of those subsystems.

It doesn't have to be a good looking or sophisticated UI. It can be a ugly, as long as it's practical and validated by the end user.

But this works only well the if the problem can be well defined and you know how to solve it.

If you need to figure it out, top down doesn't work as well.

MrLeap• 3 years ago

A downside to this is that some types of stakeholders lament the apparent glacial state of progress backfilling a UI with functionality.

A tick/tock of UI/functionality keeps those people from getting depressed during demo meetings.

BiteCode_dev7• 3 years ago

If you have demo meetings that matter, you likely have a designer, in that case the UI won't be ugly.

jugg1es• 3 years ago

I have always had great success when systems are designed from the UI first. It really helps you break a new business case down to the point where the backend code just starts writing itself.

I suspect you are coming from a design background because people who have never built a UI before are always very suspicious of this suggestion.

BiteCode_dev7• 3 years ago

I don't come from a design background but my father drilled into my skull that nobody cared about my beautiful skills at work. They cared about their problems and if I could solve them.

Coding just happen to be a mean to solve their problems, but they don't understand code, and I don't understand their problem.

After a lot of frustrating client/dev interactions, I came to read "don't make me think" from Steve Krug. I realized the UI was the only language I shared with my customers to start a meaningful discussion.

This turned out to work well, and got me paid, so I sticked to it.

sdze2• 3 years ago

I coach Product Owners the same way when trying to create a backlog.

retrocryptid4• 3 years ago

I know it's passe, but I generally like to start with a set of requirements first. In the old days, when we were doing XP instead of agile, we didn't feel the pressure to "code something, anything." Now, unless I check in a couple thousand lines of code per day people think I'm slacking.

But... not trying to say "build the UI first" is a bad choice if it works for you. I do mostly backend work on "high trust" transactions, so I tend to look at the API first and see if it matches the requirements i have at the time.

tabtab6• 3 years ago

A good many times the customer doesn't spot important issues until they see it in a UI, preferably interactive. The design process is done best with a combination of formal written requirements and mockups/demos.

cestith2• 3 years ago

For some applications, like an API-driven backend made to serve other applications, you can consider that the API itself really is your UI. It's how your users interface to your code. Your users just happen to be other developers whose code had it's own UI too, maybe a text or graphical one or maybe another API.

skybrian• 3 years ago

XP as I remember it was about implementing one customer-relevant feature at a time. This isn’t too-down or bottom-up, it’s doing a thin vertical slice. There shouldn’t be code that isn’t used or buttons that don’t work.

We did work from a designer’s mockup, though.

BiteCode_dev7• 3 years ago

Trouble is, how do you get this set of requirements?

The UI is the answer to that.

beached_whale• 3 years ago

This is how I start with libraries. How do I want to use it, and start with some code that will eventually compile. It acts as an integration test too. But this starts to expose issues earlier with my understanding. Then if the interface doesn't work, it's easier to fix earlier.

bakul2• 3 years ago

There is a third way: define, explore and build a model of the problem space your program is supposed to be addressing. This forces you to understand its properties much better. Once you understand it better you can likely implement in any of the common programming languages. This is also why I like languages like Scheme or APL or k for exploration.

Often the UI addresses only a subset of this space -- it is only the visible part of the iceberg as it were! There is a lot going on under the UI surface. Not to mention you may want more than one (kind of) UI.

leobg• 3 years ago

Sounds interesting. Can you give or link to an example?

bakul2• 3 years ago

The idea is quite old -- See for instance https://en.wikipedia.org/wiki/Formal_methods

For an example see https://en.wikipedia.org/wiki/Vienna_Development_Method#Bank...

The idea is to model the key aspects as mathematically rigorously as possible while ignoring lower level details. But even if you are not as rigorous, the process of creating such a model can help clarify things. As an example, if you are developing a file system you can abstractly define operations on it (read, write, create file, dir, etc.) and one can come up with a huge number of implementations satisfying this but as you add other requirements or constraints, quite a few choices are no longer meaningful. For example you want to implement a FS that takes advantage of NVMe SSDs, they put certain constraints. So then you need a model for an SSD (erase blocks are multiples of data blocks, write must be preceded by erase, etc. etc.). The idea is to model all this using a few abstract data types and state. You need not strive to programmatically derive an implementation from such a spec for it to be useful.

zwieback• 3 years ago

It's not all one or the other, I'm more of a layer-at-a-time developer and within a layer development can be inside-out (start with an algorithm and add an interface) or outside-in (start with an interface and mockup, then add the internals later).

cestith2• 3 years ago

I mostly like to choose a data structure that's handy for modeling the data I'm worried about first. Then I write code to manipulate that data, including getting it into and out of other formats if necessary. Then I worry about what the application really needs to do with it. Once everything's a presentation or transformation of the central data structures, the code becomes fairly obvious. One I have the data handling and the UI, often the UI is just another case of presenting or updating the data.

sumosudo• 3 years ago

Meh. Just Common Lisp. Have your cake and eat it from the top or the bottom, at the same time?! mmmm, maybe a new paradigm is in order ;)

lebuffon• 3 years ago

Forth will echo your "Meh". How much of top-down is because bottom-up is so @#$%! hard in non-interactive languages.

Jtsummers• 3 years ago

In a non-interactive language, you need to create a "harness" (aka, a program) to run your lower level functions (here I mean in the "top-down" sense of functions at the bottom of the call tree) independently. A test framework could accomplish this, and so you'll see people who do a more TDD-inspired approach will have an easier time conceiving of how to develop their program from the bottom-up. If you don't have a test framework or other framework assisting you then you end up writing "throwaway" programs (I often don't throw these away, they become utility programs when I'm done, though more for diagnostics and testing than for end users). People see that as wasteful, so they don't do it even if it might speed things up for them because it gives them earlier validation of their design and requires less rewriting if they find their design is invalid.

Interactive systems like CL and Forth (as it's commonly implemented) give you that framework for free, it's already written. So why not take advantage of it?

charlieflowers• 3 years ago

If I'm optimizing for my brain, my strengths, and my programming style, bottom up is better. With a particular focus on nailing down the hardest or murkiest part, then the next hardest/murkiest, and so on. This phase can be ugly code if that helps ... the point is to uncover unknowns and let points with the fewest degrees of freedom become the first stakes in the ground.

When there are no particularly hard/murky parts left, it's top-down from there, and that let's you choose the approach for various portions of the solution (such as, this part will be pure functions, this part will be based around an arena allocation, etc.).

Even though I mentioned both bottom-up and top-down, the 10,000 foot summary is bottom-up, because it starts there and the most time is spent in that mode.

Whenever I get away from that and go top down too early, I'm likely to run into some area that either gets reworked, or that I compromise on and _wish_ I could justify the time to rework it.

(And also with the caveat that some things are simple enough you could do them any way you want, and those are good candidates for top-down, which also seems more comfortable for more junior devs).

UPDATE: However, I completely agree with another commentor -- start with the UI first! I don't think that contradicts bottom-up ... it's about fleshing out both the requirements and the _potential_ requirements.

Sakos• 3 years ago

I have to do the same thing. If I spend too much time on everything else, I procrastinate on the hard stuff until I've wasted too much time. If I start with the hard stuff, I'm a lot more motivated and productive. That doesn't mean I'm not often paying a lot of attention to the overall design of the code and where/how things are being called. I'll often step up and down the layers to refractor and improve the code surrounding what I'm working on, since that code often has a big effect on how I can implement my changes.

However, I'm also in a lucky position where I'm not responsible for the UI or any customer facing stuff. My users are internal services that interface with my APIs, which rarely need to be changed.

brandly• 3 years ago

That site's layout is like 80% ads, 20% content!

chankstein38• 3 years ago

In my more complicated personal projects, I'll usually do this in comments in the code. Basically pseudocode to just expand out and then I'll either write things in between the comments that actually add the functionality or I'll just leave it at the top as a kind of "this is what this iteration does" because I'll also frequently complete an idea and then I'll have some other idea with it that requires refactoring and I'll make my second file etc!

hinkley4• 3 years ago

In a top-down vs bottom-up world, the Mikado Method is the middle-out approach.

tpoacher• 3 years ago

Thanks! Will read more carefully, but at a glance it sounds suspiciously similar to effect sketches and feature sketches from M. Feathers' book on working with legacy code, rather than a design methodology method per se. (is that the case?)

EDIT: oh I see, it's basically the same concept as effect and feature sketches, but at the design stage rather than at the "dealing with existing code" stage. The intent is to figure upstream and downstream dependencies in your design in a structured manner. Cool.

xyproto• 3 years ago

Top-down programming works great when using GPT-4 as a coding assistant, as long as it is told to not stub functions, use placeholder or TODO comments, but use as many undefined functions as it wants. But when it starts calling the same function recursively, it's time to tell it to stop the top-down approach and write the functions that does the actual work.

intalentive• 3 years ago

Frameworks often impose a kind of top down structure.

adamddev1• 3 years ago

This is basically the approach I learned from HTDP, and it's helped me greatly for tackling some tougher problems. Basically you make a "wish list of" the functions/steps you'd like to be able to do, and then as you work on writing those functions in your wish list, you do the same thing again, going down and down until you hit the bottom and do concrete action with primitives or whatever. It's really cool to see how this can work by composing pure functions and types.

https://htdp.org

In Praise of Top Down Programming

Related Stories