Using a logbook to improve your programming

https://routley.io/tech/2017/11/23/logbook.html

by jmlr • 9 years ago

295 74 9 years ago

leviathan• 9 years ago

This article seems lacking some examples. I have found that such ideas are nice in theory, but when it comes to practice, they tend to fall short.

So the author says that it's a good idea to write logbooks, and that it will help. But no mention on how it would help, and what would you actually do with them after you write them up.

A few concrete examples would have been nice.

If the idea is to help you solve the problem more efficiently, why should it be formalized? as opposed to scribbling notes on a notepad that you throw away at the end of the day.

taeric• 9 years ago

Advice of this sort is tough. I really really want something prescriptive.

I suspect it is ninety percent confirmation bias. Looking back at successes and assuming something in common of all of them had to be contributive.

Not claiming it is all luck. Just saying that outside signals usually bear more meaning than given credit.

barrkel2• 9 years ago

Well, I use this logbook approach, in a text editor with persistent undo (which allows a few tricks, like emulating folding by deletion and relying on undo history to unfold). I approximately follow this algorithm:

- Consider the high-level goal; can it be completed immediately? if yes, do it, otherwise break it into smaller parts.

- First part is usually some kind of research. If it's a bug, it'll be a reproduction. If it's a well-specified feature or a refactoring, it'll be locating all areas of code that need to be updated. If it's design, it'll be an initial sketch.

- Subsequent parts depend on the task at hand. For a bug, I generally try and work through to a minimal set of reproduction steps. For a feature, it'll be a generated list of tasks that came out of the research step.

- When working on any given task, I add sub-entries in the form of a dialectic. For a bug reduction, I come up with a hypothesis, then try and prove or disprove it by testing. For design, I propose one approach, then consider the implications; then another approach, and so on. For a feature, tasks generally get added for any "todos" that crop up as I implement things, to ensure I complete everything.

I essentially record salient details of my internal monologue to ensure that I can pick up context again easily, and generally fit more into my working memory by offloading it into text.

I reckon it made me perhaps twice as effective an engineer, particularly in distraction-prone environments. It's also made me a lot more confident that I'm not relying too much on memory as I get older - not that I think I forget much now, but I believe I won't be very dependent on a large working set.

nine_k• 9 years ago

Emacs Org mode works wonderfully in a scenario like this. Consider trying it.

fao_2• 9 years ago

Do you have any ideas on how to adapt this to someone who doesn't have a "present" internal monologue?

If I concentrate, I can sense something akin to the typical "internal monologue", but it isn't present enough to be copied down, and even when it is it tends to move so fast that I only really get the conclusion.

firmgently• 9 years ago

"It provides a framework for solving hard problems by encouraging you to break them down into a series of smaller ones.

"It helps you focus on the task at hand by providing immediate context on what you’re doing. If you forget or become distracted, you can quickly get back to your train of thought.

"It helps you learn quickly. You can observe your method for problem solving, see what works and what doesn’t and make improvements."

Saying "no mention on how it would help, and what you would do with them after you write them up" feels inaccurate to me (although the observation about the lack of specific examples is correct).

geezerjay3• 9 years ago

Saying "no mention on how it would help, and what you would do with them after you write them up" feels inaccurate to me (although the observation about the lack of specific examples is correct).

To me it is an accurate description, as the author focuses on what he believes are the results while not presenting any tangible example or framework on how to get them.

In short, without any information on the "how" this blog entry boils down to a page and a half where the author states that he believes that writing down things he is doing in no specific way and following no specific method may help him in some way. Well, good for him.

Mikhail_Edoshin• 9 years ago

It works. There's even a psychotherapy method called "problem-solving therapy"; all it does is that it formalizes the process of finding a solution to a problem: define the problem (what you have and what you want to have), outline possible approaches, evaluate, choose. I myself use something like that in my work (software development) with a text file and I can attest that following the steps often helps me to find or choose a different solution than what initially popped up in my head.

It doesn't solve all problems, e.g. a complex problem that is hard to understand are not a good fit; if I notice that what I write is lengthy or that I keep returning and restating it, then this is that kind of problem; here I normally leave the desk and try to sketch it on paper.

Very seldom I look through the file to find why I've made a particular choice; I don't think logbooks are of any use here, it's better to document the design elsewhere, closer to the product. I'd say logbooks are write-only (although they may be a source for the docs).

nicoster• 9 years ago

I have similar ideas. At the beginning (20 years ago) I use windows notepad.exe. It has a trick that if you put “.LOG” on the top of a txt file, every time you open it with notepad, it appends current date to it. Later on I switched to Evernote, then Apple Notes.app and settled. Recently I added a service with Automator to allow me insert current date to a note. Quite convenient.

j_s• 9 years ago

F5 inserts the current date / time in Notepad.

partycoder• 9 years ago

Keeping a journal is particularly useful to resume your work.

Before lunch, or a meeting, or end of day, or when someone interrupts me and needs something, or any form of break, I "serialize" what I am working on in the journal, and then I read it when I am coming back.

In this way I can keep my mind clear and focus in the next activity.

It is also useful for standup meetings, retrospective meetings, etc... when you often need to describe what you did yesterday or this week.

krupan• 9 years ago

Emacs org-mode is great for this.

yjftsjthsd-h• 9 years ago

As is vimwiki, for those on the other side of the fence.

gavinpc• 9 years ago

I've been using org-mode for about ten years to document moment-to moment progress on tasks. I'll give a basic summary of the problem, copy in significant commands and results as I run them, and ask the next questions to drive the next steps.

Some features that make Org useful for this:

- I'm clocked into the task so I can always jump to it easily, even as I hop around other emacs buffers

- supports code snippets with formatting in any language

- it's easy to add timestamped notes or write in free form

- can export the subtree to HTML or Markdown or Latex/PDF, to share with others

- GitLab will apply basic Org formatting (as with markdown), so I can share them directly just by pushing

- supports regex search across all agenda files, regardless of where they are. This is nice if, say, I know I had a similar issue with e.g. some docker command but don't remember which project it was on

meesterdude• 9 years ago

i'd be interested in learning more about this and maybe seeing some real-worldish examples.

deerpig• 9 years ago

Org is already 10 years old? It seems like yesterday that I was using Muse Mode. Then, seemingly overnight the muse discussion lists went very quiet because everyone switched to Org. I still have a bunch of notes in files that was created in Muse.

Derbasti2• 9 years ago

Shameless plug: org-journal automatically creates one file per day, integrates with the calendar, implements searching within date ranges, and optionally rolls over TODO items to the current day.

henryslater2• 9 years ago

In between commit messages, bugs and design docs, I'm not sure what I'd put in a log book.

iAMAGuest2• 9 years ago

- The logic/design of parts of the code base which you might newly encounter

- What work was done for the day, yes you might use something like Jira but a journal can be also good at the same time

- New things you learn.

etc etc etc

henryslater2• 9 years ago

Each to their own of course but it seems to me, there's better places for all of that. If there's logic or design that's complicated and not properly commented or documented, then that's worth doing and sharing with the team.

Work done for the day for me is typically captured in commits or bug updates - that way my colleagues can easily take over if I win the lottery or are suddenly taken ill.

And stuff I learn is learned, so I don't need to write that down :-)

Working in a team, shared information is critical and logbooks just aren't the best way to achieve that.

khebbie• 9 years ago

I think maybe documenting architectural decisions might make sense: http://thinkrelevance.com/blog/2011/11/15/documenting-archit...

keithnz• 9 years ago

I keep a log of things I don't really know but should invest some time in to learn it properly as I make use of it. That's quite useful, not so much the log, but the forced awareness of making a mental note. I've also been thinking of keeping a log (repo with notes) of the incremental experiments I do when learning things.

kilon2• 9 years ago

I don’t keep a log but I use pen a paper to create diagrams which visualize my problem and potential solutions. It has been very helpful and good excuse to rest my eyes from the monitor. I don’t care about dates cause I like to work in a chaotic kind of way and I don’t like planning ahead I prefer exploration and critical thinking during execution.

mattfrommars• 9 years ago

I find this interesting. Mainly because, I've been trying to implement similiar strategy on my daily works to fight off low productivity hours and really figure out where all my time is going. One of the things I've been doing is noting down the time and date when I set to do things. Seeing something similar presented in the post with more formal and clean approach made me happy - as in being on the right track.

But than again, there was TED talk of all this organization is ultimately shifting the energy from actual. Which led me reducing number of 'productivity' application and sticking with a few.

Wunderlist Onenote Evernote - mainly for the phone.

ioddly• 9 years ago

I find writing documentation as I go to be a particularly good form of this. I start before I fully understand how I'm going to solve something, and only check the documentation in once it's correct.

lucisferre• 9 years ago

Based on a some of the comments so far I think people are glossing over the key elements of the logbook. This isn't simply an activity log.

The first step is to describe the nature of the problem as well your planned solution before beginning. I think this can be a pretty powerful practice in being mindful.

All to often I find myself jumping into a half-baked solution while at the same time having only half-understood the problem.

dominotw• 9 years ago

All to often I find myself jumping into a half-baked solution while at the same time having only half-understood the problem.

This is what kills me in tech interviews. Unfortunately its hard to calm down and think quietly when you are being hounded by the interviewer to "think out loud".

cgrusden• 9 years ago

I did not go to college/university for engineering, but just hearing the tools of 'real' engineers is great.

I've been opening up Jekyll in the morning as the start of my day. I write the blog post for the work I'm about to do. I've noticed that I'm getting better at explaining the problem clearly and the solution. When there is a paragraph that feels out of place I yank it into a different file immediately to get it out of the current flow of the post. The paragraph felt out of place because it was probably a distraction from the problem Im trying to solve today. Also, the great thing about this technique is that I have a blog post at the end of the work day that I can look back on and say, "Damn, I shipped something" if I didn't feel like I shipped anything else (common problem in our field of software developers)

torbjorn• 9 years ago

What the author says about aliases in the bashrc being evaluated only once when the bashrc is run when the shell session is opened is wrong.

Go ahead and try this:

Put,

    alias lb='vim ~/logbook/$(date "+%s")'

in your bashrc.

Source the bashrc.

Run the lb command. It creates a new file with a different epoch timestamp each time. This `lb` command doesn't need to be a bash function. It can be an alias and work just fine.

Maybe how this works in zsh is different...

nerdponx• 9 years ago

Zsh is different, although I'd be surprised if this wasn't the Bash behavior.

It doesn't make sense. If you do

    alias e=vim

it's not like Vim opens when your shell starts.

The only way to have a "precomputed" alias is to get the quoting wrong, and leave $() in double quotes instead of single quotes, meaning it will be evaluated before the alias is even assigned.

Izkata• 9 years ago

Their function uses single quotes on format string, so I'll bet they tried it like this:

   alias lb="vim ~/logbook/$(date '+%Y-%m-%d').md"

..which would do what was described.

icc97• 9 years ago

Seeing as the author is using vim he could just have a command within vim.

  :exec ':e ~/logbook/' . strftime("%Y-%m-%d.md")

HumanDrivenDev• 9 years ago

Good idea, but as others noted it was a bit light on details.

Some more concrete suggestions from what appears to be the university of Idaho's Electrical Engineering department can be found here:

http://www.ee.uidaho.edu/ee/power/jlaw/COURSES/CAPSTONE/F05/...

perseiden• 9 years ago

Heh funny, I started writing a "labbook" like this ~6 months ago with markdown writer in atom for every day. I have a background as a Lab technician where I was used to keep a daily detailled labbook so I didn't have to develop that habit..

I like to structure my programming lab book in a similar way, for every task/ bug/ project I try to jot down a description, hypothesis, expected outcome and document my progress with code snippets / outputs / plots. I also keep a Todo list at the top to visualize my current workload. It really helps if you have to come back to a problem later on or just to get a good overview of your current task. I feel lik this also works nicely in conjuncture with GTD to increase my productivity.

kayoone• 9 years ago

I like the idea, i just think MD files might be too limited. I really came to enjoy Onenote because it is so easy to put unstructured data, images, links, videos and even draw on top of it all with a stylus but i didn't use it for every task i do. Will try that in the future as i often have issues resuming old tasks or remembering some details to a problem in the past.

lldata• 9 years ago

Free tip: commit often and write proper commit messages.

loving-g• 9 years ago

If you change organisation and were working on a closed source project, you lose everything. You need to keep your notes on a higher level though (i.e. not describing confidential information). Plus I believe that a commit is generally only covers the third point ("Describe the process of carrying out the method").

jungletime2• 9 years ago

Typora is a great markdown editor for something like this.

neya• 9 years ago

I believe many of us already do what the author suggests, maybe in a slightly different way.

Here's how I use my own flavor:

1) I create a GOAL. The GOAL is my ultimate big picture, which will give me immense happiness. So, the most recent example is, I wanted to build an E-Commerce application.

2) I map out the PATHS to achieving my GOAL. So, what I consider PATHS is:

a) What programming language should I use? b) Which APIs should I use, Paypal or Stripe to accept payments[1]? and so on...

3) I follow a flavor of DDD[2] to come up with the important modules I need to have for a functioning E-Commerce project.

Examples: I will need a Order Management module, I will need a Marketing module (for sending emails, running campaigns). I will need a CMS module to host product pages.

4) For each of the modules, I will create an issue on Github and add a checklist[3] of the stuff I need to get done.

5) Everytime I make a commit, I will reference this issue and the tasks along with it. Eg. "Removed Paypal #12"

6) At the end of the week, I will review the commits and evaluate whether this particular method I used was worth it. A good use case would be experimenting with a library and comparing it with another - Eg. Bootstrap vs Semantic UI. I will use the learnings from this workflow in my future projects.

7) I will also see if something can be automated. For example, for my current E-Commerce project, I use Phoenix/Elixir. It has it's own set of scaffold generators, but they're opinionated (they use Bootstrap). So, for every project I got tired of replacing bootstrap and customizing the generated scaffolds. So, I wrote a custom generator that drastically saves me a lot of time for large projects.

Hope this helps :)

[1] Not sure why anyone would use PayPal in this day and age tho.

[2] https://stackoverflow.com/questions/1222392/can-someone-expl...

[3] https://github.com/blog/1375-task-lists-in-gfm-issues-pulls-...

r41nbowdash• 9 years ago

looks a lot like "ends ways means" approach

http://www.dtic.mil/dtic/tr/fulltext/u2/a222381.pdf

chillax• 9 years ago

Somehow got me thinking about impact maps again: https://www.impactmapping.org/drawing.html

JAFTEM• 9 years ago

Surprised no one has mentioned doing this in a physical notebook with a pen. I started doing what the author describes a long time ago to document bugs to come back to later. I soon realized it's just a good way to keep my thoughts organized and be more productive. I tried doing something similar with a text editor but grew tired switching screens or tabs. A notebook is also a bit more accessible than a laptop when you leave your desk. During lunch I can read through my notes.

afarrell• 9 years ago

I've done this with a notebook for a long time and it is indeed helpful, especially if I didn't get enough sleep the night before or didn't start my day with a run and I find myself more distractable.

kilon2• 9 years ago

I actually I mentioned this 10 hours ago. So no need to be suprised ;)

rwmj• 9 years ago

I use a physical notebook to record my work too, and I find it very effective. If I sit through a meeting and write nothing down I will inevitably forget everything about the meeting, writing it down helps me to remember it. You can also go back and find out what was discussed before - a surprising number of meetings just retread things which were talked about or even decided upon previously.

I use: Oxford Black n' Red A5 Matt Casebound Hardback Notebook, and UM-153S Signo Impact Gel Pens. I get through one of those notebooks per year on average.

Edit: https://rwmj.wordpress.com/2017/11/24/keeping-a-work-noteboo...

swah• 9 years ago

Most likely you love high quality notebooks and writing instruments, and you probably have neat handwriting.

I rather have files that I can understand later, and easily grep as a bonus...

(I do admit that crossing done tasks on a piece of paper feels very good.)

50CNT2• 9 years ago

I like to think of blank pages as mini whiteboards. With a multifunction copier i can print out lined, grid paper or specialty layouts and whip it through the scanner once im done with it keeping a digital copy. 3USD buys me 500 pages, and with a laser printer [250USD], toner refill kit, etc. that comes out to less than a cent a page, or 3 cents if I amortize the printer over 10k pages (assuming it breaks once it hits its one month maximum duty cycle, which is stupid conservative).

That plus a nice fountain pen [Lamy Safari 18.7USD], ink [J. Herbin Perle Noir 7.2USD/30ml], mechanical pencil [Lamy Safari 12USD], pencil leads [Uni Nano Dia 0.9USD/20leads], technical pen [Rotring Isograph 16.3USD], and I've got a nice little setup.

cholantesh• 9 years ago

I would love for a decent midpoint solution, ie, a way to digitize my notes. A livescribe would be nice, but the issue is that my handwriting is...not too great. I can understand it,certainly, but I'm not sure it would escape OCR unscathed.

deburo• 9 years ago

Unless you have a good tablet, like the most recent Surface Pro or iPad. Writing on them feels very good, almost no noticeable lag. That's what finally replaced physical note-taking for me.

geezerjay3• 9 years ago

I tried doing something similar with a text editor but grew tired switching screens or tabs.

I've found that using a text editor that supports code folding, and following a markup language that enables the text editor to automatically set fold points, helps navigate log entries more efficiently.

I also use a notebook to jot down tasks, but when a job requires a high number of different unrelated small tasks that are completed rather quickly I've found that a physical notebook actually makes the process very inefficient.

tomtompl• 9 years ago

I started doing this "naturally" :)

sent-hil• 9 years ago

This is similar to what I've been doing on and off for nearly 5 years now: https://gist.github.com/sent-hil/3444793, except on paper.

https://news.ycombinator.com/item?id=4448361 has some good tips and tricks.

amelius• 9 years ago

What could help is if you take pictures of the relevant logbook pages with your smartphone, and then commit them into your git repo, along with your code.

wordpressdev• 9 years ago

I keep a simple text file to log features and bugs. This approach of creating date-based files seems cool. May try it out as well.

rmchugh• 9 years ago

Might as well auto generate a header for the file...

  function lb() {
    today=$(date '+%Y-%m-%d')
    fp="/home/you/logbook/${today}.md"
    if [ ! -f $fp ]; then
        echo "# ${today}" > $fp
    fi
    vim $fp
  }

notinventedhear• 9 years ago

I've found keeping a software-engineering logbook very useful at work.

Recommend the vim-wiki plugin for this https://github.com/vimwiki/vimwiki; in any vim-window the logbook for today is accessible via [leader-W-leader-W].

mmrezaie• 9 years ago

or maybe http://orgmode.org/ if you are using emacs. I do not know how awesome orgmode was, and I was missing it. plus you can export it even to latex if you wanna track your research and write your thesis.

tigershark• 9 years ago

Maybe I go against the common thought here, but for me I think it is completely useless. Whenever I start working on a problem I obviously have an approach in my mind, but very often I find out that is the wrong one. And I can realise it only after I start working on the actual problem and solving it. At a certain point I will find some obstacle that will make me rethink the whole approach. For this reason I can't really see the added value in spending more time before actually tackling a problem once you have a plan in your head. The most important thing to do when solving problems is to be prepared to change your point of view multiple times and to attack the problems from several different directions. I'm sorry but I really can't see how writing in stone your approach before you fully understand the problem can help you in any way...

andreareina• 9 years ago

The way I interpreted it was basically an iterated Feynman Algorithm[1]:

1. Write down the problem

2. Think real hard

3. Write down the solution

4. Write down how well it worked

5. GOTO 2

I do something similar using org-mode, and it's useful when, after some time has passed and I need to do the same thing again, I can go back to my notes and not start from scratch again.

[1] http://wiki.c2.com/?FeynmanAlgorithm

pwelch• 9 years ago

I’ve been exploring keeping a logbook for awhile. Some apps I recommend: Quiver, Boostnote, and jrnl.sh

All of them have open data formats so you’re not locked in and can save everything in git which are requirements for me.

nohope• 9 years ago

This is a nice and simple! Usually I keep a logbook separated for each project but it looks more like a diary where I record failed approaches and new tries to solve a problem.

For more structured documentation, I keep separated text files for each solution that I can easily find if I need it later.

EDIT: Also, scratching your plan with pencil and paper before implementing it helps a lot.

jungletime2• 9 years ago

I've been using Typora to make my todo lists recently. It's a markdown editor. Super simple to type, and you can really got some professional looking layout, with headings and titles quickly too. Recently they built in a file browser. Open one file in a directory, and it will show you all the files in side panel. Which in my case are just todo lists for day of the week. But it could be any type of notes. Highly recommend it. For Example typing ###Title would give a large title typing - [ ] will give you a checkbox, and so on.

samuell• 9 years ago

My "bash function" adds just a few more lines, so as to git add/commit/push after closing the editor, and adding a good title by default:

----

journalfile=$(date --utc +%Y%m%d-%u.md);

if [[ ! -f $journalfile ]]; then

echo "# "$(date "+%A, %e %B, %Y") >> $journalfile;

git add $journalfile;

else

echo "File already exists: $journalfile";

fi;

vim $journalfile;

git commit -m "Changes in journal file $journalfile" $journalfile;

git push origin master;

----

Source: https://github.com/samuell/mdnote/blob/master/editnewjournal...

Using a logbook to improve your programming

Related Stories