Hledger: Robust Plain Text Accounting

by lwhsiao • 7 years ago

236 94 7 years ago

sctb• 7 years ago

Discussion from 2017: https://news.ycombinator.com/item?id=15818682.

Nerada• 7 years ago

Does Hledger support encrypting the ledger file? I don't mean placing the ledger in an encrypted container and mounting/unmounting as necessary, but I guess 'natively' as part of the file (think an encrypted Excel document).

I'm a GnuCash user, but my biggest gripe is not being able to encrypt my accounting file. I'd love to have it sync across my devices via x cloud platform, but I'm always paranoid about uploading anything that isn't password protected. If I was able to do that it'd be the push that gets me to switch from the GnuCash GUI to a plaintext account.

Maybe there's some other way I haven't thought of without a high barrier to entry like constantly mounting and unmounting an encrypted container to make changes.

YaxelPerez• 7 years ago

Just having the ledger in cryptomator works fine for me. Not sure how secure it is.

Jtsummers• 7 years ago

Hledger just works on text files. You’d need some other system to keep them encrypted. I use KBFS which keeps things synced between my devices well enough.

icebraining• 7 years ago

You can use a syncing application that encrypts beforehand. The one I use is git-annex, which supports many cloud platforms (as well as plain SSH/FTP/SMB/etc), but there are others.

AdamGibbins2• 7 years ago

beancount does, you can transparently encrypt your data.

dhruvkar• 7 years ago

is there any documentation for this?

atmosx• 7 years ago

I use gitcrypt.

AdamGibbins2• 7 years ago

Personally I prefer beancount, incredibly pluggable in python (imo a friendlier language to haskell), additionally much stricter in multiple aspects. Been maintaining my ledger with ledger-cli and then beacount for 10 years now.

dragonsh4• 7 years ago

I am a user of both ledger-cli with emacs lisp bindings and beancount, fava with bank PDF statement preprocessing for journal entries. Used statistical inference to generate journal entries automatcomatically with tools like reckon (now use another one, as it's not maintained) and self written algorithms in Python with sci-kit learn. Works very well so far for over 6 years.

nsomaru• 7 years ago

I’d love to see your scripts if you could share them? Especially the part to automatically generate entries

HorizonXP• 7 years ago

I just spent the last week reconciling my books, using my scores of Excel files as a source, moving them to ledger.

I learned rudimentary double-entry accounting in high school, and I love that ledger enforces it so that I can be strict about my tracking. I still have to figure out how to produce balance sheets and income statements so that I can confirm that my accounts are balanced correctly. Honestly wish I had done this sooner.

Edit: Looking at beancount again, I should just use that instead of ledger. Ledger's simple, elegant, and nice, but I'm a Python guy, and I should just use what I know.

edwintorok• 7 years ago

I've recently come across two systems that use hledger for processing bank statements: https://pauley.org.za/hledger-flow/ https://github.com/adept/full-fledged-hledger/wiki

They treat the statements as the single source of truth, with preprocessing and rules that can be used to turn them into hledger journals.

apauley• 7 years ago

I'd be happy to hear if you've found hledger-flow useful. Feel free to create an issue on https://github.com/apauley/hledger-flow if you think anything can be improved.

techntoke3• 7 years ago

hledger looks pretty amazing, but there are so many dependencies and updates with Haskell programs. Almost every other day I'm updating like 20 Haskell packages for hledger or pandoc when I run a system update. It isn't inherently bad, but never had this issue with a C or Go program.

smichael12• 7 years ago

May I ask what system is that ?

Maybe one of the other packaging methods will give you a more stable installation ? https://hledger.org/download

jeremyjh• 7 years ago

I'd bet money its Arch. I love `pacman` but the package spam for Haskell is a bit much as its automatically updating everytime a new package is hosted on hackage. I'd recommend skipping it all together, just use stack and build it from source, let stack manage GHC and you'll never update except when you want to.

baroffoos2• 7 years ago

Why does this matter? Are you on a limited network where this is causing issues?

techntoke3• 7 years ago

No one wants to upgrade 20+ libraries every couple of days just for a couple Haskell-specific packages. I am going to try some of the other solutions mentioned for now.

gmfawcett• 7 years ago

Well, there's always ledger, the C++ program that HLedger was originally based on:

https://github.com/ledger/ledger

I used Ledger once upon a time, quite successfully! Worth a look.

mark_l_watson• 7 years ago

I am just curious: why do you update so often? I will stick with a stack LTS version for many months, then update all my projects at once when I decide to update.

Off topic, but as a student of Haskell for many years, I found reading (this morning) some of the code in the ‘tools’ directory of the github repo to be useful for picking up a few useful idioms.

simonmic6• 7 years ago

I'm hledger's lead developer, thanks for the mention. If you have any questions or hit any snags trying it out, I'm ready to help. Critique of product, docs etc. is welcome. Last time (2017) we got 2 out of 41 comments about hledger, so I'm hoping for a few more. :) https://hledger.org/release-notes has the (software) changes since then. Our chat is #hledger on Freenode: http://webchat.freenode.net/?channels=hledger

wtvanhest• 7 years ago

Just curious, do you guys have any power users? Like large companies? Also, what does the distant future look like for hledger?

jsmith99• 7 years ago

Large companies don't usually post everything manually. They journal adjustments, but the day to day entries will be made automatically by their ERP from systems like Purchase to Pay.

smichael12• 7 years ago

wtvanhest, not that I know of. It's hard to collect good usage data without nagging people. I hear from small business owners, the occasional accountant, and I have the impression there are other business/institutional users that don't interact.

Distant future is hard to predict but here I go: hledger or any good parts of its design are still around and still providing value. Any past data kept with hledger is still readable and easy to convert, with or without a working hledger. Highly polished and efficient UIs are available everywhere they're useful. Data import and multimachine syncing are routine and just work. Data hygiene/security and personal privacy are routine and just work. Performance is high, "big data" is no problem. Resource usage is low, these tools continue to work in the turbulent and resource-constrained future. There is a strong global community of financially literate, capable, empowered, prospering individuals and organisations using efficient high quality universally available accounting tools to promote, enforce and create widespread transparency and societal insight in finance, business, politics and every ecological issue.

spindle• 7 years ago

Thank you for your work. (Does that count as a question?)

smichael12• 7 years ago

I appreciate it!

doomrobo• 7 years ago

Are there any iOS apps that interop with hledger? I've been using iQIF [0] for a bit, but it only exports things in QIF format.

[0] http://www.iqif.info/iQIF/

dragonsh4• 7 years ago

Use this https://github.com/giacomos/qifparse combined with https://github.com/quentinsf/icsv2ledger to automatically generate journal entried in ledger-cli format which can be used by hledger directly.

You can also fork and modify it to generate hledger format files directly.

smichael12• 7 years ago

Basically any that export CSV. There are several that are specifically hledger-aware, but they might be all on Android so far, I forget. See https://github.com/simonmichael/hledger/wiki/Mobile-apps for more on this topic.

cicko• 7 years ago

Would you mind trying out https://cashier.alensiljak.ml/ and provide feedback about how it works on Apple devices? It is meant as a less-maintenance alternative to native apps and works offline if "added to desktop", which is a browser feature for Progressive-Web Applications. The source is here - https://gitlab.com/alensiljak/cashier - if you wish to provide some feedback or have a look at the code. Currently it can record transactions and export in ledger format. Perhaps more functionality is added if a need arises.

OskarS• 7 years ago

I've been using ledger (the original one) for the last couple of months, and I'm loving it. I'm way into the whole plain text accounting thing.

I guess it's the most obvious question in the world, but: what are the big reasons I should switch to hledger? What do you see as the primary advantages of hledger over ledger?

PhilippGille• 7 years ago

This answer in their FAQ contains some differences: https://github.com/simonmichael/hledger/wiki/FAQ#features

simonmic6• 7 years ago

As hledger's proud parent, having preferred it for my needs since the early releases, I can talk about this for a long time. :) The https://hledger.org/FAQ addresses it, as PhilippGille mentioned. I still find it hard to answer concisely. Let me give it another shot:

Here are some things I've tried to do differently with hledger compared to Ledger. I feel we've had some success with all of these:

Project:

- be actively maintained and contributed to

- build out Ledger's concepts a bit more selectively/thoroughly/to a higher level of polish, taking advantage of hindsight, a fresh codebase, a well-suited language

Docs & design:

- all behaviour fully specified and documented

- all docs up to date and accurate

- practice documentation-first design and development

- more intuitive, learnable, consistent concepts and UX

- be usable for less-technical users as well as techies

- be easy to get started with as well as efficient for regular use

- explicitly try to model accounting concepts and help meet real world accounting needs

Software:

- be robust and dependable, with few user-visible bugs

- be easy to install and run well on all major platforms

- provide APIs/libs enabling more plain text accounting hacking among Haskellers

- be readable, clear, and a useful source of algorithm & architecture ideas for other implementors

- remain maintainable and cost effective over the long term

- achieve a long lifespan and provide a safe long term format for accounting data

Technology:

- clarify whether Haskell is good for real-world apps (yes!)

- learn more Haskell by building a real-world app (yes!)

- attract contributors by being a Haskell project (yes)

Out of time for now, I hope it's not too hand-wavy..

drieddust2• 7 years ago

Thanks for this. A few questions comes to mind:

1. What was the motivation to create this instead of extending ledger?

2. How to track time?

smichael12• 7 years ago

You're welcome. Re 1, see https://news.ycombinator.com/item?id=20020196 above. Re 2:, check out the timeclock and timedot manuals on hledger.org front page (I use the latter) and my https://hledger.org/Time-planning.html hacks in the cookbook and more https://plaintextaccounting.org/#time-logging tools and https://plaintextaccounting.org/#time-tracking tips.

simonmic6• 7 years ago

PS Thanks meruru for mentioning http://plaintextaccounting.org. I started it to knit and grow our community, give more visibility to the non-Ledger projects like hledger, beancount and the rest, help answer VFAQs on IRC and mail lists, collect best practices, etc.

That site is also worth checking out for more hledger materials, such as comparisons, talks, slideshows, and blog posts. Eg Andreas Pauley's quick intro slides (https://pauley.org.za/functional-finance-hledger) or my slow "Hands-on with hedger" talk and FLOSS Weekly appearance.

z3t4• 7 years ago

A ledger should be a database with only select and insert privileges. eg. No update or delete queries should be allowed.

p1necone• 7 years ago

If you're writing a ledger for your own personal record keeping, and not for some third party to be able to audit you wouldn't it be helpful to be able to keep history tidy and clean up mistakes without having to add extra entries?

repiret• 7 years ago

Yes in the sense that data shouldn't be destroyed, but you still need to be able to correct mistakes - those corrections just need to leave an audit trail.

dmos623• 7 years ago

That's a solved problem: git.

tty2300• 7 years ago

Git allows rewriting history though.

ryanobjc2• 7 years ago

Hledger is pretty good for basic accounting. I recently had to do some tracking of stocks and lots and forex and the developers decided not to implement lot-based tracking. Which made it impossible to accurately track money.

So back to plain ledger. If I was starting over I'd probably consider beancount. But there's only so much I can do a day.

smichael12• 7 years ago

I guess you haven’t seen https://github.com/simonmichael/hledger/issues/1015 ?

ryanobjc2• 7 years ago

I have but I couldn't make it work. The issue seemed to boil down to "when do you price commodities?". Hledger doesn't let you report the price at the actual sale. The history flag didn't work I forget the reasons at this moment but if I recall correctly it boiled down to "price at market" type of features.

That works fine for asset type accounts. But when we are taking about as-was accounting it doesn't work well enough. Beancount has some interesting ideas about lot tracking but switching cost is too high - I have years and 35k lines.

meruru• 7 years ago

See also https://plaintextaccounting.org for more on the broader ledger ecosystem.

wodenokoto• 7 years ago

Which is also a really good introduction to accounting. I based my spreadsheet accounting on this guide.

smichael12• 7 years ago

wodenokoto, may I ask which guide ?

enriquto5• 7 years ago

I would appreciate a small section on the README file that explains how to compile the program, what are the requirements (e.g., what debian packages to install), and a minimal cli use case.

Also, running "make" prints a daunting 96 lines of text of which the last 50 (the ones that you get to see) are mostly useless. If I run "make build" it starts to download stuff from the internet, which I find extremely impolite, as I already downloaded the source code. What is the point of not including all the neeedded source code in the source distribution?

TomK32• 7 years ago

Haskell is a bit rare and so are its libraries rarely installed on systems. I can't remember, as arch has a proper package and I built hledger only once on a different system, but those downloads might be just libraries.

enriquto5• 7 years ago

Whatever. Either they are dependencies that are required to be installed beforehand (just like a haskel compiler is required), or they are included in the source code distribution. Downloading them at compile time is very strange, and in my eyes unacceptable.

smichael12• 7 years ago

Thanks for the feedback, I'll take a look at the README. Does http://hledger.org/download, prominently linked on the front page, have the right info ? My intention was for you to find that, so I'm curious how you travelled from this HN post to the README.

PS: "make" prints its help, a summary of all targets. I agree it's more than a screenful and will need to be viewed with a pager (I'm used to viewing things in an emacs shell). The Makefile is assumed to be used mainly by contributors and does a lot. What would you recommend "make" to do ?

accrual• 7 years ago

I've used the original ledger [0] for a few years to track my day-to-day cash flow and appreciate the simplicity. I create a new file every year and manage it with standard Unix tools, plus a few shell and Python wrappers to make entries and reports a little easier.

The hardest part is just getting started and getting into the habit of maintaining the ledger. Once you start tracking it's difficult to stop! Seeing all of your accounts line up is a reward in itself.

[0] https://www.ledger-cli.org

tangue• 7 years ago

If you're using Emacs and are not familiar with double-entry accounting this video is a good introduction : https://www.youtube.com/watch?v=cjoCNRpLanY (it's based on ledger not hledger)

stevesimmons• 7 years ago

These ledger formats have what feels to me an overly verbose multi-line structure for each transaction.

For 20 years, I have been using a format that combines a diary and ledger. The 'to' and 'from' accounts are either bank/asset accounts (sav, cc, hl for home loan, ...) or spend categories (snack, shopping, clothes, ... ). Each year I finish 31 December with a `# Closing balances` section for each bank/investment/pension account, then start a new text file `2019.txt` with an `# Opening balances` section before 1 January's entry.

Here is a typical day's format:

  # 2019-05-27 Mon
  2.50    sav>lunch    Cafe Nero - Coffee
  100.00  sav>cc       Transfer - Credit card repayment
  amt     from>to      forex: shop - desc [txn #] 
  # Diary
  * <topic> - <what>
  * Times - Sleep 01:00-06:00 5h. Run 45m. Work 08:00-18:00 10h.
  * Work - Meeting with xxx on xxx. We agreed to xxx.
  * Friends - Lunch with xxx
  * Note - Running shoes are Asics GT-2000, size 48, model T500N.
  # Links
  * Hledger: Robust plain text accounting - https://news.ycombinator.com/item?id=20012499

For the first 5 years, I was single and reconciled everything down to the cent, using an AWK script to generate totals to match against my bank statements.

When I got married, all our accounts switched to joint ones. I didn't want to track my my partner's side of our spending. So these logs became more a personal diary than a fully reconciled ledger. Anything I want to remember long-term goes either there or secured in a password safe.

kd5bjo• 7 years ago

You reinvented the accounting daybook: Shops doing paper accounting never went through the hassle of double-entry accounts for each sale. Instead, each was a single line in the day’s journal, and closing for the day involved reconciling the till with the sales journal, and entering the entire day’s revenue as a single transaction in the master account book.

fiatjaf2• 7 years ago

I agree. They are too verbose. One line is enough.

I actually think personal logs are useful for all things, not only money. In my business we use a text-file with logs for all the events, money, business stuff that happens, and then there's a program that reads all the lines, parses them and turns them into a final "state" of the company with everything that is being done and so on.

From that we generate dashboards, reports and stuff.

This is the system we use: https://github.com/fiatjaf/journalstate (but really the power is in the concept, not the implementation).

I've also turned that into a TodoMVC app: https://journalstate-todomvc.alhur.es/, so you can see it in action in a fancy, useless way.

simonmic6• 7 years ago

hledger has builtin support for four formats so far (journal, timeclock, timedot, csv). The current journal format can be quite light:

  1/1
   (food)  5
  1/2
   (car)  15

But I'll sometimes use csv for quick one-line entries:

  1/1,food,5
  1/2,car,15

Some hledger users have their own custom formats like you, and use a script to preprocess it. More format experiments are always welcome.

An advantage of something like hledger over a home-grown script is more robust and versatile reporting.

mongol• 7 years ago

I looked into plain text accounting but Swedish bookkeeping/accounting practises forbid practises where past records can be modified. It needs to be append-only. I started developing a command line accounting tool using SQLite and Go but it is only half-finished.

estsauver• 7 years ago

Git + Open Timestamps maybe?

magnamerc• 7 years ago

Or maybe gasp a blockchain.

Someone• 7 years ago

Binary files can be rewritten, too.

To check that the first N transactions in a file weren’t tampered with, you have to either compare each transaction with a backup, or hash them and compare the result with the hash you got earlier.

If, as I suspect, this ledger is append-only, both are fairly simple. If the ledger, say, had 42 lines earlier, do either

    head -42 ledger | diff ledger.copy

(I’m not sure that’s valid shell, but you get the idea) or

    head -42 ledger | my-super-hash

and compare the hash with one you got earlier.

Also, some OS-es (e.g. Mac OS. See https://developer.apple.com/library/archive/documentation/Sy...) have the notion of append-only files. These, too do not safeguard against somebody rewriting the entire ledger, but they can make it harder to do that tampering.

Finally, I haven’t thought it through, but possibly, you can create a user on your system for your accountant, set the owner of the ledger to that user, and make it impossible for anybody but that user to delete or replace the file.

If that’s possible, it even would protect against reinstalling the system. You could recreate your accountant’s user ID, but not with the same password, so your accountant would still notice.

noufalibrahim• 7 years ago

I used to be a heavy ledger-cli user and wrote about my setup here http://nibrahim.net.in/2015/11/07/ledger_and_personal_financ...

malicz• 7 years ago

I am a freelance programmer and I use hledger in combination with beancount for a couple of years. why did you change and to what?

fiatjaf2• 7 years ago

See also http://hledger.alhur.es/, hledger compiled for the web in a playground (you can also save your stuff to remoteStorage).

Source: https://github.com/fiatjaf/hledger-web

billfruit• 7 years ago

Does it do 'envelope' style budgets like YNAB?

simonmic6• 7 years ago

You can do those with it, in a manual way (I did for a year or two). More info on this can be found at https://plaintextaccounting.org/#budgeting.

There's also non-envelope style budget reports built in. For more on these see https://hledger.org/manual.html#budget-report and https://hledger.org/Budgeting-and-forecasting.html.

kazinator• 7 years ago

This is exactly how I do my accounting: record transactions in a plain text file, which is processed by the system, resulting in various ledger and account type objects being in a certain state, which can then be subject to queries and reports.

Mine is written in TXR Lisp. It is not released to the public.

All input is recorded via function call expressions, which are of course S-exps. There is only one single macro in the whole thing: def-date-var defines a variable that exhibits different values depending on a effective date context (established by a dynamically scoped date variable). With this we can do things like different tax rates for different periods, without having to introduce such things as parameters appearing in every input item.

The amounts within a transaction must always sum up to zero. As a convenience, one amount may be left blank; it will be inferred so as to balance the transaction.

I also have transactions that sum to zero. There can be three or more account deltas in a transaction, not only two.

This is not how accounting is taught in North America, though; I came up with it myself.

Inferring one amount is a bad idea. My system throws an exception if the transaction amounts do not add to zero.

However, the entry of items is done via various convenience functions, most of which generate the necessary transactions against all the right accounts. For instance, recording a new asset:

  (record-asset 1 "Discombobulator" :cca-50 ;; ID, description, capital-cost-allowance class
                (date 2017 1 23) ($ 79.99) ($ 4.00) ($ 5.60)) ;; base price, GST, PST taxes.

From the looks of the journal format of hledger, it's too generic. I wouldn't want to use anything of the sort.

Different items need their own syntax. I don't want to record a simple purchase of a recurring expense in the same way that I record an invoice.

I bind a new variable when recording an invoice:

  (defparml %inv-0042%
    (new invoice
         number 42
         provider %my-addr%
         client %foo-client-addr%
         remit-to "Kazinator"
         remit-days 25
         date (date 2017 6 30)
         items (list (new time-unit
                          start (date 2017 6 1)
                          end (date 2017 6 2)
                          hours 16
                          rate %rate-foo-client-2017%)
                     (new time-unit
                          start (date 2017 6 5)
                          end (date 2017 6 9)
                          hours 40
                          rate %rate-foo-client-2017%)
                     (new time-unit
                          start (date 2017 6 12)
                          end (date 2017 6 16)
                          hours 40
                          rate %rate-foo-client-2017%)
                     (new time-unit
                          start (date 2017 6 19)
                          end (date 2017 6 23)
                          hours 40
                          rate %rate-foo-client-2017%)
                     (new time-unit
                          start (date 2017 6 26)
                          end (date 2017 6 30)
                          hours 40
                          rate %rate-foo-client-2017%))))

There is no custom DSL here at all; just the new macro of the object system.

This is then recorded using a plain function call to the convenience function record-invoice:

  (record-invoice %inv-0042% "Invoice 0042 issued to Foo, Inc.")

This function will digest the invoice and post the right amounts into various accounts. The GST tax has to be charged, and such,

When the invoice is paid:

  (record-paid-invoice %inv-0042% "Invoice 0042 paid by Foo, Inc."
                       (date 2017 7 31))

this function will also update multiple accounts, including the automatic withholding of income tax, based on the withholding rate configured for the date into which the invoice lands.

To generate a nice HTML invoice, at the REPL, I just call the .(html ...) method of the invoice object, which takes a stream argument.

  1> (with-stream (s (open-file "invoice.html" "w"))
       %inv-0042%.(html s))

Hledger: Robust Plain Text Accounting

Related Stories