Quartz: A Deterministic Time Testing Library for Go

https://coder.com/blog/introducing-quartz

by ammario • 2 years ago

133 37 2 years ago

coder.com

wizzard0• 2 years ago

Very cool! Anything similar for .net?

playingalong• 2 years ago

It feels like every second, sorry every other time-related library is called Quartz.

rob74• 2 years ago

Well, in PHP the de facto standard library is called Carbon (as in radiocarbon dating I guess - which is different, but at the risk of sounding, well, fossilized)

Joker_vD• 2 years ago

Yeah, why not something like Κλεψύδρα?

elchief• 2 years ago

not a great name considering Quartz schedular for java...

lloydatkinson2• 2 years ago

Or for .NET

lloydatkinson2• 2 years ago

only on HN would a .NET comment get downvoted

01HNNWZ0MV43FF• 2 years ago

Or Quartz Composer, the rendering thingy for macOS

zikani_03• 2 years ago

In particular, there is also another library for task scheduling named go-quartz: https://github.com/reugn/go-quartz.

Naming is indeed hard.

neild• 2 years ago

Of possible interest to anyone testing concurrent Go code in general and time in particular, a proposed standard library package: https://go.dev/issue/67434

matttproud• 2 years ago

This looks infinitely preferable.

padraicmahoney• 2 years ago

What do you like better about it?

--- (btw: just read your blog on R -- love it)

slantedview• 2 years ago

I'm curious if the author of Quartz has seen this proposal and could provide a comparison?

spikecurtis1• 2 years ago

I'm Quartz's primary author, and I hadn't seen this proposal until today.

They are similar in the sense of using a faked clock to make things deterministic. Other than not having to inject the Clock interface, tests using Quartz will probably look fairly similar to this proposal in many cases.

The proposal has a simpler API than Quartz, being able to just run some goroutines and wait until everything is blocked.

However, this brings some limitations on what can be tested, such as time elapsing while computation is occurring (not just elapsing while things are waiting for something else). IMO this is an important limitation, as many interesting time-of-check, time-of-use bugs and edge cases occur because time does elapse during computation in real systems.

An example of this from coder/coder that motivated the design of Quartz's API: https://github.com/coder/coder/blob/a5e4bf38fec66c5e7ecc96b0...

Groxx4• 2 years ago

That looks amazing. Advancing time while things are idle sounds practically perfect, and that's essentially impossible to build as a library. As much as I like the sound of Quartz (the issues they call out are real and very painful and I was getting close to building my own version to solve some), that issue might remove my need for it entirely.

bheadmaster2• 2 years ago

Thanks for sharing this. This is one of the best testing features I've ever seen - I've had so many troubles with timing tests and refactoring around them. I really hope this gets merged.

oefrha• 2 years ago

Related: the behavior of time in this proposal was implemented in Go playground more than a decade ago: https://go.dev/blog/playground#faking-time. Unfortunately it was never exposed.

jerf• 2 years ago

I still don't like that it assumes global monotonic time. It is completely possible for an event to be scheduled in 100ms on one thread and in 200ms in another, and for the latter to run and/or finish first. A time testing system ought to let you test that case, because you're going to hit it in production.

For that you really have no choice but to abandon the time package's API anyhow because you're going to need some way to trigger those things out-of-order.

I've been using this for a while now: https://github.com/thejerf/abtime (my package) and it really lets me get down and dirty with order of operations, when that matters.

bheadmaster2• 2 years ago

I think the purpose of this package is simple - to allow tests to use time.Sleep without having them actually sleep. A lot of things will not be tested with this model - e.g. you can't really test correctness in all traces of events in your concurrent program - but that goal itself is much harder to achieve.

wolfspaw• 2 years ago

Nice, Golang is Awesome!

myvoiceismypass• 2 years ago

Why is Golang awesome? Because in 2024 there is now a decent mock clock library for testing? I thought that one of go's selling points was the stdlib and tooling being great out of the box?

nsguy• 2 years ago

I have a fair bit of experience writing tests for concurrent code that uses timers on Go. We started with an open source test library ( https://pkg.go.dev/github.com/benbjohnson/clock ). It had a lot of problems. This was many years ago, looks like it's seen some maintenance since so maybe it's better? Then we tried to fix it. Fixed all the obvious bugs but still had a lot of problems using it in practice. It's not enough to just handle the calls without context of who is calling them in concurrent code. Then we switched to using gomock which ended up also being hard to use.

It's quite tricky is sort of the bottom line. It's not enough to just create fake time there's a lot more to it.

jiveturkey• 2 years ago

It's been abandoned/archived. I've since been using github.com/jonboulle/clockwork which is much less problematic. For my use it is perfectly fine.

Groxx4• 2 years ago

Clockwork has been mostly very good for me too. There are some sharp corners that I've run into periodically (e.g. it can't observe tickers, they're just an always-present Waiter) that I've been trying to figure out how to resolve in a new library, but overall I've been happier with clockwork than with any other I've seen so far.

Quartz looks quite interesting though. Gonna have to explore it in detail, because I agree with most of their claims in the article. I'm not 100% sold on their solutions being actual solutions and not just "often an improvement", but that'll probably be clear after using it for a bit / reading it in detail.

alpb• 2 years ago

I think https://pkg.go.dev/k8s.io/utils/clock and https://pkg.go.dev/k8s.io/utils/clock/testing is already doing these?

Groxx4• 2 years ago

At a glance, that looks almost identical to (an older version of) https://github.com/jonboulle/clockwork (there are several that are structurally very similar)

In which case no, it falls into the same kind of issues that the post is describing as being problematic.

rmetzler2• 2 years ago

For linux, there is faketime, which will set a specific date and time for all child processes. This enables you to test software you don't even compile for time-based problems, e.g. logic around Feb 29th or daylight saving time.

zokier• 2 years ago

LD_PRELOAD is bit yucky though in my opinion. I suppose ptrace is the only other option though :/

deivid• 2 years ago

It's not the only other option! It's not less yucky, but you can creatively stomp on the vDSO functions which back the time-related functions (gettimeofday, clock_gettime, time); I've written about it at [0] and have a "library" (more like a proof of concept) at [1].

[0] https://blog.davidv.dev/posts/cursing-a-process-vdso-for-tim...

[1] https://github.com/DavidVentura/tpom

TheDong• 2 years ago

Since go don’t use the libc related time functions on linux, but rather makes syscalls directly, faketime will not work for any programs written in Go.

rmetzler2• 2 years ago

Thank you for clarifying.

jamesrr39• 2 years ago

At the risk of appearing low-tech, a much more simple, Goroutine-safe solution for dealing with "now-dependent" code:

type NowFunc func() time.Time

func getGreeting(nowFunc NowFunc) string {

  now := nowFunc()

  if now.Hour() < 12 {

    return "Good Morning"

  }

  return "Good day"

}

And just pass in `time.Now` in for live code, and your own inline function for simulating a time in tests.

Groxx4• 2 years ago

For just calls to time.Now, sure.

Once you've done that same "make a low-tech simple replacement" for all of sleep, context deadlines, tickers, AfterFuncs, etc all of which are quite commonly used... you've basically done everything these libraries do. At about the same level of complexity.

Quartz: A Deterministic Time Testing Library for Go

Related Stories