Show HN: Pocache, preemptive optimistic caching for Go

https://github.com/naughtygopher/pocache

by bnkamalesh • 2 years ago

97 32 2 years ago

Hey all, I recently published this Go package, and would like to show off as well as get feedback!

so.... if I initially got the key "foo" at time T=00:00:00, this library would re-query the backing system until time T=00:00:60? even if I requery it at T=:01? vs... being a write-through cache? I guess you're expecting other entries in the DB to go around the cache and update behind your back.

if you are on that threshold window, why not a period where the stale period is okay? T0-60 seconds, use the first query (don't retrigger a query) T60-120 seconds, use the first query but trigger a single DB query and use the new result. repeat until the key is stale for 600 seconds.

that is, a minimum of 2 queries (the first preemptive one at 60 seconds, (in the cache for 10 minutes total)

and a maximum of 11 queries (over 10 minutes) (the initial one that entered the key, and if people ask for it once a minute, a preemptive one at the end of those minutes, for 20 minutes total in the cache).

zimpenfish• 2 years ago

if I initially got the key "foo" at time T=00:00:00, this library would re-query the backing system until time T=00:00:60? even if I requery it at T=:01?

From what I understood of the README (10 minute expiry, 1 minute window) only cache requests between 09:00 to 09:59 will trigger a pre-emptive backing fetch.

ie. T0-539 seconds uses the first query (no re-fetch), T540-599 does a pre-emptive re-fetch (as long as no-one else is currently doing that), T600- would do a fetch and start the whole timer again.

bnkamalesh9• 2 years ago

@zimpenfish yes you are right. refetch is initiated on the first Get between 9-10mins, and the timer is reset as soon as the back fetch is successful

tbiehn• 2 years ago

Interesting idea - do you handle ‘dead keys’ as well? Let’s say you optimistically re-fetch a few times, but no client re-requests?

bnkamalesh9• 2 years ago

since the underlying storage is an LRU, I just ignored dead keys. Nope there's no client re-request or retries. That is left upto the "updater" function

serialx• 2 years ago

PSA: You can also use singleflight[1] to solve the problem. This prevents the thundering herd problem. Pocache is an interesting/alternative way to solve thundering herd indeed!

[1] https://pkg.go.dev/golang.org/x/sync/singleflight

bnkamalesh9• 2 years ago

interesting, thanks for that. I'll check it out

kbolino3• 2 years ago

I'm confused by the decision in DoChan to return a channel (instead of accepting one supplied by the caller) and then, given that, also not to close that channel (is something else going to be sent to the channel in the future?). Both seem like strange/unnecessary design decisions.

neild2• 2 years ago

Returning a channel avoids questions of what happens if sending to a caller-supplied channel blocks. DoChan returns a channel with a single-element buffer, so a single send to the channel will always succeed without blocking, even if the caller has lost interest in the result and discarded the channel.

DoChan doesn't close the channel because there isn't any reason to do so.

bnkamalesh9• 2 years ago

thank you for the recommendation, was a good read as well. I could even use it to replace how I'm handling the call suppression/debounce mechanism. Though I think Pocache does 1 extra thing, which is to keep the cache updated before it expires, i.e. for keys which are frequently fetched it'd serve up to date data always from the cache. If we only relied on call suppression, then the concurrent requests would just have to wait during the update stage, or the read-through mechanism would keep hitting the main database.

bww• 2 years ago

You may be interested in Groupcache's method for filling caches, it solves the same problem that I believe this project is aimed at.

Groupcache has a similar goal of limiting the number of fetches required to fill a cache key to one—regardless of the number of concurrent requests for that key—but it doesn't try to speculatively fetch data, it just coordinates fetching so that all the routines attempting to query the same key make one fetch between them and share the same result.

https://github.com/golang/groupcache?tab=readme-ov-file#load...

Thaxll• 2 years ago

It's using singleflight which was later on added to the Go std lib:

https://github.com/golang/groupcache/tree/master/singlefligh...

https://pkg.go.dev/golang.org/x/sync/singleflight

ecnahc515• 2 years ago

Just a note that `x/sync` is not part of the Go std lib.

iudqnolq• 2 years ago

Is groupcache suitable for current use? I don't see commits in years and the issues have reports of panics due to bugs.

stock_toaster• 2 years ago

Indeed. It also looks like there is a maintained fork[1], but no clue with regards to the quality.

[1] https://github.com/golang/groupcache/issues/158#issuecomment...

derekperkins• 2 years ago

Another maintained fork https://github.com/mailgun/groupcache

bnkamalesh9• 2 years ago

hey thank you for sharing this. Based on what I understand, this package focuses on the underlying storage mechanism itself rather than helping with the cache strategy. It seems like a solid storage extension which can be used Pocache!

sakshamhhf• 2 years ago

Not work

bnkamalesh9• 2 years ago

indulona2• 2 years ago

I have implemented my own SIEVE cache, with TTL support. It solves all these issues and requires no background workers. Author, or anyone else interested in this, should read the SIEVE paper/website and implement their own.

bnkamalesh9• 2 years ago

interesting, will check out

latchkey• 2 years ago

https://github.com/opencoff/go-sieve

https://github.com/scalalang2/golang-fifo

bnkamalesh9• 2 years ago

that was an interesting read (https://junchengyang.com/publication/nsdi24-SIEVE.pdf), thanks for the recommendation. It looks like a good fit for replacing the underlying storage mechanism of Pocache, instead of the LRU. Though I do not think it addresses the thundering herd problem, where the underlying database would be flooded with calls when the cache expires. I think Pocache is focusing more on the caching strategy itself rather than the cache eviction or storage mechanisms. Hence the store is configurable for Pocache.

indulona2• 2 years ago

SIEVE is merely eviction strategy.if you need to put cache above the database, that has little to do with the cache itself. thundering herd is again different thing altogether and can be easily mitigated by simple queue/custom logic. usually none of these things belong into cache but next to it. meaning, you "get" value and if there is no entry, you proceed to fetch it from wherever you need and set it. to avoid concurrent fetching, you synchronize your code with mutex or queue or whatever. all these things belong into your code, not the cache.

Show HN: Pocache, preemptive optimistic caching for Go

Related Stories