j_kao

FYI the mooncake team was acquired by Databricks so it's basically vendors trying to compete on features now :)

1 - I think if we were sticking with the JVM, I do wonder if Lucene would be the right choice in that case

2 - It's a great tool with a lot of tuneability and support!

3 - We've been using it for K8s logs and OTEL (with Jaeger). Seems good so far, though I do wonder how the future of this will play out with the $DDOG acquisition.

These are great projects, we use DuckDB to inspect our data lake and for quick munging.

We will have some more blog posts in the future describing different parts of the system in more detail. We were worried too much density in a single post would make it hard to read.

It's a bit difficult at the moment, given we have a lot of proprietary data at the moment and a lot of the logic follows it. I'm hoping we can get it to a state where it can be indexed and serving OSM data but that is going to take some time.

That being said, we are currently working on getting our Google S2 Rust bindings open-sourced. This is a geo-hashing library that makes it very easy to write a reverse geocoder, even from a point-in-polygon or polygon-intersection perspective.

Author here! We were really motivated to turn a "distributed system" problem into a "monolithic system" from an operations perspective and felt this was achievable with current hardware, which is why we went with in-process, embedded storage systems like RocksDB and Tantivy.

Memory-mapping lets us get pretty far, even with global coverage. We are always able to add more RAM, especially since we're running in the cloud.

Backfills and data updates are also trivial and can be performed in an "immutable" way without having to reason about what's currently in ES/Mongo, we just re-index everything with the same binary in a separate node and ship the final assets to S3.

Companies like Netflix with bigger market caps are still on AWS.

I can imagine the productivity of spinning up elastic cloud resources vs fixed data center resourcing being more important, especially considering how frequently a company like Figma ships new features.

Those are people who would probably steal software, anyways, so for most businesses I think worrying about those folks would be a waste of time

A bit of an aside, I'm a big fan of how Mike Perham managed to start a business around Sidekiq. I think it really gives us Hacker News folks that there's hope starting a viable bootstrapped business around open-source infrastructure.

Perhaps if the fee was more expensive for a single person, but economical for >2 people then the math works out.

To be fair, there are a number of job openings where you can "Apply with LinkedIn", so it's structured data end-to-end.

One area which an external monitor (or high resolution) helps a lot with is UI or frontend programming. It's a huge pain to play with the chrome or Firefox debugger and have to jump between windows to get feedback.

When it comes to backend or non-UI work, a laptop is just fine.

Testing in production will trend upwards among companies because everybody's workload is shifting towards the cloud and/or making use of external SaaS services. There are a number of cloud services that are not open source that can't be run locally, or run at the same scale as production.

It is not a good use of time to mock everything, because you have no control of external systems. The only reason I'd see it being important is if these external systems are tightly coupled to complex local logic that should be tested locally. However, there are a number of strategies to deal with such "tight coupling" in such cases.

I wonder if the ageing population combined with weakening currency are big factors here.

This is an even more effective strategy given the Japanese culture of constantly rebuilding housing [1]. It does beg the question of how sustainable that practice itself weighed against the benefits of solar installation.

[1] https://www.theguardian.com/cities/2017/nov/16/japan-reusabl...

This is a great release! Now we have a way to share code between our CI and Terraform (we use CDKTF).

Imagine this scenario:

1. You onboard new developers and their on-boarding task is to add their name and public key to a DEVELOPERS.ts constants file

2. git submodule update your CDKTF repo and spin up a new dev box / playground for them to use

3. git submodule update your CI scripts and now your Dagger repo/definitions can deploy to said dev boxes

All with one source of truth (DEVELOPERS.ts), type-safety, and version-control!

Not to mention, if you wrote your server code in TypeScript, there's a lot of options here.

The great part is you can do this in any of the matrices of languages that are shared between these tools.

I hope my comment didn't come off as an attack and certainly admire what you've accomplished!

So long as the medium of techno allows you to embed your own creative characteristics, then I think that's a great form of expression (as opposed to churning out formulaic techno... though there's really nothing wrong with that, either).

In fact, setting constraints on yourself is a great way to inspire creativity and nuance (e.g. there's a lot you can do with a single note - rhythmically, harmonically, effects, dynamics, etc).

I also agree that after a certain point, genres become too simplistic of a model to describe music. I'm happy that you've put so much thought into this.

For fans of Dilla's music (you may not know the producer, but I'm sure you'll know some of his songs https://en.wikipedia.org/wiki/J_Dilla_discography), I'd highly recommend the book Dilla Time.

The author goes into a lot of detail about what swing is, its origins in African music (polyrhythms) and the racial tensions that came from these rhythms, and how it eventually made its way into modern music.

I also wonder if writing music was chore-like due to the genre the author chose, specifically four-on-the-floor techno music that has largely fixed and repetitive structures?

This music is meant to be mixed by DJs, after all, so consistency is really key to a lot of the techno genre. Of course, there are very creative divergences in this genre from artists such as Four Tet and Floating Points.

Would the author still have the same level of output writing songs with lyrics, where you're explaining a life-story or concept?

I wonder if the 10% is purposely vague so that they can use a random 10 percent of content to A/B test platform engagement (maybe plan upgrades) and ad engagement.

That's true, there are definitely ways to make this work if we're confining these to very restricted spaces with active monitoring.

I look forward to a future LA where car ownership is optional. There's a lot of aggressive urban planning that needs to happen and implemented for it to work. Downtown Culver City is a good example.

To be frank, especially for anyone who hasn't lived in LA, the LAPD doesn't do anything for the public unless it's something involving guns and/or gangs.

It's not unlike how developers mark most bugs as "won't fix".

I've seen active break-ins and robberies in LA, and called the police only to have nothing being done in those situations. While anecdotal, I'm afraid this is a really common experience that has only gotten worse with COVID-flation.

This will be interesting to see if they can crack LA - drunk and overly aggressive drivers, crack addicts running around the streets, and vandalism.

You can already see this "tragedy of the commons" type of behavior with the unusable public transport system (it's a nightmare to use the trains between Koreatown and DTLA, especially for women) and in the private sector with Lime and Bird scooters.

Very exciting if it works out, but I don't have high hopes.

Thanks for the reply, I suspected that was the case. Was your payroll done in the US or Canada?

Hi proberts! Thanks for doing this.

I'm a Canadian on a TN visa at a startup in the US. Would it be possible to continue to work for this company on this status but stay in Canada?

Is there a minimum amount of time I need to spend in the US or can I just fly in for occasional onsite meetings?

Yeah, I could see that even outside of the context of the Cassandra virtual nodes primitives.

These could be K8s nodes for example, that don't make full utilization of the underlying VM, which would completely make sense at APPL scale.

For some background to other HN users out there, "virtual" nodes refer to logical nodes in distributed database software where # of virtual nodes >= physical nodes. This means if the size of data passes a certain threshold, physical nodes can redistribute the virtual nodes, reducing the amount of data shuffled across physical nodes (as opposed to a naive hash function that mods a key by a fixed number and requires all nodes to reshuffle data when a new node is added).

I'm wondering if this refers to "virtual" nodes which are not coupled with physical nodes.

Great job shipping!

When it comes to open-source infrastructure projects like this, the first thing I generally look for is an architecture diagram on how to operate something like this.

Having a great developer-friendly API is important, but operating the system(s) is just as important and often a differentiation point of using the system (e.g. scale up some nodes, and you're good vs a mish-mash of an OLTP database, distributed WAL, a denormalized search cluster, etc).

I don't get a good sense of how this works right now with the docs.

How does one go about removing a phone number off of these sales data aggregators?

I don't think I've ever explicitly given these phone numbers to tools like this (e.g. signing up to Datadog with a phone number), so this seems like sensitive PII that must have been leaked and scraped in some shady source that these sales "data-enrichment" tools happily take.

To be honest, I don't think using programming language (e.g. learn Go) books is a great way to approach learning as it's a bit too passive and often ends up as coffee table material. I'd argue trying to build a project and using the latest documentation helps things stick better. Obviously better if you have to do it for work.

On the other hand, programming and architecture concepts (e.g. designing data intensive applications) make a lot more sense as books, since they are high level concepts. In that case, use whatever medium reduces distraction. That's often in physical book form.

We bought it with watery eyes, but let's just say not over-hiring helps a lot with the burn rate ;)