ivansmf

The technology is amazing. The marketing around it is a decade ahead of its capability, and the pushiness to make LLMs do what they still can't is just irritating. The question to me is "who is seeing the goal posts?" And the answer is "the marketing department of whoever sells it".

The article severely underestimates deployment times for large, world wide services. Usually the strategy is to have a smaller "blast radius" for deployments and going in stages that are also usually time bound ("let it bake"). It also does not account for outages and fixing things you only find in deployment. Programming languages like Python it using injection in Java (e.g. using Guice) either need pristine testing, and all test teams were converted to dev 20 years ago, or have a magical way to destroy all the help compilers and static analysis can give you. So yeah, you take the 4 weeks of development from your 6 month deployment, then add 6 weeks of debugging and retries by using AI. You're welcome that will be 3 million tokens, of which you wrote 1k, the rest was system prompts and "reasoning", which you do not control. This whole AI space is highly fixable, but requires investment no one seems to be willing to do, particularly in areas that were mistakes from the past.

I suspect the entire industry uses "auto-raters", where an agent instance is used to scores the agent's output. The idea is similar in intent as using adversarial networks to train image generation, minus the human labelers. Raising the scores of the auto-rater then becomes the metric teams optimize, and it is no wonder the end result is that the agent scores its own generated content the highest.

The existence of a kill switch plus tracking in legislation very likely means the manufacturers wanted to track and sell user data and needed a scapegoat to avoid customer backlash. I would profoundly surprised if we don't find a lobbyist at the bottom of this.

I have not read the books, so I guess I'm the target audience. It was very hard to watch. The actors were fine, actually better than fine, but the writing was painful. There are lots of standard adventure and fantasy arcs that are just impossible to carry forward with the type of "modern" they wanted. For instance, you cannot have the most diverse village in the history of villages anywhere, then later the Orc (?) stares at one of the kids once and goes "you are not from this village, you are certainly from this other village because of how you look". How? Aura color? At least change it so the kid has a tattoo or some birth mark then. I could go down a long list of dumb stuff like this that makes me come back to reality instead of allowing me to stay with the flow and enjoy the fantasy.

Calling failing to show up or nominate a legal representative to a judicial hearing in a country that offers a ridiculous number of opportunities to appeal as "feud" shows some bias in reporting.

You. don't. say. Really?!? OMG, what an insight! What are they going to say next? That startups also use the good will of the internet to create their IP to then take it private as a way to get free labor? I'm shocked! And because this is also the internet, that was sarcasm.

Allegedly Costco has a policy of making their white collar folks work on the floor of their stores and rotate positions before getting to the office, same with McDonalds. I would love to hear from someone with first hand experience to confirm or correct my impression. I did confirm that Costco gives good benefits and decent pay for all their automotive section employees, and since then I only buy tires there (since 2000) for that very reason. Every time I visit I reconfirm that is the case. It is not much in the great scheme of things, but it is an honest effort to buy from good employers.

Yes! I still can work on my open floor plan. Let's open the windows and connect to the awesome traffic jam outside.

I wrote last year's benchmark. The clusters are completely different, and so is the workload. Last year's cluster had 300 VMs, which was a much higher price point, and the workload was write only. This benchmark uses YCSB workloads A and B, which we though matches the usage we'll have on BigTable. The cluster is much smaller as well. I shared my scripts from last year, it is pretty easy (although a bit expensive) to repro the numbers. Let me check if we can share this year's benchmark scripts as well.

Hi, The cost we published includes the time to setup the whole cluster, warm up the data nodes, and run for 5 minutes at 1M per second.

Our run rate is $281 per hour, which is the same as AWS a couple of years back. What changed is that we are using quorum commit, the data is encrypted at rest, we have very low tail latency, and we look at all samples when computing that.

Computing our price is easier because we do not charge per access.

Here is the formula for our run rate:

30 loaders (n1-highcpu-8) at $0.522 per hour: 248.7

300 nodes (n1-standard-8) at $0.829 per hour: 15.66

300 1TB PDs that run at 0.055555556/hour: 16.67 Total: 281.03

But keep an eye on us. This is for today prices.

Hi - I figured you know what you're doing based on the details you picked on. I suspect you'd have way more fun reading the raw logs, but that would make for a post that is incredibly hard to parse. I will try to figure out a venue for them.

It was 100M keys per loader, so 30x100M for the low latency cluster and 45x100M for the high latency cluster. I sized to keep running long enough to go over half dozen major compactions, not necessarily to eat storage space.

Most of the IO is sequential, which really makes no difference for our backend (sorry, I have to leave it at that). Read performance ends up affecting the performance when the system reaches its limits of pending compactions, which I limited in the config. I could probably get a higher number without it, but it looked more realistic to set limits and I'd not deploy a server without the limits.

I had to make the fsyncs more frequent, not less, to reduce the odds of an operation sitting behind a long flush. This was a counter-intuitive finding for me, but it makes sense considering Persistent Disks throttle both IO sizes and IOPS. We do that to make sure a noisy neighbouring VM won't affect yours. So I turned on trickle_fsync and tuned the size of the maximum syncs to never have a slow flush, but a lot of smaller flushes.

Tuning the Java GC was for the same reason - I did not want long GC cycles. The settings I used were based on the guidance from Java memory ergonomics. The DataStax distribution has tips and hints in the cassandra-env.sh file itself, which I read it and followed through. I also read more about Java memory than I ever want to read again from various vendors. As the joke goes "I did even the things that contradicted the other things" before getting to the settings you saw.

I hope you appreciate the fact I set limits on some of the most dangerous knobs, for instance limiting the RPC server and using HSHA as opposed to unbounded sync.

I understand the limitations of the benchmark I used, and the limitations of the post. I tried to stay within recommended settings for all subsystems, have safe limits for everything I found dangerous, and I picked the workload because that is what our customers do. I don't advocate one storage solution over another, we love everyone that buys our solutions :)

BTW, while we do not sell Cassandra as a service, we do sell Cloud SQL (https://developers.google.com/cloud-sql/). Maybe the sales guys will give me a cut :)

Sharing the config was more work, but I suspected I was going to learn something by doing it. Thank you for the feedback!

PS: sorry for the delay replying - Y Combinator says I am posting too much. I am not quite sure at what point I will hit my daily quota of replies.

Network average utilization was low by design. Keeping it steady was more important than low, though, and harder too.

Latency spikes come from Cassandra flushing data to disk (large sequential IO), Java garbage collection and heap resize, and page faults during compactions (random reads).

What I did to even traffic out was to enable trickle_fsync and size the flushes, set Java's max and min heap sizes, as well as to tune the Java heap ergonomics. I treated random reads as a fact of life - I did nothing to tune that.

Here is the GIST link: https://gist.github.com/ivansmf/6ec2197b69d1b7b26153

This is hilarious - it would have saved me days of work. Here is the GIST with my step by step. https://gist.github.com/ivansmf/6ec2197b69d1b7b26153

I wrote the test - Yep. Tail latency is one of the key things here. And I took 100% of all samples, as opposed to the middle 80% the tool usually reports.

I ran those tests. One of the challenges in benchmarking is to pick the workload, so I chose one our customers do more often. For IO testing I prefer to run FIO tests, and for networking iperf, but unless you do this for a living it is hard to relate to microbenchmarks. It would also leave out the Java settings.

The reason I left the tests running for an hour is because at some point Cassandra gets into compactions, which are CPU and read intensive. The engine memory maps the files, so the IO subsystem backing the storage sees a lot of random IOs as page faults kick in. Streaming IOs get in during fsyncs. Again, easier to see using FIO.

The cluster was brought up at once, and I added the ramp up time on the chart so folks could see it.

I had three goals for this test: - Show low latency is possible with remote storage and proper capacity planning. This is why I used quorum commit, which forces the client to wait for at least two nodes to commit. - Share the settings I used. If you open the GIST and download the tarball you'll find all changes I did to the cassandra.yaml and cassandra-env.sh. This benefits our customers directly because it gives them a starting point. - Recommend that customers look at all samples, not only the middle 80% the stress tool reports. Otherwise the cluster looks much better than it is.

Thank you for the comments! I can assure you I read them, and I'll incorporate suggestions as possible.