HN user

jcrites

2,978 karma

https://www.linkedin.com/in/justin-crites/

Former: Chief Software Engineer @ HumanInterest.com (YC) (2020-2022). Principal Software Engineer @ Amazon & AWS (2007 - 2020), WhatsApp (2021-22).

I am not a spokesperson. Everything I post is my own opinion, and does not reflect the views of any other person or company.

Please feel free to reach out any time: jcrites at {gmail.com}

I'm a swimmer and standing desk evangelist. I love building and writing software, and using platforms like Linux, Rust, AWS, and am happy to chat with other folks who do too.

[ my public key: https://keybase.io/jcrites; my proof: https://keybase.io/jcrites/sigs/0LyZppXBpyBYCjr6wo1N-b6fVRKQFvFxhd44Z8blDfM ]

Posts4
Comments640
View on HN

I think this is ultimately true, in a sense, but the challenge is correctly handling all of the edge-cases. It's a challenging problem tantamount to the self-driving car problem.

It happens by humans over VHF because a lot of unpredictable things happen in busy airspace, and it would require a massive investment for machines to automate all of it.

I'm also not sure that people would accept the safety risk of airplanes' autopilots being given automated instructions by ATC over the air. There's a large potential vulnerability and safety risk there. I think there's some potential for automation to replace the role of ATC currently, but I suspect it would still be by transmitting instructions to human pilots, not directly to the autopilot.

Lastly, for such a system to ever be bootstrapped, it would still need to handle all of the planes that didn't have this automation yet; it would still need to support communicating with pilots verbally over VHF. An entirely AI ATC system, that autonomously listens to and responds by voice over VHF seems like a plausible first step though.

Doesn't the budget to pay these people already exist? They are current employees, after all. Their terminations will still shrink the budget, just not as soon as a termination without a severance package.

Maybe there are some legal differences in offering what would otherwise be wages as a lump-sum payment, but the budget for those wages already exists (else they could not be employed).

Looking at the source, I don't think there's an actual need for the constructor to take a `Class<T>` type. It's used internally to initialize an array [1]:

  this.entries = (T[]) Array.newInstance(type, capacity);
However, I think this alternative would work equally well, and would not require a `Class<T>` parameter:
  this.entries = (T[]) new Object[capacity];
There are some other choices in the library that I don't understand, such as the choice to have a static constructor with this signature:
  public static RingBuffer<Object> create(final int capacity) {
    return create(capacity, false);
  }
This returns the type `RingBuffer<Object>`, which isn't as useful as it could be; with appropriate idiomatic use of generics it could return a `RingBuffer<T>`:
  public static <T> RingBuffer<T> create(final int capacity, final boolean orderedReads) {
    return new RingBuffer<>(capacity, orderedReads);
  }
It's possible that this code was written by someone who is still learning idiomatic Java style, or effective use of generics.

I'm also curious about the choice to have `get()` return `null`. I think I'd rather have seen this modeled with `Optional`. My preferred style when writing Java code is to employ non-nullable references wherever possible (though the return from `get()` is marked `@Nullable` at least).

[1] https://github.com/evolvedbinary/j8cu/blob/94d64cfc0ec49a340...

True, but accusing him of a violation privately, to him, is not defamation. Accusing him publicly is still probably not defamation. Whether he violated certain terms of service is something that would ultimately need to be decided by a court, so them believing he did so isn't defamation even if they're wrong.

Let me take a shot at explaining continuations.

In normal programming, functions "return" their values. In Continuation Passing Style (CPS), functions never return. Instead, they take another function as input; and instead of returning, they call that function (the "continuation"). Instead of returning their output, they pass their output as input to the continuation.

(Some optimizations are used such that this style of call, the "tail call", does not cause the stack to grow endlessly.)

Why would you write code in this style? Generally, you wouldn't. It's typically used as an internal transformation in some types of interpreters or compilers. But conceptualizing control flow in this way has certain advantages.

Then there are terms like the "continuation" of a program at a certain point in the code, which just means "whatever the program is going to do next, after it returns (or would return) from the code that it's about to execute". That's what "call with current continuation" (call/cc) is about. It captures (or reifies) "what will the program do next after this?" as a function that can be called to do, well, do that thing. If your code is about to call `f();`, then the 'continuation' at that point is whatever the code will do next after `f()` returns with its return value.

Thus if you had some code `g(f())`, then the continuation just as you call `f()` is to call `g()`. CPS restructures this so that `f()` takes the "thing to do next" as input, which is `g()` in this case. The CPS transformation of this code would be `f(g)`, where `g` is the continuation that `f` will invoke when it's done. Instead of returning a value, `f` invokes `g` passing that value as input.

You can use continuations to implement concepts like coroutines. With continuations, functions never need to "return". It's possible to create structures like two functions where the control flow directly jumps between between them, back and forth (almost like "goto", but much more structured than that). Neither one is "calling" the other, per se, because neither one is returning. The control flow jumps directly between them as appropriate, when one function invokes a continuation that resumes the other. The functions are peers, where both can essentially call into the other's code using continuations.

That's probably a little muddy as a first exposure to continuations, but I'm curious what you think. I generally think of continuations as a niche thing that will likely only be used by language or library implementors. Most languages don't support them.

Also, I'd probably argue that regular asynchronous code is a better way to structure similar program logic in modern programming languages. Or at least, it's likely just as good in most ways that matter, and may be easier to reason about than code that uses continuations.

For example, one use-case for coroutines is a reader paired with a writer. It can be elegant because the reader can wait until it has input, and then invoke the continuation for the writer to do something with it (in a direct, blocking fashion, with no "context switch"). But you can model this with asynchronous tasks pretty easily and clearly too. It might have a little more overhead, to handle context switching between the asynchronous tasks, but unlikely enough to matter.

Could that also be because he reviewed the papers first and made sure they were in a suitable state to publish? Or you think it really was just the name alone, and if you had published without him they would not have been accepted?

Introducing S2 2 years ago

I think they're saying that you should provide some example use-cases for how someone would use your service. High-level use-cases that involve solving problems for a business.

For what it's worth, I am already familiar with this design space well enough that I don't need this kind of example in order to understand it. I've worked with Kinesis and other streaming systems before. But for people who haven't, an example might help.

What kind of business problem would someone have that causes them to turn to your service? What are the alternative solutions they might consider and how do those compare to yours? That's the kind of info they're asking for. You might benefit from pitching this such that people will understand it who have never considered streaming solutions before and don't understand the benefits. Pitch it to people who don't even realize they need this.

From context, I would infer that this means they are not changing the Java language itself. It’s a feature expressed through the existing language, like as a library feature. I could be wrong though.

The single best metric I've found for scaling things like this is the percent of concurrent capacity that's in use. I wrote about this in a previous HN comment: https://news.ycombinator.com/item?id=41277046

Scaling on things like the length of the queue doesn't work very well at all in practice. A queue length of 100 might be horribly long in some workloads and insignificant in others, so scaling on queue length requires a lot of tuning that must be adjusted over time as the workload changes. Scaling based on percent of concurrent capacity can work for most workloads, and tends to remain stable over time even as workloads change.

For what it's worth, I disagree. I think it would be difficult to design a language more elegant than Rust while accomplishing the same goals. Maybe those goals aren't ones that everyone cares about, and that's OK.

It's a language that's definitely worth learning; it expands the mind in a way that languages like Go (or Java, or any GC language) will not. There is a great beauty and elegance to its design.

For servers, I think the single most important statistic to monitor is percent of concurrent capacity in use, that is, the percent of your thread pool or task pool that's processing requests. If you could only monitor one metric, this is the one to monitor.

For example, say a synchronous server has 100 threads in its thread pool, or an asynchronous server has a task pool of size 100; then Concurrent Capacity is an instantaneous measurement of what percentage of these threads/tasks are in use. You can measure this when requests begin and/or end. If when a request begins, 50 out of 100 threads/tasks are currently in-use, then the metric is 0.5 = 50% of concurrent capacity utilization. It's a percentage measurement like CPU Utilization but better!

I've found this is the most important to monitor and understand because it's (1) what you have the most direct control over, as far as tuning, and (2) its behavior will encompass most other performance statistics anyway (such as CPU, RAM, etc.)

For example, if your server is overloaded on CPU usage, and can't process requests fast enough, then they will pile up, and your concurrent capacity will begin to rise until it hits the cap of 100%. At that point, requests begin to queue and performance is impacted. The same is true for any other type of bottleneck: under load, they will all show up as unusually high concurrent capacity usage.

Metrics that measure 'physical' (ish) properties of servers like CPU and RAM usage can be quite noisy, and they are not necessarily actionable; spikes in them don't always indicate a bottleneck. To the extent that you need to care about these metrics, they will be reflected in a rising concurrent capacity metric, so concurrent capacity is what I prefer to monitor primarily, relying on these second metrics to diagnose problems when concurrent capacity is higher than desired.

Concurrent capacity most directly reflects the "slack" available in your system (when properly tuned; see next paragraph). For that reason, it's a great metric to use for scaling, and particularly automated dynamic auto-scaling. As your system approaches 100% concurrent capacity usage in a sustained way (on average, fleet wide), then that's a good sign that you need to scale up. Metrics like CPU or RAM usage do not so directly indicate whether you need to scale, but concurrent capacity does. And even if a particular stat (like disk usage) reflects a bottleneck, it will show up in concurrent capacity anyway.

Concurrent capacity is also the best metric to tune. You want to tune your maximum concurrent capacity so that your server can handle all requests normally when at 100% of concurrent capacity. That is, if you decide to have a thread pool or task pool of size 100, then it's important that your server can handle 100 concurrent tasks normally, without exhausting any other resource (such as CPU, RAM, or outbound connections to another service). This tuning also reinforces the metric's value as a monitoring metric, because it means you can be reasonably confident that your machines will not exhaust their other resources first (before concurrent capacity), and so you can focus on monitoring concurrent capacity primarily.

Depending on your service's SLAs, you might decide to set the concurrent capacity conservatively or aggressively. If performance is really important, then you might tune it so that at 100% of concurrent capacity, the machine still has CPU and RAM in reserve as a buffer. Or if throughput and cost are more important than performance, you might set concurrent capacity so that when it's at 100%, the machine is right at its limits of what it can process.

And it's a great metric to tune because you can adjust it in a straightforward way. Maybe you're leaving CPU on the table with a pool size of 100, so bump it up to 120, etc. Part of the process for tuning your application for each hardware configuration is determining what concurrent capacity it can safely handle. This does require some form of load testing to figure out though.

Having DNS records leak could actually provide potential information on things you'd rather not have public.

This is true, but using a regular domain name as your root does not require you to actually publish those DNS records on the Internet.

For example, say that you own the domain `example.com`. You can build a private service `foo.example.com` and only publish its DNS records within the networks where it needs to be resolved – in exactly the same way that you would with `foo.internal`.

If you ever decide that you want an Internet-facing endpoint, just publish `foo.example.com` in public DNS.

Are there any good reasons to use a TLD like .internal for private-use applications, rather than just a regular gTLD like .com?

It's nice that this is available, but if I was building a new system today that was internal, I'd use a regular domain name as the root. There are a number of reasons, and one of them is that it's incredibly nice to have the flexibility to make a name visible on the Internet, even if it is completely private and internal.

You might want private names to be reachable that way if you're following a zero-trust security model, for example; and even if you aren't, it's helpful to have that flexibility in the future. It's undesirable for changes like these to require re-naming a system.

Using names that can't be resolved from the Internet feels like all downside. I think I'd be skeptical even if I was pretty sure that a given system would not ever need to be resolved from the Internet. [Edit:] Instead, you can use a domain name that you own publicly, like `example.com`, but only ever publish records for the domain on your private network, while retaining the option to publish them publicly later.

When I was leading Amazon's strategy for cloud-native AWS usage internally, we decided on an approach for DNS that used a .com domain as the root of everything for this reason, even for services that are only reachable from private networks. These services also employed regular public TLS certificates too (by default), for simplicity's sake. If a service needs to be reachable from a new network, or from the Internet, then it doesn't require any changes to naming or certificates, nor any messing about with CA certs on the client side. The security team was forward-thinking and was comfortable with this, though it does have tradeoffs, namely that the presence of names in CT logs can reveal information.

Many log aggregation stores are not optimized for performing row-level updates or deletes like this. In my experience, the majority of log aggregation stores are immutable and support primarily time-based retention only.

(Though perhaps one can meet compliance needs by keeping these logs only for a fixed maximum period of time, e.g. 30 days, and keeping only appropriately anonymized data longer.)

Please consider presenting the evidence without the personal swipes, and please consider reviewing the HN Guidelines: https://news.ycombinator.com/newsguidelines.html

Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes.

When disagreeing, please reply to the argument instead of calling names. "That is idiotic; 1 + 1 is 2, not 3" can be shortened to "1 + 1 is 2, not 3."

Your comments in this thread would be more constructive without the swipes.

That's workable for many situations which only require data "storage", but the moment when you require cloud-side data "processing" too, the situation changes.

I would agree that if you have no use-case for cloud-side data processing, then the cloud doesn't need the encryption keys; however there are a lot of use-cases for data for which processing is highly advantageous.

For example, if you just want to store data in the cloud, sure, use client-side encryption. But what if you want to query it with, using an S3 example, S3 Select or Athena or load it into a data lake? There are a lot of things one might do. It's useful to be able to query data from within the cloud – without having to transfer it out and separately decrypt it.

This use-case is likely one of the primary reasons why customers are willing to trust cloud providers to manage the keys too – for many data sets, though not all. And that's what technologies like cloud KMS are good for. Access to the actual keys is tightly limited within the cloud provider, both among employees and technologically, and not even other cloud services are capable of accessing them (either accessing the keys or encrypting/decrypting using them) unless you grant them explicit permission, with internal technology that enforces this.

Maybe you didn't intend on using "new fancy foo service" to process your stored data yesterday, but today it sounds good, so you can alter an access control grant and allow it to interact with your existing stored data and the necessary encryption keys. Maybe every time some new data arrives in S3, you want to process that data with Lambda and synchronize it with another system. Then it's helpful if you can grant Lambda permissions to decrypt the data so that it can process it :-)

Generic server-side encryption does do something: it protects against threats that exist at the data center layer, such as certain attacks through lower layers of infrastructure that don't necessarily involve a full compromise of the machines involved, or inappropriate disposal of old storage devices, mistakes by employees, etc. It protects against some insider attacks, just not those by capable administrators.

Please take a moment to review the HN guidelines: https://news.ycombinator.com/newsguidelines.html

Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes.

Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith.

Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something.

Please don't post insinuations about [] shilling[] and the like. It degrades discussion and is usually mistaken.

I'm just interpreting the context (guessing) but it sounds like his presence in the CODEOWNERS file means that he's being added to PRs automatically, and without people clearly asking for or wanting his input on them specifically. And perhaps without waiting for his input before merging. So he may want to be tagged on PRs only if people deliberately want his input, rather than by default.

Just reasoning about this from first principles, but intuitively, the more dimensions you have, the more likely that you are to find a gradient in some dimension. In an N-dimensional space, a local minimum needs to be a minimum in all N dimensions, right? Otherwise the algorithm will keep exploring down the gradient. (Not an expert on this stuff.) The more dimensions there are, the more likely it seems to be that a gradient exists down to some greater minimum from any given point.

They seem better geared for usage in databases as primary keys, specifically UUID versions 6 and onwards:

Motivation. One area in which UUIDs have gained popularity is database keys. This stems from the increasingly distributed nature of modern applications. In such cases, "auto-increment" schemes that are often used by databases do not work well: the effort required to coordinate sequential numeric identifiers across a network can easily become a burden. The fact that UUIDs can be used to create unique, reasonably short values in distributed systems without requiring coordination makes them a good alternative, but UUID versions 1-5, which were originally defined by [RFC4122], lack certain other desirable characteristics [...]

UUIDv6 is a field-compatible version of UUIDv1 (Section 5.1), reordered for improved DB locality. It is expected that UUIDv6 will primarily be implemented in contexts where UUIDv1 is used. Systems that do not involve legacy UUIDv1 SHOULD use UUIDv7 (Section 5.7) instead.

Instead of splitting the timestamp into the low, mid, and high sections from UUIDv1, UUIDv6 changes this sequence so timestamp bytes are stored from most to least significant. That is, given a 60-bit timestamp value as specified for UUIDv1 in Section 5.1, for UUIDv6 the first 48 most significant bits are stored first, followed by the 4-bit version (same position), followed by the remaining 12 bits of the original 60-bit timestamp. [...]

UUIDv7 features a time-ordered value field derived from the widely implemented and well-known Unix Epoch timestamp source, the number of milliseconds since midnight 1 Jan 1970 UTC, leap seconds excluded. Generally, UUIDv7 has improved entropy characteristics over UUIDv1 (Section 5.1) or UUIDv6 (Section 5.6).

Why wouldn't the "script" be all of the necedssary commands to create the entire database?

If any migration was necessary to transform one table structure to another, that wouldn't be useful to keep around long term, nor interesting once the new table is established. It might be kept as a historical artifact, but why would you on average care beyond what the current schema is now, along with its documentation?

Hmmm, I don't agree. They do seem to understand some things.

I asked ChatGPT the other day how fast an object would be traveling if "dropped" from the Earth with no orbital velocity, by the time it reached the sun. It brought out the appropriate equations and discussed how to apply them.

(I didn't actually double-check the answer, but the math looked right to me.)

It also seems to have a calculation or "analysis" function now, which gets activated when asking it specific mathematical questions like this. I've imagined it works by using the LLM to set up a formula, which is then evaluated in a classical way.

There are limits on what it can do, just like any human has similar limits. ChatGPT can answer more questions like this correctly than the average person could from off the street. That seems like understanding to me.

True, but modern CAPTCHA rate limiting is not easily bypassed, and a lot of the solutions are free.

Together with cookies, you can show the captcha only to visitors that are not already recognized in some way, giving them a limited number of actions before showing the captcha. And regardless of whether you want one on your password reset page, you almost certainly want one on your login page anyway.

For domesticated cats and dogs, being inside the home is their natural environment (or can be, to varying degrees).

I've owned cats and dogs. I can let them outside and leave the door open, and they may go outside to explore or play briefly, but they'll want to come inside shortly afterward. Especially my dog - he doesn't even want to be outside by himself, except to relieve himself. He has herding instincts and is uncomfortable being alone outdoors for more than a short time. He would much rather be around people, but if he had to be by himself, then he'd rather be indoors. (Observation: if I were to leave the house, and left the back door open, he'd still spend most of the day inside rather than in the yard.) My cat acts downright frightened being outside.

It's not really possible to discuss what is "natural" here, and I'd recommend being cautious about the naturalistic fallacy [1]: the idea that because something is 'natural' it must be better. It is 'natural' for humans to die of all kinds of diseases, yet we do not wish to, and so we treat ourselves with antibiotics and other medicines. It is 'natural' for children and women to die in childbirth, but we take substantial steps to prevent that, and so on.

In the case of domesticated animals, it's hard to reason about what "natural" even is. The animals have adapted to living with humans; they want to be around humans. They aren't feral, and wouldn't want to live by themselves in the wilderness any more than a human would. They have a choice every time I open the door.

(And yes, if they ran away, I would feel obligated to find them, but that's because I'm responsible for their health and safety, not because they are my prisoners. The point is that they don't even come close to making the choice to do that. It is far more likely that they would simply get lost outside than actually intentionally run away. This has happened once before to my dog, and his emotional reaction upon me finding him demonstrated compellingly that he felt lost and was happy to be found and reunited.)

I might feel conflicted if I ever had a pet that behaved like it wanted to escape from me, but that's never been the case. Even pets that do enjoy to spend time outdoors alone - and I'm happy for them to do that - always want to return to the safety of the home after a time. I would not enjoy having a pet if I thought at all that it felt like a prisoner.

Lastly, I would remark on the fact that these animals were raised from infancy by humans and around humans. They are socialized to be part of the family, just like people are. While it is true that they would have different personalities if they were raised in the wilderness, the same would also be true of a human child. If a human child were abandoned alone in the wilderness and somehow survived, then this human would undoubtedly have great difficulty socializing as an adult, and fitting into a family - yet we would not say that this wild person is somehow better or more natural. We would say that their upbringing was cruel.

[1] https://en.wikipedia.org/wiki/Naturalistic_fallacy

Inner Speech 3 years ago

I do think it is interesting to explore, compare, and contrast the operating modes and limitations of our minds.

Like you, I can also vary volume and tone of an imagined voice. I can make it arbitrarily loud, but there does seem to be a limit on how quiet I can make it, unless I'm imagining hearing a voice from elsewhere.

I don't think I dream in color, though, but I wouldn't call it black and white either. It's more like I'm dreaming of objects in an abstract space where color isn't relevant. I have memories of my dreams from time to time, and those memories aren't of a scene with color in them. I absolutely dream of conversations, and so perhaps I dream with audio, but I'm not exactly dreaming visually. I'm dreaming of circumstances. I can totally believe that other people might dream in color though.