chrischen

They've also been making things smaller. Some would argue that it's a cultural thing, but being poor also means you have to start adopting austere cultural habits as a coping mechanism.

If I understand the correctly the initial models were done by scraping the internet off public data. They now probably pay for access, especially to companies that hold the data. Even in the latter case, the content creators probably don't see anything because they signed away their rights using whatever free service they uploaded their work / comment to. In hacker news' case I'm sure some bot is scraping my prose right now and training something with it, which I'm totally fine with because the act of trying to rent seek my 0.00001 cent of value in this post is not worth the detriment to AI advancement.

They already "stole" it. They aren't giving it back and they've established their valuations based off of that. If they start paying now, it's simply going to be impossible for any more upstarts to do this or even release open-weight models because everyone with data will become rent seekers. Imagine if they started off with rent-seekers, we'd simply not have the benefit of these models at all at this point.

This is the same argument people were making about how stealing music was the same as stealing a physical product.

Yes but all the AI companies took all the public data, so when you pay for an AI model you are paying for the marginal service of building a model off that data, not for the data itself. What we should do is ensure that the data is available to more people to train AI models... but sadly this doesn't seem to be happening. Instead AI companies that were first-movers got to train off public data, and as the companies and businesses that own this data get wise they're going to start charging people to train off the data. This will make it much more difficult for anyone to train a model in the future as it will become expensive, and the companies that did happen to already train off public data will get a bit of incumbent's advantage.

One the things I hate about Google is being forced to have location-aware search. I love how Kagi actually lets me override the country.

I guess it doesn't matter if they allowed OpenAI to do it or not because it seems other models were allowed to train off it too. I guess we should probably be giving kudos to GitHub and Microsoft for not trying to charge for access to this data.

What exactly did they train? Copilot is powered by claude, gemini, or ChatGPT these days.

Did they train autocomplete? I mean the code is open source so anyone can scrape it and train it too. I'm kind of glad they did train it because otherwise we'd still be stuck with Apple level AI models right now.

The whole reason we have so many models, including open weight models, that are all competitive with each other is because the data is free and anyone can be training off it. If the goal was to monetize the source code I guess the authors shouldn't make it open source.

The US starts wars… they just often happen to be with dictatorships. The US definitely also supported dictatorships (like Taiwan and South Korea).

You can argue all day about whether A is slightly more rotten than B, but if they are both rotten then in the grand scheme they will both end up being the same thing if something doesn’t get fixed.

The US starts wars… they just often happen to be with dictatorships. The US definitely also supported dictatorships (like Taiwan and South Korea).

Whether a country massacres its own people is not really a good litmus test since there are countries that treat its own citizens well but foreigners really badly. One such country is… oh the US!

A photo is easy to take but hard to reproduce.

The whole concept of intellectual property rights is a social and legal construct designed to promote innovation in an economy. If you don't care about that, then there really isn't any moral or immoral aspect to it. The immorality of it and associating it with stealing was just MPAA propaganda to try to shame people into paying for stuff.

If I found some DVD lying on the ground and watched it and I didn't pay for it, it's really up to me to decide if I want to pay the creator so they can continue to produce content. If I don't pay then obviously it doesn't help them produce more content... but the consumption of the content itself neither felt nor heard by the creators.

Yesterday I was trying to figure out if my expired nacho dip would be safe to eat and wanted to know how much botulism would be toxic if I ate it and so I asked Claude. It refused to answer that question so I could see how the current safeguards can be limiting.

It beeps at you if you stop paying attention, which is superior. Hands on wheel is an arbitrary design decision more likely to placate what a layman would think is necessary to ensure safe AI steering.

This doesn't mean that it's over for SO. It just means we'll probably trend towards more quality over quantity. Measuring SO's success by measuring number of questions asked is like measuring code quality by lines of code. Eventually SO would trend down simply by advancements of search technology helping users find existing answers rather than asking new ones. It just so happened that AI advanced made it even better (in terms of not having to need to ask redundant questions).

To be fair with the languages I use there are only a finite number of ways a particular line or even function can be implemented due to high level algebraic data structures and strict type checking. Business logic is encoded as data requirements, which is encoded into types, which is enforced by the type checker. Even a non-AI based system can technically be made to fill in the code, but AI system allows this to sort of be generalized across many languages that did not implement auto-complete.

Was it the github issues copilot integration? I found that to be slow compared to natively running copilot in the IDE.

With coding agents AI almost never manually type code anymore. It would be great to have a code editor that runs on my phone so I can do voice prompts and let the coding agents type stuff for me.

I also try to do verbose type classes using Ocaml's module system and it's been handling these patterns pretty well. My guess is there is probably good documentation / training data in there for these patterns since they are well documented. I haven't actually used coding agents with Haskell yet so it's possible that Ocaml's verbosity helps the agent.

Actually I found the coding models to work really well with these languages. And the type systems are not actually complex. Ocaml's type system is actually really simple, which is probably why the compiler can be so fast. Even back in the "beta" days of Copilot, despite being marketed as Python only, I found it worked for Ocaml syntax and worked just as well.

The coding models work really well with esoteric syntaxes so if the biggest hurdle to adoption of haskell was syntax, that's definitely less of a hurdle now.

Instead, you're writing non-trivial proofs about all possible runs of the program.

All possible runs of a program is exactly what HM type systems type check for. This fed into the coding model automatically iterates until it finds a solution that doesn't violate any possible run of the program.

We already have verification layers: high level strictly typed languages like Haskell, Ocaml, Rescript/Melange (js ecosystem), purescript (js), elm, gleam (erlang), f# (for .net ecosystem).

These aren’t just strict type systems but the language allows for algebraic data types, nominal types, etc, which allow for encoding higher level types enforced by the language compiler.

The AI essentially becomes a glorified blank filler filling in the blanks. Basic syntax errors or type errors, while common, are automatically caught by the compiler as part of the vibe coding feedback loop.

I would guess HN readers are not an average cross-section of broader society, but I would also guess that because of that HN readers would be pretty bad at understanding what broader society is thinking.

I guess the translation can always update itself in real time if the model is fast enough.

Some of your points are already considered with current implementations. Airpods live translate uses your phone to display what you say to the target person, and the target person's speech is played to your airpods. I think the main issue is that there is a massive delay and apple's translation models are inferior to ChatGPT. The other thing is the airpods don't really add much. It works the same as if you had the translation app open and both people are talking to it.

Aircaps demos show it to be pretty fast and almost real time. Meta's live captioning works really fast and is supposed to be able to pick out who is talking in a noisy environment by having you look at the person.

I think most of your issues are just a matter of the models improving themselves and running faster. I've found translations tend to not be out of whack, but this is something that can't really be solved except by having better translation models. In the case of Airpods live translate the app will show both people's text.

This is the sad reality of most if these AI products and it’s that they are just taking poor feature implementations on the hardware. It seems like if they just picked one or these features and doing it well will make the glasses useful.

Meta has a model just for isolating speech in noisy environments (the “live captioning feature”) and it seems that’s also the main feature of the Aircaps glasses. Translation is a relatively solved problem. The issue is isolating the conversation.

I’ve found meta is pretty good about not overdelivering on promised features, and as a result even though they probably have the best hardware and software stack of any glasses, the stuff you can do with the Rayban displays are extremely limited.

Real time translations is a real good use case. The problem is most implementations such as the Airpods live translate are not great.

Current LLMs are doing this for coding, and it's very effective. It delegates to tool calls, but a specialized model can just be thought of as another tool. The LLM can be weak in some stuff handled by simple shell scripts or utilities, but strong in knowing what scripts/commands to call. For example, doing math via the model natively may be inaccurate, but the model may know to write the code to do math. An LLM can automate a higher level of abstraction, in the same way a manager or CEO might delegate tasks to specialists.

But these models are more like generalists no? Couldn’t they simply be hooked up to more specialized models and just defer to them the way coding agents now use tools to assist?

I think a bigger problem is the HN reader mind reading what the rest of the world wants. At least when an HN reader telling us what they want it's a primary source, but reading a comment about an HN reader postulating what the rest of the world wants is simply more noisy than an unrepresentative sample of what the world may want.