HN user

dwallin

1,257 karma
Posts1
Comments287
View on HN

Some people point at LLMs confabulating, as if this wasn’t something humans are already widely known for doing.

I consider it highly plausible that confabulation is inherent to scaling intelligence. In order to run computation on data that due to dimensionality is computationally infeasible, you will most likely need to create a lower dimensional representation and do the computation on that. Collapsing the dimensionality is going to be lossy, which means it will have gaps between what it thinks is the reality and what is.

The question is not whether the alternative is perfect, the question is can it be made better than the status quo. It’s not that hard to come up with potential mitigations for the problems you state.

- A taxable threshold, so people who can’t afford lawyers and accountants don’t need to deal with it. Works well for family gifting.

- You don’t need to tax immediately, tax it when it the profit is realized, eg. When you sell that art.

- Taking out a loan against an asset at an increased valuation should trigger a taxable event. (Eg. Stocks go from 1b to 2b valuation and you take out a 500m loan. You are realizing 250k of gains and should pay tax on that gain.)

- Eliminate stepped up cost basis. This is a ridiculous give away.

When did Ben Thompson go so far down the path to autocratic sympathizer? This is such an anti-democratic, anti-free market, anti-free speech view on this whole situation.

First, everything revolves around a core conceit that "Might makes right". The idea that entities might might push back with the tools at their disposal is treated as a fools errand, you should just acquiesce.

The role of the legislative branch in deciding what private entities are allowed, or not allowed to do is treated as a side note. He equates the dictates of the executive branch as if it was the will of the United States itself, above even the Constitution.

It's dismissive of the rights of private companies and individuals to make decisions for themselves about the actions they take and whether or how they choose to transact within the law with parts of the executive branch.

He acts as if it's a foregone conclusion that every AI company should be considered an arm of the executive branch of the US government. The analogy to nuclear weapons is super flawed, there are multiple laws on the books (written into law by Congress) specifically regulating Nuclear research and development.

And most astonishingly he ends it dropping an implied threat of violence towards Anthropic (and assumedly anyone else who doesn't agree with his point of view):

I don’t want that, and, more pertinently, the ones with guns aren’t going to tolerate it.

Wow.

It’s a non-clause that is written to sound like they are doing something to prevent these uses when they aren’t. “You are not allowed to do illegal things” is meaningless, since they already can’t legally do illegal things. Plus the administration itself gets to decide if it meets legal use.

This is such a bad faith question, it's annoying to see it come up again as if there's any utility to asking it.

The question itself never specifies that the car you would be driving is the same one that you need to be washed. The car that needs to be washed could be waiting in the parking lot of the car wash already. It doesn't state that you plan on washing your car at the car wash. Perhaps the car wash sells car cleaning equipment that you can bring back to wash your car at home?

The question is designed to be ambiguous so the llm answers it in a way that seem facially absurd to the people who are in on the scheme. What it's actually showing is a failure of imagination for those asking the question.

Do you want your chatbot to be suspicious of you trying to trick it? To me this seems patently unhelpful outside of LLMs tuned for roleplay or to operate in a highly adversarial environment.

Do you want it to assume you are an idiot asking the question because you didn't realize you need to have the car at the car wash to wash it?

Or do you want it to take the best faith assumption as to what you are asking and try to be as helpful as possible given the poor question?

Given that these tendencies are not evenly distributed throughout the population, you can have structures that leverage the large mean to mitigate the worst tendencies of the extreme tails. Given that the natural state of things is that power begets more power, these are harder to build and maintain, but it can be done. In particular, Democracies and Republics are major historical examples of this.

When it comes to new emerging technologies everyone is searching the space of possibilities, exploring new ways to use said technologies, and seeing where it applies and creates value. In situations such as this, a positive sign is worth way more than a negative. The chances of many people not using it the right way are much much higher when no one really knows what the “right” way is.

It then shows hubris and a lack of imagination for someone in such a situation to think they can apply their negative results to extrapolate to the situation at large. Especially when so many are claiming to be seeing positive utility.

IMO this is a mistake, for basically the same reason you justify it with. Since most people just want the code to work, and the chances of any specific repo being malicious is low, especially when a lot of the repos you work with are trusted or semi-trusted, it easily becomes a learned behavior to just auto accept this.

Trust in code operates on a spectrum, not a binary. Different code bases have vastly different threat profiles, and this approach does close to nothing to accomodate for that.

In addition, code bases change over time, and full auditing is near impossible. Even if you manually audit the code, most code is constantly changing. You can pull an update from git, and the audited repo you trusted can be no longer trustworthy.

An up front binary and persistent, trust or don't trust model isn't a particularly good match match for either user behavior or the potential threats most users will face.

Very much agree with this. Looking at the dimensionality of a given problem space is a very helpful heuristic when analyzing how likely an llm is going to be suitable/reliable for that task. Consider how important positional encodings are LLM performance. You also then have an attention model that operates in that 1-dimensional space. With multidimensional data significant transformations to encode into a higher dimensional abstraction needs to happen within the model itself, before the model can even attempt to intelligently manipulate it.

In order to make better ai tools for generating specific parts of a song, you ideally want models that understand what good music sounds like when put together. These sorts of "generate whole songs" are a predecessor to more specific tooling. These tools are slowly moving downstream (look at the evolution of Suno) and will almost certainly eventually move to a place where they are just a part of the music production workflow. We increasingly have improved tools to break down full tracks into stems, stems to/from midi/lyrics.

Lots of potential musicians / producers that can write a catchy tune, lyrics, create midi work, etc; but maybe can't play / don't own the instruments they want to use (could be disabled) or maybe don't have a great singing voice. These ai tools can lower the bar for more people to create music at a higher level. It can also act as a improvisational partner, to explore more musical space faster.

As a personal anecdote of where AI might be useful, as a hobby I occasionally participate in game jams, sometimes working on music / sound effects to stretch my legs form my day job. One game jam game I worked on was inspired by a teammates childhood in Poland. So I listened to a bunch of traditional Polish music and created a track inspired by said music. I'm pretty happy with how it came out, but with current AI I'm sure I could have improved the results significantly. If I were to be making it now, I would be able to upload the tracks I wrote, see how the AI might bring it closer to something that sounds authentic, and using that to help me rewrite parts of the melody where it was lacking. Then I could have piped in my final melody with it's inauthentic midi instrument (I neither own, nor play traditional polish stringed instruments) and used it to make something that sounds much closer to my target, with a more organic feel.

I would say that the partial counterpoint to that is, for most people their values are also largely tribe based, in that their values are not purely fixed, but rather tend to adapt to loosely track the tribal consensus. Very few are the ones willing to stick to their convictions under pressure.

There are clearly some (many?) shared average axiomatic values that seem to be common between very different cultures/religions (although individuals vary much more significantly), but it's much easier to obsess on the places we differ.

Where I strongly disagree is the idea that groups with different fundamental values can't necessarily find common policy ground. A good example is Basic Income, where you can find agreement between groups on opposite sides that both embrace the idea, but for very different value-driven reasons. In many cases, you can also agree to disagree, and just keep your collective hands out of it (eg. separation of religion and state).

The biggest click bait is the “It” which can clearly be seamlessly replaced with BYD, providing vastly increased utility at basically no increased complexity; no hard value determination needed. Intentionally removing useful information in an attempt to abuse the curiosity of readers and drive numbers is a type of enshittification in the news and information sphere.

I agree that what ai music needs to become an industry tool is the ability to create, access and remix parts, but I think tools like Suno have more of the right idea vs tools like this. In order to be able to write intermediate parts properly, you need to be able to understand the whole and what things should sound like when put together, or when the notes are actually played by a musician. Then it’s easier to work back from there, split your tracks apart into stems, transcribe your stems into MIDI, etc.

Suno et al are moving in this direction but I honestly think development will be somewhat stunted until we get a good open source model(s), and something like control-nets.

There are a couple of things that human’s in general are bad at, and intuition is unhelpful, even if you are otherwise an intelligent, learned person. Two I see pop up a lot are Statistics and Systems Thinking. In particular with Systems Thinking, people tend to assume that things that have a simple clear relationship at the scale they operate in, will have the same effect when applied at the scale of the system as a whole. Which is quite often not the case.

The constitution also explicitly sets up formal departments with specific purviews, with heads that need to be approved by congress. It also outlines that the president has the right to get the opinion of said principal offices about their duties (while seemingly failing to state any right to direct said opinions) This implies that the president’s executive authority over the departments is far from absolute, since if it was, why would you need to explicitly bestow a right to merely seek opinions?

If anything, the constitution implies that department heads SHOULD have independent opinions related to the purview of their departments.

“The President shall be Commander in Chief of the Army and Navy of the United States, and of the Militia of the several States, when called into the actual Service of the United States; he may require the Opinion, in writing, of the principal Officer in each of the executive Departments, upon any Subject relating to the Duties of their respective Offices, and he shall have Power to grant Reprieves and Pardons for Offences against the United States, except in Cases of Impeachment.”

You can group your first three (food, water, shelter) under your fourth (survival). Then your next two examples are cases of how people feel.

On top of that you can find plenty of clear evidence for situations where humans choose feelings over survival, whether that’s giving up one’s own survival for the sake of people or a cause, or giving up on survival because emotional needs aren’t met.

It’s not as clear cut of a hierarchy as you seem to think it is.

The cost of having excess compute is less than the cost of not having enough compute to be competitive. Because of demand, if you realize you your current compute is insufficient there is a long turnaround to building up your infrastructure, at which point you are falling behind. All the major players are simultaneously working on increasing capabilities and reducing inference cost. What they aren’t optimizing is their total investments in AI. The cost of over-investment is just a drag on overall efficiency, but the cost of under-investment is existential.

There's a nugget of something clever here but it's missing something, there's too much luck and not quite enough skill for a golf game.

My first thought was selecting from various dice with different distributions to represent the various golf clubs.

My second was that a hex grid would offer a much better variety of options. The square grid only has 1 or 2 useful directions that can bring you closer. A hex grid would typically have 2 to 3 useful directions.a hex

Then I thought, this sounds like a good base for a board game, did a search and turns out there already is one that uses that formula! https://boardgamegeek.com/image/6370656/roll-in-one

I can't take an Andy Warhol painting, modify it in some way and then claim it's my own original work.

This is a hilarious choice of artist given that Warhol is FAMOUS for appropriating work of others without payment, modifying it in some way, and then turning around and selling it for tons of money. That was the entire basis of a lot of his artistic practice. There was even a Supreme Court case about it.

I do want to point out, gaussian splats don't really offer anything in particular for realtime relighting, if anything it adds additional challenges. Under the hood, most implementations leverage spherical harmonics for baked-in lighting.

Did you by chance mean anisotropic spherical gaussians? ASG is a new-ish technique often used to model specular lighting but is unrelated to gaussian splatting.

From an amazon employee reply in the thread:

"When we say “Manufacturing cost”, it means your cost to source a product from a manufacturer/wholesaler/reseller, or produce the item if you are the manufacturer. You can provide the proof of your cost of sourcing and we will reimburse you accordingly. If you do not wish to provide your cost, we will provide our cost estimate and we will reimburse you for it. We calculate our estimate by evaluating the sourcing cost of comparable products sold by Amazon, by other sellers, and through other wholesale channels.

It excludes costs such as shipping, handling, customs duties, or other costs."

You could possibly try setting up your own "reseller" entity, where they act as an in-between, handling the shipping costs, etc and then reselling it to upon arrival for near the price sold?