someguyorother

Why do LLMs use these phrases so much if humans rarely use them in written form?

As far as I understand, it's due to RLHF. The reviewers the AI companies use don't necessarily know what kind of question is a good one, so when the LLM answers "That's a good question!", they tend to rate the answer higher because they like being flattered. Proxy models that are themselves trained on RLHF inherit this pattern. Similar effects contribute to sycophancy.[1]

[1] https://arxiv.org/abs/2310.13548

The mental transaction cost is the hard part. The effort required to decide whether to pay at all is significant enough that payments don't scale down to the micro- level.

It's only mathematics-adjacent, but Stephen Hawking was known for making quite a few bets.

https://www.science.org/doi/10.1126/science.359.6382.1317

I don't think that's people who refer to Tyranny of Structurelessness mean.

At least I read it more as that you can't just declare 'there be no hierarchy here' and be done. Unless you carefully engineer the system, the implicit hierarchy will reclaim the void and, all else equal, an implicit hierarchy is harder to undo because it isn't supposed to exist.

In political terms: if all you do is kick the ruler out, you may get a corrupt patronage network instead of democracy. Actual equality doesn't come from just the absence of strong explicit hierarchy; it requires proper institutional design.

Sure thing, here's your neural VR interface and extremely high fidelity artificial world with as many paperclips as you want. It even has a hyperbolic space mode if you think there are too few paperclips in your field of view.

Users of practical software would probably not accept the program taking forever, so you could implement a runtime constraint. With a runtime constraint, every TM effectively halts, so making nontrivial observations about them should at least be computable.

Not that it would be easy.

The dark humor in this is that any such technologically advanced future where humans have a meaningful say will eventually look like one of abundant luxury communism: it's just that the oligarchs' version will have a lot of people die first before the oligarchs enjoy their abundance.

The third option is that the oligarchy fully internalizes its pursuit of ruthless concentration of power. But in that case, someone will probably create an AI that's better at playing the power game, and at that point, it's over for the oligarchs.

I think you could do most of it as a point and click. Perhaps with the exception of that one command (if you know what I mean) because the mere possibility of it would be revealing in a point-and-click. But you could do that in a Sierra AGI type graphical adventure because that still has a parser.

On topic, I would myself recommend Coloratura - https://ifdb.org/viewgame?id=g0fl99ovcrq2sqzk - for the sense of wonder/unusual protagonist.

Perhaps you could do a hierarchical approach somehow, first generating a "zoomed out" structure, then copying parts of it into an otherwise unspecified picture to fill in the details.

But perhaps plain stable diffusion wouldn't work - you might need different neural networks trained on each "zoom level" because the structure would vary: music generally isn't like fractals and doesn't have exact self-similarity.

You can do MCMC like AlphaGO and see ten moves ahead.

The existence of adversarial attacks shows that most neural networks have pretty bad worst-case performance. Thus sticking GPT-3 into alpha-beta or MCTS could just as easily give you an ungeneralizable optimum, because optimizers are by nature intended to find extreme responses. Call it a Campbell's law for neural nets.

The actual AlphaZero nets are probably more robust because they were themselves trained by MCTS, although they still don't generalize very well out-of-sample: IIRC AlphaZero is not a very strong Fischer Random player.

In the same way. Most proposed fusion systems use deuterium-tritium fusion where a significant amount of the energy is carried away as neutrons, so direct energy conversion wouldn't be possible anyway.

From the article you referenced:

ITER will not produce enough heat to produce net electricity and therefore is not equipped with turbines to generate electricity. Instead, the heat produced by the fusion reactions will be vented.

So in a fusion plant, the particle energy would turn into heat (by the particles interacting with matter), this would heat up water (or some other carrying fluid), turning a turbine that produces electricity. See also https://en.wikipedia.org/wiki/DEMOnstration_Power_Plant which contains some diagrams showing just how that would be done.

More exotic reactions (e.g. p-B11) have been proposed, where almost no energy is in the form of neutrons. Theoretically, you could then use electrostatic devices to capture the energy directly without any of the mess with Carnot efficiency. However, getting p-B11 fusion going is much harder than d-t.

What you call "american ideas" is the only thing that works in the anonymous environment.

What about BitTorrent or its various file-sharing predecessors? It has no cash, they had no cash. Or Tor? Exit nodes don't demand money as compensation from attracting the attention of people in authority.

The current crowning achievement of formal methods is, as I understand it, seL4. It is a formally proven microkernel of about 8500 LOC. There's still a while to go until they can scale to 100kLOCs, unfortunately.

There are ways to keep an AI in sealed hardware and making sure it can't affect the world, for instance by using an objective function that only deals with mathematics, and doesn't deal with the real world at all.

E.g. the AI is given a fixed amount of hardware and told to produce an algorithm that solves some NP-complete problem (say integer programming) in expected time as close to polytime as possible, as well as a mathematical proof that the algorithm satisfies the claimed close-to-polytime complexity bound. Then humanity can just solve the NP-complete problems separately once they have the algorithm.

This objective function doesn't care about the physical world -- it doesn't even know that a physical world exist -- and so it's about as likely to directly affect the physical world as MCTS or AlphaGo.

The "AI is going to run out of control" is a very compelling narrative (as everybody who has read the Sorcerer's Apprentice understands). But that doesn't make it true. Beware the availability heuristic.

(Incidentally, I think AI destroying mankind because it's too smart is an unlikely outcome. It's much easier for the AI to subvert the human-designed sensors linked to its objective function; and if the AI is sufficiently smart and the sensors aren't perfect, then it can always do so.)

That seems to be a DRM problem. Let's say that you want the camera to track all modifications of the picture. Then, analogous to DRM, there's nothing stopping the forger from just replacing the CCD array on the camera with a wire connected to a computer running GIMP.

To patch the "digital hole", it would be necessary to make the camera tamperproof, or force GIMP to run under a trusted enclave that won't do transformations without a live internet connection, or create an untamperable watermark system to place the transform metadata in the picture itself.

These are all attempted solutions to the DRM problem. And since DRM doesn't work, nor would this, I don't think.

Just make it zero-knowledge. You use the ID server to prove that you're not a sock puppet of someone already registered, but that's all the site needs to know.

That's for reinforcement learning, right? What is the adversarial learning problem in say, classification based on Solomonoff?

If hypercomputation is possible, then anything based on Kolmogorov complexity would be SOL, but if not... is Solomonoff induction just too expensive in practice?

E.g., their ability to survive high levels of radiation or vacuum.

If I recall correctly, the tardigrades (and extremophiles like D. radiodurans) have evolved to handle damage brought on by desiccation. As a fortunate side-effect, this general robustness also protects against radiation.

Nostalgia is a hell of a drug.

That's interesting that you're familiar with how it scales to different cores; I've never played around with the core parameters that much.

It's simply the combinatorial explosion: it's easier to find a surprising and good program in ten lines with pointers limited to a max value of 800, than with hundred lines and a pointer max value of 8000.

Later I discovered CoreWar [2] and enjoyed that until I learned all of the main classes of algorithms/bots had been identified.

The evolvers sometimes break the bomber-scanner-paper stereotype. They just don't scale well to normal sized cores.

I wonder if one could make a better ML system than genetic programming for creating CoreWar warriors. Perhaps a neural net connected to a differentiable SAT solver?

It's not really immutable as you can change the parameters of an entangled pair. You just can't communicate any information by doing so, because you need a classical signal to make sure you don't read one of the particles the wrong way.

If there's an infinite number of civilisations out there, and one of them is so advanced that we are insects to them, why wouldn't they just exterminate us, or use us as food?

We want to exterminate mosquitoes because they're actively detrimental. Nobody's advocating for the extermination of, say, daddy long-legs spiders even though they're everywhere; and nobody would be advocating for the extermination of mosquitoes if they were all located in Antarctica.

So it takes a very precise level of inferiority to be extermination fodder. If you're too unassuming, there's no reason and so it doesn't happen. If you're dangerous enough that you can fight back and hold your own, it doesn't happen either.

That aliens would find us just the right shade of annoying seems... implausible.

quantum communication system that would allow instantaneous information transfer over large distances.

However do they intend to accomplish that? Quantum entanglement can't be used to accelerate classical communications!

Calculus captchas do exist, which I guess proves the point!

https://nakedsecurity.sophos.com/2011/03/09/a-complicated-ca...

That sounds right.

A zero-knowledge password proof is a way for one party to prove to another the knowledge of a password, without revealing anything else about the password.

Such a protocol prevents an attacker (eavesdropper or man in the middle) from brute-forcing the password offline even if they capture the whole exchange, so insecure passwords become much less of a risk as long as the verifier rate-limits login attempts on its end.

Some of these also have the property that a malicious verifier can't fake a success unless it already knows the password, thus making password phishing pretty much pointless: the only thing a phisher can verify is whether the user uses some predetermined password, and if not, the user is immediately made aware that the site expected another password.

IIRC, the most recently developed ZKPP is OPAQUE: https://blog.cryptographyengineering.com/2018/10/19/lets-tal...

Looks like a relative of the Gosper curve! https://en.wikipedia.org/wiki/Gosper_curve

Perhaps a more general observation is that a measure that faces pressure by intelligent agents needs to be monitored and adjusted by equally intelligent agents to patch the exploits.

So you can either use a simple measure that doesn't work, or a complex measure that does work, but at potentially great cost (evaluating all the terms, bureaucratic inertia by the controllers, etc.)

Some metrics seem to provide more of a free lunch - be more robust - than others. It's not obvious which are robust, however.

You would need to use zero knowledge crypto on both sides so the state doesn't know what platforms you're subscribed to, and the platform doesn't know your real identity (or what other users you have, only that you're below the limit).

It's hard but it shouldn't be impossible.

As an optimization mechanism, failures of the market can be either one of two things:

- It optimizes the wrong thing, or

- It fails to optimize what it intends to optimize.

Externalities fit into the former category. Business cycles and some types of path dependence (e.g. Keynesian demand deficiency) fit into the latter.