kalavan

Also called the door-in-the-face method.

Sounds like classic crowding-out of intrinsic motivation.

There's a story, I can't find the page at the moment, of someone who was getting pranked all the time (his house TPed or egged or something). So he offered the miscreants $1 to do it tomorrow. He kept on doing it like this, and then a few days later, he offered a quarter. By the time he had got down to a dime, they said "there's no way we're going to do it for such a measly sum" and left.

Better sourced examples also exist: fewer citizens supported a decision to build a nuclear waste repository in Switzerland enjoyed more support if they would be offered compensation: https://www.bsfrey.ch/wp-content/uploads/2021/08/crowding-ef... p. 96 (sixth page of the PDF).

Human lockpickers use feedback when picking. I'm wondering if a bot could do the same - e.g. measuring the travel distance to find a binding pin, or the resistance to moving the wire?

It's surprising that series and movies with gazillion-dollar budgets don't seem to have money for decent writers. About the only explanation I can think of is that the way the series or movie is made itself makes story too hard to do.

E.g. an action movie is designed around its stunts and then the plot is stitched together to support them. And series that are made one episode at a time can suffer from serious plot drift when they aren't planned ahead properly, or when executives can't decide whether they're going to have one more season or not.

And the idea of spammers using bot nets (therefore not paying for computer themselves) would be less relevant to LLM scraping.

It's possible that the services that reward users for running proxies (or are bundled with mobile apps with a notice buried in the license) would also start rewarding/hiding compute services as well. There's currently no money in it because proof-of-work is so rare, but if it changes, their strategy might too.

There's this paper from 2004: "Proof-of-Work Proves Not to Work": https://www.cl.cam.ac.uk/~rnc1/proofwork.pdf

The conclusion back then was that it's impossible to make a threshold that is both low enough and high enough.

You need some other mechanism that can distinguish bad traffic from good (even if imperfectly), and then adjust the threshold based on it. See, for instance, "Proof of Work can Work": https://sites.cs.ucsb.edu/~rich/class/cs293b-cloud/papers/lu...

That then raises the question: what is a unit of communication?

If communication is 20% verbal and 80% nonverbal, and if communication is very nonlinear in understanding (as with your book example), how do we know what 1% of communication is? What does it mean, and how can we tell that the figure is correct, when our main or only way of detecting whether communication succeeded is through understanding or lack thereof?

It's more likely a reference to France currently being the Fifth Republic.[1] The transition from the Fourth to the Fifth happened in 1958 without much violence.

[1] https://thegoodlifefrance.com/short-history-of-the-five-repu...

It's probably getting amplified by the RLHF stage because the earlier models didn't do that.

But that just shifts the question to "what kind of reviewer actually likes 'it's not just X' cliche?" I have no idea.

It might be that democratic countries are more resilient to that kind of effect because (and to the degree that) they already decouple productive power from representation.

E.g. a welfare state doesn't make sense from a purely GDP-selfish perspective, beyond as a crime-prevention tool, since people on disability benefits don't work. But they still exist.

Doesn't Norway bring that conclusion in doubt? The state gets massive revenue from oil as well as oil-financed investments, but is still very much a democracy.[0]

[0] https://ourworldindata.org/grapher/democracy-index-eiu?tab=t...

Every other field is aligned "aligned" when the humans in it are "aligned",

That doesn't seem like the whole story. Pick two countries, for instance, one of which has evolved to be democratic (with high regard for rule of law, etc.) and the other is dictatorial. How did these countries end up the way they did? It probably has to do with rules, not just default human qualities.

Let's say you consider popular participation to be good. Then you could say the humans who live in the first country are more "aligned" than the second, but the mechanisms of their forms of government also play part. E.g. if the bureaucracy is set up so that skillfully stabbing others in the back gets you political clout, the selection process will marginalize or kick out people who don't want to engage in backstabbing.

Any organization's behavior depends on some combination of what its incentives promote and on the qualities of its members. This makes AI alignment just an extreme on a scale, not a thing set apart from all other kinds of alignment. The AI alignment problem is the "all rules" extreme of the scale, and organizational alignment is some combination of rules and the inclinations of the humans who are part of it.

The ethics problem of "what does 'aligned' mean anyway" would both apply to the AI situation and the mixed organization situation. A dictator might want an AI "aligned" to maximize his own power, and would also want a human organization to be engineered in such a way as to be both obedient and effective. Someone of a more democratic predisposition would have other priorities - whether they are of what AIs should do or what human organizations should do.

I just have to find a way to deal with all this prisoner's honey first.

It's considered mysterious because of the hard problem of consciousness. Describing a mechanism that could be considered analogous to consciousness "from the outside" is pretty easy (just do self reference).

But the subjective quality of "what is it like to be X" is not easily captured by such descriptions - not unless you make some kind of panpsychist assumption that everything that has self-reference is subjectively conscious.

That said, a number of materialists say that there's no there there, and thus no problem. We're just all deluding ourselves into thinking that we have subjective experience. I don't think that argument is very strong, but it is made, and could explain why some find the whole business of consciousness seemingly trivial while others consider the hard problem to be very hard.

More information can be found at https://iep.utm.edu/hard-problem-of-conciousness/