sendes

I think the "living community" thing is the answer to this. It' ecology.

I agree. A body of knowledge, mathematical knowledge being one of them, is a give-and-take between its producers and consumers; a market for ideas. It grows in that ecology of people with its pathfinders, specialists, generalists, historians, educators, etc. Committing to a body of knowledge is becoming part of its living culture.

Where I disagree: I believe some of the loss is inevitable. Keeping in mind the example of a body of knowledge above, as the scale of what's accumulated until now grows, the role each of us play in the sustenance of its culture shrink. This is a direct consequence of the modern developmental process (ie division of labour to the point where it feels like we are all modular parts of a much larger whole).

I can't say whether its better to focus on recovering what's lost, or, trust the process, as it were.

This is an already apparent problem in academia, though not for the reasons the article suggests.

It is not so much that the "tells" of a poor quality work are vanishing, but that even careful scrutiny of a work done with AI is going to become too costly to be done only by humans. One only has so much time to read while, say, in economics journals, the appendices extend to hundreds of pages.

Would love to hear if other fields' journals are experiencing a similar pressure in not only at the extensive margin (no of new submission) but the intensive margin (effort needed to check each work).

I'd say most people most of the time outsourced actual thinking to someone else.

Someone else being human, until now. That may change. That's the whole point!

But I concur with your general point on the upstream production of thinking and knowledge. Indeed, such elite thinkers are those in economic history referred to as the "upper-tail human capital". Terence Tao being one of them giving license to the kind of thinking that accepts AI as a simple tool that is not fundamentally breaking our relationship with technology is what exactly I am protesting.

But it's equally true that those people who weren't doing much thinking due to access or language barriers can actually start doing some thinking now with the help of AI.

If only we keep thinking that thinking is a comparative advantage of our species, I suppose!

A common error in historical thinking tends to see human tools essentially as a positive linear plot between time and progress. But these tools until AI had the common property of being enhancing of human cognition, because they couldn't do the thinking _for you_. AI can do just that, and for all the benefit it brings, seeing it simply as the next step in the "natural evolution of human tools" is alarmingly disarming coming from frontier thinkers.

We assert that artificial intelligence is a natural evolution of human tools.

While nowhere in the paper this is actually asserted but the abstract, a whiggish narrative of a genuinely unprecedented technology --such that it can replace and supersede human "labour" altogether (one is reminded of The Evolution of Human Science by Ted Chiang)-- sounds naive at best, dangerous at worst.

Or, as the author's preferred title, The Evolution of Human Science.

its becoming very likely the user is becoming delusional or may engage in dangerous behavior.

Talking to AI might be the very thing that keeps those tendencies below the threshold of dangerous. Simply flagging long conversations would not be a way to deal with these problems, but AI learning how to talk to such users may be.

As an avid coffee-lover, I am sympathetic to the efforts to make coffee-brewing more of a science than an art. After all, it seems that doing so improves science just as much! https://today.ucsd.edu/story/coffee-and-turbulence

That said, I am less sympathetic to the concept of a "perfect cup", which seems to make coffee an exclusively competitive endeavour. I mean, why not enjoy coffee-brewing as one might enjoy thinkering?

I see the hyperbole is the point, but surely what these machines do is to literally predict? The entire prompt engineering endeavour is to get them to predict better and more precisely. Of course, these are not perfect solutions - they are stochastic after all, just not unpredictably.

Around May, Altman said to FT that his job was the "most important job maybe in history" (FT: https://www.ft.com/content/a3d65804-1cf3-4d67-ac79-9b78a10b6...). He did come back from brink of death before as well. But steering OpenAI into an "ecosystem" rather than a focusing on the product when you are up against the likes of Google? Seems like cashing in on the hype too early.

True, but the horses' population started (slightly) rising again when they went from economic tools to recreational tools for humans. What will happen to humans?