But does this mean that a 99.99% reliable LLM would turn us back into the mode of us building it again? I would not say so.
I think for tasks that are about decisions, having the LLM make decisions is what makes it feel like the LLM did something for me.
Consider mowing the lawn. Imagine I had a lawnmower robot that does the mowing all on its own. Despite perfect accuracy, I didn't mow the lawn; it did. If I sit on that lawnmower the whole time and start driving it instead of letting it go on its own, then I mowed the lawn. Even if I stand there and control it with a joystick, I still mowed the lawn. Ownership comes from the decisions about where to mow.