It's Starting to Get Strange (GPT 4 as a "first rate data analyst")

https://twitter.com/emollick/status/1653382262799384576

by byt143 • 3 years ago

21 2 3 years ago

The author notes that in the data analysis, ChatGPT hallucinates an incorrect fact…there seems like such a huge gulf between “ChatGPT can do 90% of a data analysis but some might be wrong” and “ChatGPT can do data analysis with great accuracy.”

It’s a difference between proof of concept and production ready software. Maybe these shortcomings can be addressed in a larger software ecosystem, but ChatGPT still feels pretty far from properly replacing human work—it’s just not reliable enough

jiggawatts• 3 years ago

I tell my coworkers that ChatGPT is useful whenever you can verify the output. Don’t take its advice on topics you don’t understand yourself.

For example, I ask it to solve simple maths problems, then ask it to output the solution in Wolfram Language format so I can numerically verify it in Mathematica.

Similarly, I ask it to write simple scripts I could write myself but can’t be bothered to. I can understand the script and verify that it compiles and runs, so this is safe.

I don’t ask it for a step-by-step guide for brain surgery.

It's Starting to Get Strange (GPT 4 as a "first rate data analyst")

Related Stories