Building a ChatGPT-enhanced Python REPL

https://isthisit.nz/posts/2023/building-a-chat-gpt-enhanced-python-repl/

by synergy20 • 3 years ago

103 42 3 years ago

isthisit.nz

asah• 3 years ago

Cool!

Neptyne spreadsheet has a GPT-enhanced python repl as well: https://neptyne.com/

wpietri7• 3 years ago

I am a big casual user of anthropomorphic language for what computers are up to. I do it not because it's true, but because it's more engaging to say that "the computer is confused about X and so it's mad". But I'm wondering if that's dangerous when it comes to things that produce pseudo-human output. E.g.:

I think that the LLM thinks that it cannot generate code for the simple task. Unlike every other computer API in existence we can prompt the LLM to tell us why it responded in the way it does.

I agree the LLM is going to produce text that is going to be narratively consistent with the assumptions of the question. But we already know LLMs are giant bullshitters. Is there any reason to think they are doing actual introspection and description of internal state? Or are they just going to give plausible-sounding words?

isthisit• 3 years ago

Author here. Fully agree.

I agree the LLM is going to produce text that is going to be narratively consistent with the assumptions of the question. But we already know LLMs are giant bullshitters. Is there any reason to think they are doing actual introspection and description of internal state? Or are they just going to give plausible-sounding words?

This sounds like something we could test by inverting the LLM-provided reason of why it can/cannot do something and playing that back through the model.

The example I gave in the blog was, unprompted, the LLM thought it should not generate code that it thought had already been executed. If we take it at face value, I could add a line in the code generation prompt to instruct it to generate code even if it "thinks" it has already executed it. We then assert the outcome.

Not very scientific, but might give us some insight when trialed across thousands of prompts.

vba616• 3 years ago

Is there any reason to think they are doing actual introspection and description of internal state? Or are they just going to give plausible-sounding words?

Someone needs to call out the double standard applied to AI.

If machines that bullshit are philosophically indistinguishable from intelligent humans, how is it that humans that bullshit are distinguishable from intelligent humans?

wpietri7• 3 years ago

Sorry, I'm not following your point.

randmeerkat• 3 years ago

Someone needs to call out the double standard applied to AI.
If machines that bullshit are philosophically indistinguishable from intelligent humans, how is it that humans that bullshit are distinguishable from intelligent humans?

Finally someone gets it. A brand new state of the art AI was trained on the entire digital corpus of human knowledge. Said AI then sometimes bullshits people, and people are surprised..?

Scarblac• 3 years ago

It depends on what the goal is. If the goal is to make human-like AI, then that's a valid criticism. But if we want AI to be a useful tool, then bullshit isn't helpful.

Why hold AIs to the same standards as humans. We already have humans, and can't really change them. Let's apply standards to AIs that make them useful tools.

famouswaffles4• 3 years ago

humans don't always give faithful reasoning either just so you know. even removing any willful obfuscation, we simply don't know all the biases that can influence a certain action. even more than that, Your brain will happily make up a reason for an action. This is demonstrated in the split brain experiments. Knowing that, if what it says in response to a given problem can meaningfully change output for the better following its directive later then what else ? might as well go with it

wpietri7• 3 years ago

You could make exactly the same argument for consulting psychics, astrologers, and tarot readers.

Indeed, I think what a lot of people are doing with LLM output is the same thing customers of paranormal scams do. Faced with artfully constructed ambiguity, they fill in the blanks themselves, not even noticing how much they're supplying the work. Some bit go well and some poorly; the successes are proof that the psychic has paranormal powers, and the rest are quickly forgotten about.

dragonwriter• 3 years ago

If machines that bullshit are philosophically indistinguishable from intelligent humans, how is it that humans that bullshit are distinguishable from intelligent humans?

Well, if we use the ReAct model, the flow of action involves the LLM narrating its reasoning and then acting on it. Which can still be bullshit, of course, but its at least a different thing than a post hoc rationalization when questioned. (Of course, we know that when questioned about past actions, humans often construct post hoc rationalizations, too.)

drvdevd• 3 years ago

Perhaps this is one of the greatest paradigm shifts for traditional software engineers as “AI” becomes more commonly integrated: maybe we will have to use anthropomorphic language and maybe it will be considered the most technically correct way of talking about these systems due not only to their non-deterministic nature but also their emergent properties.

(This was a great post btw - highly recommend if you’re just perusing the comments)

usrbinbash3• 3 years ago

Meeeeeehhhh...I don't think that is likely to happen any time soon. Could be wrong, but I don't think it's likely.

The reason being: All software engineers are trained to read and write code, aka. precise formal languages. All feedback from systems about errors in that are also precise and formal languages.

What will likely happen, is that we will use anthropomorphic language more and more to generate code, especially code that is easy to describe but just a boring hassle to type (aka. boilerplate or things like simple unit tests).

bheadmaster3• 3 years ago

Is there any reason to think humans are doing actual introspection and description of internal state? Or are they just giving plausible-sounding words?

spherebroken2• 3 years ago

People with separated brain hemispheres, for example sight and speak capabilities, try to make plausible reasoning. There are some well known physiological experiments to prove this. This proof that our brain try to provide a plausible explanation that has nothing to do with what is happening in reality.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7305066/

(1)

spherebroken2• 3 years ago

People with brain-split separated brain hemispheres, for example sight and speak capabilities, try to make plausible reasoning that are not grounded. Here is an example From (1) about brain-splitted subject called Joe: In a particularly dramatic recorded demonstration, the famous patient “Joe” was able to draw a cowboy hat with his left hand in response to the word “Texas” presented in his left visual half field. His commentary (produced by the verbal left hemisphere) showed a complete absence of insight into why his left hand had drawn this cowboy hat.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7305066/

(1)

simonw• 3 years ago

Asking LLMs why they made a specific decision is a very common misconception.

The state that lead to their previous generation was GBs of floating point calculations which were discarded as soon as the output had been generated.

If you ask them why they said something they will generate a brand new explanation based on the previous text they generated, but any relationship to their actual decision process will be entirely coincidental.

Update: I just noticed that the article uses this:

If you cannot return executable python code return set the reason why in the description and return no code.

That DOES work - getting it to generate justification during a single response isn't affected by the lost state issue I'm describing here.

heisenzombie• 3 years ago

People also can’t really recreate an earlier mental state, and have to resort to post-hoc rationalisation. E.g. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3196841/

I feel like the answer to the question “give me a possible reason that a person/llm would give this answer given this situation” is all that’s ever really possible!

IanCal• 3 years ago

Hang on I think you're assuming there's some internal state that I don't think exists.

Each time it just generates one more token until it hits a stop token, using the previous input. It shouldn't matter if it created a token or you did.

OrderlyTiamat2• 3 years ago

You are right on fhe first point, I believe the second needs to be corrected. LLMs have no state. Each word- each token- is made anew, the network being oblivious to the process that led to the previous token. It cannot even distinguish between text it generated and text you added unless you tell it that. So any relationship between output and explanation is 'coincidental'. Actually, any relationship between two tokens next to each other is 'coincidental' in the exact same way, so it cannot explain its reasoning. If you prompt it to comply or explain why not, the network may generate an explanation of why it can't comply, but it is doing that simply as a good continuation of the input text.

I'm not saying you can't call that an explanation if you want to use that word, I'm trying to make a distinction between the generated output and an explanation of that output. All of this is the former. The latter is not possible unless you analyse the model as it is running, and we don't really have a good understanding of that process yet.

There was a cool paper where researchers analysed a finetuned model playing othello, and they found some neurons that clearly represented the game state- if they changed those neuron values, the network responded correctly, even if that game position was impossible to get to normally [1]

I'd argue work like that is getting us closer to getting explanations for llm behaviour.

[1] https://www.joyk.com/dig/detail/1674380676520322

lynx23• 3 years ago

I am pretty convinced a large majority of humans don't do anything differently, they also just generate "plausible-sounding words".

westurner• 3 years ago

"Show HN: IPython-GPT, a Jupyter/IPython Interface to Chat GPT" https://news.ycombinator.com/item?id=35580959 lists a few REPLs

alfons_foobar• 3 years ago

As the technology evolves I see there being a greater demand for deterministic responses from LLMs.

I think we will see demand for it, but I also think it completely misses the nature of LLMs / for what they are most useful.

Basically trying to shoehorn "old" patterns into a new paradigm.

kgeist• 3 years ago

What if LLM hallucinates some destructive operation, such as deleting local files, and runs it? I think the project should automatically run everything in a container, just in case.

9dev• 3 years ago

Yeah, that's a shared problem among these "let an algorithm type commands for me" projects. What I would thoroughly enjoy, however, would be ChatGPT-assisted autocomplete and suggestions in the shell. Somewhat similar to Copilot, but for the terminal, suggesting commands I might use next, directories I could want to navigate to, and so on. Maybe with a key binding to cycle through different suggestions.

Is there something like that yet?

tomviner• 3 years ago

Copilot CLI is exactly this https://www.npmjs.com/package/@githubnext/github-copilot-cli

ogogmad• 3 years ago

This might combine well with Sympy, especially to help with the var('x y z') nonsense. I think I'll install this.

Building a ChatGPT-enhanced Python REPL

Related Stories