ChatGPT4 writes Stan code so I don’t have to

https://statmodeling.stat.columbia.edu/2023/04/18/chatgpt4-writes-stan-code-so-i-dont-have-to/

by luu • 3 years ago

70 61 3 years ago

statmodeling.stat.columbia.edu

cs702• 3 years ago

This is a really useful review of what it's like to use LLMS for writing code. The key insight, for me, is this:

Working with ChatGPT4 for this task was like working with a good intern programmer who is rather forgetful and a bit scattered.

And the key take-away -- again, for me -- is this:

But: This has already changed the way I code. I am not sure I am ever going to write code completely by hand again! And I expect these tools to get better fast.

bobmaxup• 3 years ago

I use LLMs in much the same way, but lately I have been wondering how good I will continue to be at thinking, verifying, or communicating in code, if I stop writing as much of it. I suppose time will tell...

zarzavat• 3 years ago

I’ve been using Copilot since the beta and it does cause dependency, which can be dangerous because you have to be constantly vigilant of errors it might be introducing. It’s a good idea to turn it off sometimes and try to solve problems manually.

dmix• 3 years ago

I highly recommend any programmer seriously using ChatGPT to go through this entire guide:

https://learnprompting.org/docs/intermediate/chain_of_though...

Providing context, getting GPT to explain the steps before doing them, creating feedback loops, etc is super helpful when doing this stuff.

It's still useful to just go "write this function in x language" but anything beyond a Stack-Overflow type copy/paste script it helps to do a lot more hand-holding. The ROI is worth it.

grogenaut3• 3 years ago

do you have some examples you could share? where y9ou were successful and it was worth the ROI?

argondonor• 3 years ago

It is very nice not having to write code from scratch by hand, but I have noticed that by not doing so, my innate familiarity of the code absolutely craters. Making manual changes to the code suddenly becomes a much more tedious and less confident process, and in many cases I am not sure that it's a net benefit (at least in a situation like mine, where there is no oversight and things go straight to prod.)

zainhoda• 3 years ago

Same for me. I wonder if this is going to follow a similar pattern that we went through with C vs. Assembly. There was a period where everyone coded in assembly and then C came along but people still hand-coded certain parts of applications in assembly to make them more efficient. Then eventually when C compiler optimizations got good enough and computers got faster, few people bother with assembly.

agileAlligator4• 3 years ago

If it is not doing so already, ask it to comment the code it writes.

Worst case scenario, throw the code back into GPT and ask it to make the changes you want to make.

willbudd• 3 years ago

My take-away from your take-away is that LLMs might well take away the one part of my job that I genuinely enjoy doing. Writing code, by hand, myself. Reasoning from first principles.

This is also one of the reasons I transitioned to developing AI models myself. At least I'll be safe until they start rewriting themselves from the ground up. And at that point we'll all have bigger worries to deal with anyway.

JoshuaDavid• 3 years ago

Yeah, you should be safe for at least another few weeks :)

spaceman_2020• 3 years ago

I used to get the same results, but I've lately been using better prompts and the results are substantially better.

The key is to a) give it context, and b) give it constraints, and to do it from the very first prompt itself.

Most chatGPT failures that I've seen add context and constraints after the initial prompt, which someone sends GPT into a loop.

If you have data in a particular format, mention that in the first prompt. Also helpful to tell it to optimize the code for scalability and to use any external libraries if necessary.

These tools are essentially software programs, and like any piece of software, you need to know the right commands to get maximum utility out of them.

noobermin• 3 years ago

What a strange article. The bulk of it tells me it isn't helping you...yet you conclude it's changed the way you code?

famouswaffles5• 3 years ago

That's a strange take. He generated working code in less than half the time he did alone. What's not helpful about that ? Or must output be perfect before it's deemed helpful?

gwoolhurme3• 3 years ago

I don't think it is a strange take but can we meet in the middle... there are a lot of things where I would just write them by hand and not look at a 3rd party source too. I think where some people are coming from is extremes on both ends. It seems silly (to me) to allow this thing to take full reigns of your project vs as a helper.

chrisco2552• 3 years ago

Yeah, but he had the experience of building it in the first place, and understanding what he was doing and trying to accomplish. That's not a great speedup given that context. He already knew how to hand-code it, so coaxing the GPT and understanding when it wasn't entirely correct made all the difference. One should expect that even if they were programming a task a second time by hand, that there would be a significant speedup in the same ballpark.

chaxor• 3 years ago

This is just the type of thing everyone has to say right now to appease the 'expert software engineers' who despise the idea of writing anything that's not assembly, or the new version of this type of person that despises working with extremely high level specifications, such as these new linguistic interfaces. Stating it's not helpful is not a useful statement, unless more specific about the type of help to which is being referred.

milemi• 3 years ago

This encourages the worst habits, and will make a horrible programmer, and thinker, out of a bad one.

politician• 3 years ago

Per the article, the technique requires models with a minimum of 100B parameters.

bagels• 3 years ago

Who is Stan?

cachvico• 3 years ago

He lives here, https://mc-stan.org/

celrod• 3 years ago

Stan is a DSL for statistical modeling named after Stanislaw Ulam, the inventor of Monte Carlo methods. Stan's primary method of fitting models is via (Markov Chain) Monte Carlo. https://en.wikipedia.org/wiki/Stanislaw_Ulam

jpollock6• 3 years ago

I use the same programming prompt on both Bard and ChatGPT.

"write a function in java to decode a rfc 3588 diameter header"

They both need a lot of hand-holding. Too much to make it worth using them.

Things they get wrong...

1) They are confident about the structure, but are 100% wrong.

2) They get Java types wrong, and try to put unsigned data into the same size signed type (8 bits into a byte, 24 bits into an int).

3) They marshal data into ints in a way that is correct for C code, but wrong for Java code (missing the 0xFF mask)

4) They forget previous instructions and keep re-introducing bugs.

5) ChatGPT kept thinking that version only had information in bits 6-8.

If you know _anything_ about what you're trying to do, it's a waste of time to use a LLM. If the code has anything interesting, a LLM will write broken code.

On the plus side, Bard was quoting sources, complete with license. That led to my first pull request to hopefully fix Bard's upstream source, and let me understand what (byte[0]&0x77) << 16 does, why it works and why byte[0]<<16 doesn't. But then, the LLMs didn't help with that, my need to fix it did.

http://blog.jason.pollock.ca/2023/04/trying-to-write-code-wi...

gwoolhurme3• 3 years ago

That has been a bit of my experience with LLMs in general. If you know what you are doing before hand it's not too much of a boost, in fact it can be a bit frustrating to wrestle with it to do what you want when you can do it faster. I can see where it is amazing tooling though. Especially for a new project where you don't really have much of an idea or need a working demo.

worrycue• 3 years ago

But if you don’t know what you are doing, you have to worry about its hallucinations slipping pass you.

famouswaffles5• 3 years ago

What exactly is the point of using two models, both far from state of the art and saying, "LLMs are not useful". I mean, what's the thought process here ?

kyleyeats• 3 years ago

You'll see this in the workplace too. It's very hard to explain to the not-most-productive people that the most productive people on the team are also usually making the most mistakes. They'll always be happy to just sit back and criticize the mistakes. It's the only way they can be better.

jpollock6• 3 years ago

Sorry, my comment wasn't LLMs in general, it was this specific pair of LLMs at this specific time.

I've been trying the same prompt since ChatGPT was released, and I keep revisiting it.

They are getting better at writing code, Bard's addition of sourcing information is very important to me.

I do this so I can see if it will help _me_ in what I'm trying to do, and where the failure modes are. Their failure modes are very sneaky and very dangerous for the kind of programming I do.

I would use them to maybe give me a first pass at some code in a language I have zero familiarity with, and only to solve general problems.

I'm thinking throw-away, one-off code where the operational range is known up front and fully middle of the road. I'm thinking reports.

I might try to get it to write something useful in R, I have no knowledge of that.

I can see them as really handy refactoring tools. They take instruction pretty well. They just need to be stable in the code they produce - they keep reintroducing bugs you've instructed them to fix. Changing to an edit stream against code instead of re-generating the code might be a fix?

agileAlligator4• 3 years ago

Try using https://www.phind.com on expert mode. You will probably have better results.

jpollock6• 3 years ago

Thanks for the tip, phind is much better.

Still has the "won't fit in an int" problem, but it did use byte & 0x77 to get them into ints.

Amazingly, it got the field correct and in the right order!

It did not reference a source, but that seems to be a Bard feature right now.

iudqnolq• 3 years ago

Thanks for the tip. I put in an example that ChatGPT completely failed at and phind eventually got the right answer.

Oddly it went off on a tangent about how the problem can't be solved in Python, which I never mentioned.

But when I explicitly told it I wanted c++ it gave a decent answer.

It did, however, get some nuances seriously wrong.

Precision loss for large integers: When encoding to JSON, UInt64 and Int64 are encoded as strings of base-10 digits in ASCII, since JavaScript's numeric type only affords 53 bits of precision. This means that large integers might lose precision when converted to JSON.

https://www.phind.com/search?cache=319c7cf5-65e4-4704-a8f3-0...

bumbledraven• 3 years ago

Try GPT-4.

rrrrrrrrrrrryan• 3 years ago

Have you tried GPT-4? Here's the response I got: https://shareg.pt/Q0ZcWCX

Is it any better? (Your prompt is related to a completely different domain than the one I work in.)

jpollock6• 3 years ago

Nifty, thanks!

The code has the standard "you can't fit 24 bit or 32 bit unsigned values in an int" bug. I get the feeling this is a common bug. Probably insufficient testing in the hobby implementations used as training data.

My reading of "int length = first4Bytes & 0xFFFFFF" is that it will truncate values incorrectly. I'm pretty sure that "-1 & 0xFFFFFFF != 0xFFFFFFF". It goes (-1 & 0xFFFFFFF) is a long, which is then downcast back to int, truncating it?

commandCode is wrong, GPT-4 is using 28 bits instead of 24.

I _think_ the command code flag decoding is wrong, but it's been a while since I've done bit fiddling. The spec says Request is bit 0, but the code has it in 0x80 - bit 8?

That's much improved from ChatGPT and Bard. The use of the ByteBuffer avoids many of the problems. I've seen that in some GitHub Diameter decoders.

The RFC is here: https://www.ietf.org/rfc/rfc3588.txt - Page 31 for the header.

davidjones3322• 3 years ago

The guy who was trying to trash ChatGPT (gpt4) forgot that anyone can just check his example and post a screenshot I think.

captnObvious• 3 years ago

I’ve used GPT-4 as a pair programmer actively for over 40 hours now. Using it to do things you have 90% of the solution for is a bit of a waste of time.

Using it to help trouble shoot errors in logs when you’re feeling frustrated, and get a basic framework up from scratch to start modifying toward the your goal are the best use cases I’ve seen. The ability to get yourself unstuck in minutes is life changing. The biggest constraint right now is short term memory loss. It can nearly always recognize on a second pass through what mistake it made.

Keep in mind 32kb token requests are already in beta… and 4kb requests are what you have access to now. So… the memory thing is potentially a dead point soon. Though at 6 cents per 1kb if it actually has to volley 32k over to you it’ll almost cost $2.00 based on current API pricing.

That 32k option would allow GPT-4 to hold every non-monolithic project I’ve ever worked on in working memory. The pertinent parts anyway, it wouldn’t need to read CSS files or imported modules unless it really gets forced down a specific rabbit hole.

This will be one more reason new projects won’t be monolithic architectures. Easier for the AI to help out.

w4ffl35• 3 years ago

It's also good for putting together boiler plate classes and some other things. I had it code most of klotski for example and have used it pretty heavily in this project (and I think that is evident in the code)

https://github.com/Capsize-Games/airunner

coder543• 3 years ago

$2/request seems a little ridiculous. I checked the pricing and you’re not wrong… I just feel like that kind of pricing will apply even more industry pressure to develop competition. Decent open source options would be nice too.

zippyman55• 3 years ago

I used GPT to generate some RStan code last month and was very impressed with the initial starting point it gave me. Similar work w LSTM-RNN models. But the number of times it confidentially gives me output and I respond: “are you sure?” and it completely changes its answer. Buyer beware!!

gtirloni• 3 years ago

<ChatGPT> I reviewed the Stan code again and realized

What kind of data structure permits it to keep that code in particular to be reviewed? Is that part of the model? Or is this phrase a hallucination?

famouswaffles5• 3 years ago

Every new chat message is the result of a new pass of all the text so far through the model. Mistakes made can be spotted on a new pass.

causality02• 3 years ago

I'd like to know if there's been any progress in the "memory" area of GPT. As far as I know, when you tell GPT to do something it runs the entire sequence of prompts over again from the beginning. This is obviously not practical for any applications that require ongoing collaboration, like using GPT integrated with your computer to execute commands and policies, or training GPT to be a better coding partner.

grogenaut3• 3 years ago

Why is it obviously not practical? Seems to be working in many places. Or is your issue that the memory is too small?

causality02• 3 years ago

If it has to re-run the entire history every time you interact with it it will quickly slow to non-functionality in days.

Barrin92• 3 years ago

The funny thing is I've had the exact same experience as the author but come to the completely opposite conclusion. Working with ChatGPT as a daily driver isn't for me precisely because of this:

I have had to be very attentive because it sometimes produces code that runs but does not do what I asked

It's like someone putting sand on your toothbrush and in your shoes. The subtle way it messes up are in many ways worse and require more attention than dumber tools. It's even worse than humans because the ways in which it fabricates nonsense aren't the kind of mistakes a human makes, it's not like you're coding with an intern but like you're coding with ALF.

The constant context switching between "it's nice" and "it's sabotaging random things" is so taxing if you don't want to mess up. It's like self-driving that way. An automated system that works 95% is worse than one that works 20% of the time.

jimbokun• 3 years ago

I don't understand the obsession with having Chat GPT4 spit out code for problems like this, where clearly the desired goal would be to have it run the model and display a graph or chart or closed form solution. Like Tony Stark asking Jarvis to run some simulations or calculations and show the solutions in an intuitive way.

Is anyone working on something like this? I would be surprised if Wolfram wasn't working on it.

m3kw9• 3 years ago

ChatGPT is also great at teaching you code. Paste a piece of code you don’t understand, it will explain giving you try it out to make sure. It’s faster scanning docs and on exact match stack overflow answers.

At the end of the day you need to know coding to use ChatGPT to help you code, it’s a loop

awinter-py• 3 years ago

LLM -> DSL codegen is going to be powerful

in many cases DSLs do the reasoning that LLMs cannot

(though in this case the author works at the place where stan was invented and is probably doing really good prompting)

(also to the extent LLMs let us solve everything with boilerplate generation, that's horrible and I weep for the youth)

anabis• 3 years ago

I too find ChatGPT useful for programming, but:

All told, this took about 2.5 hours. And all I did was end up with a model very similar to one I wrote by hand several months ago. But that one took me something like six hours! So 2.5 hours is a huge speedup.

It's likely that the attempt being the second time explains 80% of the time reduction. Design and implementation is a learning process, and you being it through once will speed it up massively.

poulpy123• 3 years ago

one thing I found bing AI and phind[1] very useful for was to get a short tutorial of libraries/tools. Google became so bad with it that since several months I was appending "reddit" or worse "youtube"[2] on my search, and now I just ask "how to do x with y" and I get a pretty good answer.

[1] I don't know what is used behind the curtains [2] worse because I much prefer to read a text than to watch a video

satisfice• 3 years ago

The author is playing with fire.

I also have used ChatGPT for coding and it hallucinated APIs, twice now, that sent me researching down a rabbit hole.

When I have asked it to do things I know how to do and how to evaluate, it has sometimes given very good answers. Other times it misunderstood the problem in a subtle way— and if I hadn’t known enough I would have accepted a program that performed the wrong analysis.

If you have to accept the quality of the solution on faith you are playing russian roullette with your reputation.

I asked a student to analyze the randomness of set of data. He got a program from ChatGPT without knowing anything about how statistical testing for randomness works. He thus accepted a program from the bot that was hilariously unreliable.

ChatGPT4 writes Stan code so I don’t have to

Related Stories