The state of "Scientific AI". I know LLMs are accelerating at doing computer work, and I've experienced the acceleration of using LLMs to do science, but it's more along the lines of debugging and stitching together pipelines of classical in silico tools. I see a ton of value here along that entire pathway for LLMs, it's basically digital process development: take each step, make it better, repeat, repeat, repeat. But the ceiling are the unit operations themselves, which sure LLMs can improve the code of those tools next. But if science was held back by simply people not doing the same things faster, maybe this will really push us forward, but I have a nagging feeling of "is this it?".
I think what I'm really looking for is a model like GPT Rosalind, which has been steeped in "science" post-training, but with more randomness. Like, I think I'm looking for a GPT Mullis--Rosalind Franklin was careful, deliberate, and serious; Kary Mullis ate a bunch of acid, drove on the PCH, and invented PCR. Like, we need to invent psychedelics for LLMs. Some way to let them relax their weights, explore new pathways, and come up with some absurd ideas by connecting random ass dots from the natural world. I want the model to say, "hmm, that's weird". This isn't just changing the temperature, we're missing something deeper.
I think frontier science has always come from serendipity: a bright thinker, listening to a presentation after having stared at some small experiment, but having trained for years on "biochemistry" so the foundation and loose guardrails are there.
I don't know, I'm feeling adrift. Does this resonate with anyone else?