HN user

imustachyou

201 karma
Posts6
Comments9
View on HN

I’m missing something, won’t the input to the llm necessarily be plaintext? And the output too? Then, as long as the llm has logs, the real input by users will be available somewhere in their servers

Very fun! Would love to see a tier list to start checking off myself, and maybe a caption for the pictures

S4 and its class of state-space models are an impressive mathematical and signal-processing innovation, and I thought it was awesome how they destroyed previous baselines for long-range tasks.

Have there been any state-space models adapted for arbitrary text generation?

Language models like ChatGPT are trained to predict new words based on the previous ones and are excellent for generation, a harder task than translation or classification. I'm doubtful about the adaptability of text models that deal with fixed-sized input/outputs and don't have an architecture that is as natural for generating indefinitely long sequences.