A very interesting article, suzyahyah. I especially appreciated your definition of stationarity, a concept with which I struggled in my own time series class. If I understand correctly, it sounds like the basic premise is that a fundamentally statistical methodology (LLMs) can't realistically predict a non-stationary data generation, which makes sense.
Separately, I've wondered for some time if there might be some reliable way to predict non-stationary data. While I don't have the answer, it occurs to me that it will possibly be a non-statistical method due to the fundamental incompatibilities. However, it also occurs to me that, given enough information, every data-generating process actually could be predicted. For instance, in the stock example, if you could model every single input into the system of a single company's stock, including every variable affecting every human that might conduct a transaction of it (daunting and unrealistic as that might be, but this is a thought experiment), then I believe the problem of prediction stops being non-stationary and in fact becomes completely deterministic, if complex. In such a scenario, wouldn't you be able to accurately make your prediction? I believe that perhaps chaos theory could present us with some solutions here where pure statistics (or, rather, simple statistics) cannot.
Just my 2 cents..