Lets admit it - previous research loop was going for a google search then iterating on results, which takes time and effort. LLM based research (although could be misleading) is just much faster and more suitable for natural language.
It happened.
This doesn't mean that Google will be obsolete, if it leverages its collected data so far it will be a strong competitor. But Google isn't leading at the moment.
Great post. Long sequences are a problem I have been dealing with long enough now. Even though GRU outperforms LSTM in longer sequences, it still isn't enough and vanishing gradients problem is still experienced. Currently, one-dimensional convolution units are effective while dealing with longer time steps, but sadly it's not the precise unit people wish to use while processing longer time-steps.