The Transformer Blueprint

https://deeprevision.github.io/posts/001-transformer/

by nyandwi • 3 years ago

99 6 3 years ago

deeprevision.github.io

selimthegrim• 3 years ago

Some basic spellcheck would be helpful (it’s in wrong place, “parallize” missing el etc)

nyandwi3• 3 years ago

Thanks for such helpful feedback. Will ensure that is done next time!

lyapunova• 3 years ago

Wow this has a lot of great distillation in it. I am always impressed with CMU output.

nyandwi3• 3 years ago

Thank you. Really happy to hear that!

fnordpiglet• 3 years ago

Great write up. There were a few grammatical errors - I’d suggest piping it through a LLM prompted to find grammar errors :-)

Another area of interesting research is the use of transformer like models for highly dimensional time series prediction. While language and vision are interesting and have their uses, my opinion is the application of these techniques for multidimensional non linear effects in time series may ultimately have more broad and significant impact. Ex:

https://github.com/Thinklab-SJTU/Crossformer

nyandwi3• 3 years ago

Thank you for the great feedback. I will correct grammar.

Admittedly, I am biased to ward visual language and close modalities, and that made miss works happening elsewhere like time series. I will check that out and can potentially add a pointer to the paper you shared. Thanks a bunch :-)

The Transformer Blueprint

Related Stories