Recent paper highlights the difficulty of creating a new optimizer as drop-in replacement. Sophia and Lion were recently proposed as superior alternatives to Adam, but appeared worse in an independent eval
I don't need the anonymous version for this; I've already talked with a mental health professional two times on call. Both times, I changed my name. It works for me.