Nobel Prize in Physics Awarded for Machine Learning and Neural Networks 2 years ago
The story is that it is because Alfred Nobel was cheated on by a mathematician (hilarious, but unconfirmed)
HN user
lukasga
16 karma
Posts0
Comments13
No posts found.
GPT-4o Contributions 2 years ago
What does "Data flywheel" refer to here? Is it the continuous and immediate processing of user input while you're still speaking?
Maybe a bit outdated now, but reminds me of LSTMs from the recurrent update of a memory / hidden state with gating. I remember one of the biggest problems with such RNNs being vanishing gradients as a result of the long context, which vanilla transformers presumably avoided by parallellizing over the context instead of processing them individually. I wonder how this is avoided here?
Love the site, but the one thing I am missing is instantly seeing what papers are trending right now, instead of having to manually select a timeframe. Think hacker news frontpage. Would be interesting to add a "hot" filter, or similar. Average weighting by exponential decay over time? Not sure how you usually do that with pagerank.
Spotify recently open sourced Voyager (continuation of Annoy), which uses HNSW. https://github.com/spotify/voyager
Can relate to this problem a lot. I have considered starting using a Docker dev container and making a base image for shared dependencies which I then can customize in a dockerfile for each new project, not sure if there's a better alternative though.
Original paper: https://arxiv.org/abs/2310.06825
More digestible announcement: https://mistral.ai/news/announcing-mistral-7b/
I saw that Spotify recently released a similar feature for selected podcasts, could anyone recommend some recent research papers related to voice translation? Would be really interested in reading!
This study was done at my uni, and the legitimacy and rigor of it was questioned and investigated by the university. However, it was ultimately not deemed to be inappropriate. Some interesting info is that the single picture that was shown to the "jury" of a student was handpicked by the author from Facebook, and that the jury consisted of high school students.
More info: https://www.svt.se/nyheter/lokalt/skane/snygghetsstudien-ar-...
Fascinating article, I feel like quanta always puts out high quality stuff. I don't know if I misunderstood though, but the article mentions:
"In subsequent steps of the algorithm, should it pick parameters that we have already picked before in earlier steps, or should we exclude those?"
Isn't it the training examples the two different types of sampling are considered for, instead of the parameters of the model?
I'm doing my thesis right now in diffusion models (for audio) and have experienced a lot of the same things you mention. One paper which I found illuminating was this one: https://proceedings.mlr.press/v139/nichol21a.html
Particularly relevant to the noisy training you mentioned earlier is their alternative timestep sampling procedure they propose which seem to reduce gradient noise significantly judging from their experiments. Would love to hear or discuss if you have found any other design changes which have improved training / sample qualities :)
I wonder which specific ML models they're using, and if it's something along the lines of Whisper