HN user

ash-ishh

10 karma

Generalist Engineer

Posts5
Comments9
View on HN

Checkout this tweet https://twitter.com/ylecun/status/1706545305762582580 by Yan LeCun.

"Parameters are coefficients inside the model that are adjusted by the training procedure. The dataset is what you train the model on. Language models are trained with tokens that are subword units (e.g. prefix, root, suffix)."

His comment on GPT-4 parameters count

"Also: a model with more parameters is not necessarily better. It's generally more expensive to run and requires more RAM than a single GPU card can have. GPT-4 is rumored to be a "mixture of experts", i.e. a neural net consisting of multiple specialized modules, only one of which is run on any particular prompt. So the effective number of parameters used at any one time is smaller than the total number."

Interesting!

Here are some features that I will find useful.

1 - Option to customize the model parameters.

2 - Selective model comparison, once you extend the supported models it can be inefficient to run all the models all the time, selective execution can give more control to users to execute only required comparison models.

3 - Share option.