ottaborra

I think the same could be said of anything resembling technical writing. As an example aside from code writing, I think more than half of the machine learning papers out there are horribly written in the sense they rush a point or give no rhyme or reason for certain parts

And the best part, most people shallow read all of them and decide the details are needless till they are forced to deal with the details and then their understanding falls apart in front of them

Emotion management. I think this is more subtle than having a stoic front to everything. There are places/times where showing bare emotions moves the needle forward for example inspiring people, driving home a passionate point and sometimes in conflict, yes conflict, there are people who only understand emotion like anger to see the errors of their ways, for these folks reason doesn't work

And in other places/times, gulping down your emotions and being stoic is all that matters. Also no one likes a person who is inert all the time, so there's also prepping for that.

I know I can't really argue with advice but I feel intrinsic motivation is dependent on your upbringing, random factors so I have to say this rubs me the wrong way but is somewhat sound

I think that is fine advice. I wish it be ground down more as the opposite of what is comfortable encompasses a huge spectra of discomfort.

When in doubt save money. I have a friend who does the opposite. My friend of misery always regrets it and never learns

Fair heuristic. AA lot of people report it (gut feeling) doesn't scale as much as they'd like

I feel this list is missing sitting down to write stuff. It doesn't have to be journaling, it could be writing essays or whatnots

Computer scientists might argue that all programming languages are Turing complete anyway and the choice of which one to use is not that important

Efficiency and the real world will eat them(comp scientists who think like this) up for breakfast

Linguistic relativity seems to be a thing. Because of the differences I would think it could be inferred that an ordering amongst languages exists. In the least a partial ordering exists, so maybe some "specific" languages that are better exists

Agentic frameworks are nothing but somewhat impressive glue engineering and hundred percent will be passed over when models with bigger context lengths and better reasoning come about

Rule of thumb: if business folks drop a AI term that becomes the vogue lingo to use, be assured it is a fad that will meet it's doom in days to come

I don't think the bitter lesson is an absolute, but the current investment in agents is what you get when you ignore the bitter lesson

Something I've learnt the hard way. While there is merit to be able to do things without model assistance, this is a tale as old as time: purists who resist model usage will in the end be the unintentional enforcers of a culture of: everybody uses AI but the ones who come on top are people who pretend otherwise/who can best hide the fact that they are using them

You're probably aware of things like hyping up your resume, hyping up your stories and the existence of people on the other side who know of this dog and pony show and continue to play along? What you're going through in my opinion is the AI age analogue of that: everyone probably uses them but the people who come on top are the people who are able to pretend they don't

RNN with extra steps?

I don't understand. If kaggle solutions were able to do those, what the hell do these mean?

https://arcprize.org/2024-results

Given how o3 cracked the arc bench and I'm probably sounding like a broken record, this isn't as farfetched as some of you may think it is. ML models will very likely continue to scale regardless of how many bets are placed against it. I'm not sure why a lot of people aren't concerned about arc bench being cracked so fast. Our grand delusions of specialness has been shown to just that, delusions

"Humanity is a just a small step in the giant staircase of intelligence" - Geoffrey Hinton

you're saying you can't understand why you should be happy to be here?

I think you miss the aspect of how insignificant existence is locally.

Sure the odds are astronomical but you weren't there to experience that measure in it's entirety i.e the the billions of years for you to experience the specialness of the blip. Also compare that with billions of people current existing at the same time as you who are also the product of this astronomial odds. The awe of the statement of the specialness of existence quickly fades away when you take the former statement into consideration

One could take this in the opposite way, we're so special that we are barely get to live long enough to experience reality for what it is and have to make do with such a tiny drop. The unfairness of it is misery inducing. We are so special that we get to appreciate this specialness only if we're lucky enough to be born in a first world country and to decent parents and born healthy. Aside from that we have spent a very significant time sleeping, pooping, dealing with BS, dealing with things out of our control etc etc

Existing is truly miserable if you aren't living in a first world country.

Man your statement is just hollow. The astronomical odds of existence is nothing celebrate by itself just as hope by itself is useless

Building the future

Is it not true that The Arc test is designed to be one where the rules are dynamic? i.e every one of the tests are different from each other in an absolute sense. Learning about one tells you nothing of substance about the other unless of course you/the model is capable of meta-learning

Finetuning has been looked down upon because all it does is rearrange weight to learn style of the finetuning dataset. It does not teach the model anything which is in contrast to the hopes behind finetuning

If a model was able to ace the arc-test just by the merit of being finetuned, does it not imply there is something of absolute substance here? i.e the model is capable of meta-learning and all it needs to adapt to a new-task is a bit of finetuning which again I emphasize is the loweest tier in the ranks of types of training models

Something to chew on for me. But what is a manifold then if not a topological space that is locally the same as R^(some dimension) ?

tangent: any reason to assume it gets mapped to a manifold rather than something that is not?

"How much do you earn right now?"

With all due respect, that is none of your damn business

"Interesting. I have no more questions"

I think there needs to be some feedback that the goal is sustainable, one tooth a day to an adhd mind is just as bad as not doing anything as it's still a very tall staircase to climb. Too granular a step

It really feels like a scaling problem than anything else. It big companies go to shit as they do most of the time it's because tackling scale is a extremely difficult problem

Man, I don't get it. Was I the only one put off the video game portion of the story? It seemed so out of place and enough for me to go "rubbish" and toss the series aside

This sort of hubris has been the markings of the beginnings of the fall of companies. I am now fearful for Nvidia

Why someone like Henri Poincare or Polya was not included in this list is beyond me. Both were interested in figuring how mathematicians did what they did and both have brilliant insights

by taking steps to verify everything that was said

Read the Butlerian Crusade

The main point you're making is fair

The only gripe I have is > Being an empirical science does not mean that the field is a "wild west"

I think what you meant to say is: "Being an empirical science does not <b>necessarily</b> mean that the field is a \"wild west\""

you clearly haven't seen the social sciences

Good practitioners know this

sure?

Edit: Removed unnecessary portions that wouldn't have continued the conversation in any meaningful way

This makes me wonder. Is deep learning as a field an empirical science purely because everyone is afraid of the math? It has the richness of modern day physics but for some reason most the practioners seem to want to keep thinking of it as the wild west

My predictions:

1. state-space models make transformer based models obsolete

2. Cuda killer gets set loose by AMD

3. The world's first successful head transplant takes place

4. Children of Dune gets greenlit

5. Lex Friedman retires from interviewing