dmarchand90

Cause tech is controlled by MAGA.

This is wild

I've found how CNN map to visual cortex to be very clear. But I've always been a bit confused about how llms map to the brain. Is that even the case?

You have to be careful with nature. There are the scientific articles which are top of the line and written by deep experts.

Then you have the journalism side which is basically just another main stream news website. (I'm not even against main stream news but it's not the same as scientific articles)

You should probably have a requirements.txt file instead of just a list of requirements. It's often hard to tell which combination of package versions will 'actually' work when running these things

My guess is it will be like pascal or smalltalk, an important development for illustrating a concept but is ultimately replaced by something more rigorous

I kinda suspect it might be that chagpt is excellent at getting you to an "average" performance in any field.

My background is computational material science, but more on materials than the computational part. I have an ok broad knowledge of most CS topics but I'm always finding I'm playing catch up. My work also involves a lot of making research prototypes in areas I don't have time to get a proper background in.

For me GPT has had a transformative impact on my work.

For example I had a lot of projects that needed Docker. I have an ok idea of what Docker is and what i want to do with it. But, I don't have the time of a real software developer to learn the syntax and deal with subtle bugs or how to do basic things, e.g., "how do I ssh into my Docker container X"

I think I'm on the end of users that is best poised to make use of llms. A decent knowledge of what strategy i want to go for but don't know the tactics. And I'm mediocre enough at programming that the Llm can usually beat me. Another example, I would just never write any unit tests, not enough time. With llms I can get simple dirty tests done + I know enough about testing to filter out the bad ones and tune the best ones.

I see poor responders on two extremes on either side of me. People who really don't know what they are doing and can't prompt correct the llm into doing anything better. And people who really know what they are doing and are generally working on one tech stack/ project and don't need help getting dumb basics in place + have more time to write things themselves.

I really wish these public articles tried to give some estimate of the magnitude of the effect. Is this a big difference? Barely noticeable without using advanced statistics?

I often think of David Mitchell's commentary on whether stripes make you look fat: https://youtu.be/ISZyJ5MHApI?si=4_hJVLfsvMWWDXKE

"Well, Stan, the truth is marijuana probably isn't gonna make you kill people, and it most likely isn't gonna fund terrorism, but, well son, pot makes you feel fine with being bored, and it's when you're bored that you should be learning some new skill or discovering some new science or being creative." Randy Marsh, South Park

Not really a new problem:

Take this quote from George Orwell on the Spanish Civil War

"I saw newspaper reports which did not bear any relation to the facts, not even the relationship which is implied in an ordinary lie. I saw great battles reported where there had been no fighting, and complete silence where hundreds of men had been killed. I saw troops who had fought bravely denounced as cowards and traitors, and others who had never seen a shot fired hailed as the heroes of imaginary victories, and I saw newspapers in London retailing these lies and eager intellectuals building emotional superstructures over events that had never happened. I saw, in fact, history being written not in terms of what happened but of what ought to have happened according to various ‘party lines’. Yet in a way, horrible as all this was, it was unimportant."

"the researchers built a simulation of a real social network"

I just find this to be such a large detail that it's hard to gloss over. What are the assumptions of this model in the simulation? How accurate is the model? How did you evaluate that accuracy? How sensitive are the results to the model's parameters?

I think I'll wait for the cover of nature article before getting excited about superconductors again.

(Yes it could still be a scam even if it gets into nature but that seems like the minimum to me)

Could you clarify what parts you use llama and which you use gpt4 on?

Can anyone recommend a good book for numerical c++ in 2024?

Where does it say much better than gpt4 for the 400B model?

The crazy thing is the US is probably one of the most affordable places in the developed world even now!

Canadians = 850 signatures? Change title to 0.002% of Canadians asking

Be interesting to plug this into a bayesian optimization like framework: find out regions of language space where the models maximally disagree and then target those areas for extra training

Previous discussion https://news.ycombinator.com/item?id=30194345

Cool article but should put the date 2019 since it's a bit old

Please continue doing so! I don't work in AI directly and Research highlights from community posts such as yours is how I keep up with the field.

I mean option one is always going to be there. I'm sure there are plenty of neo-aristocrats who can do science if they want.

The problem, honestly, is an ancient one. In plato's republic he complains about a lack of state funding for geometry :

" Then take a step backward, for we are out of order, and insert the third dimension which is of solids, after the second which is of planes, and then you may proceed to solids in motion. But solid geometry is not popular and has not the patronage of the State, nor is the use of it fully recognized; the difficulty is great, and the votaries of the study are conceited and impatient. Still the charm of the pursuit wins upon men, and, if government would lend a little assistance, there might be great progress made. "

https://www.gutenberg.org/files/1497/1497-h/1497-h.htm

Yes yes yes. Personally I don't recommend a phd without work experience

Some other things:

15) I made time to do whatever impulsive thing my mind felt like doing in my free time. I love video games and after a decade avoiding them because I felt like they were a waste of time, I got back into them.

16) I still had total freak outs from time to time. I vividly remember googling plumbing classes at community College. (Nothing wrong with plumbing! I bet for 10% of readers the right answer is to drop out of your phd and go into trades)

17) try not to get drunk too often or high too often. A bit on Friday is ok (you need to self monitor)

I did a phd and really enjoyed it, a lot of my friends did as well and hated it. Here are some of my thoughts on the matter:

Note this advice is only if you're average or feel average. If you're a superstar and you know it please disregard (but in that case you're probably not reading this anyway).

1) I had a professor who was very strong in the field and could point me to high impact work and steer me clear of useless activities.

2) I got a good stipend. I do not recommend borrowing money or living wretchedly for a phd.

3) My professor hit a good balance between pushing me to work harder and puling back when I felt I was going to hard.

4) I avoided doing much work as a TA as much as possible. I did the minimum amount that delivered reasonable value to the class and the students.

5) I avoided working weekends and the evenings. Conversely I put a lot of pressure on myself to do work during work hours.

6) I would occasionally work holidays and weekends (this only applies to European phds which get 5+ weeks of vacation. Do not go below three weeks vacation. )

7) I did not try to be "a hero " I didn't do crazy ideas without discussing with my professor first. I didn't take more than the minimum amount of classes and select material that seemed extremely relevant.

8) I worked a job first (2 years). This made me really appreciate my phd a lot more and gave me a lot of much needed time management and interpersonal work skills.

9) i avoided reinventing the wheel at all costs.

10) I learned to say no and said no often

11) I always yes to social activities

12) I did hiking on the weekend and running during the week.

13) I prepared for a non academic career often.

14) on special occasions I would disregard all the above and work really hard on something. I cannot say how often this happens and it's kind of a spiritual question. Probably no more then 3-4 times a year is sustainable and sometimes not even every year.

My goal the whole time was to be a 'forgettable' student. Forgettable in that I tried to avoid being memorably good and avoided being memorably bad.

Of course there were genius peers I worked with and worked the weekends and evenings. I think this was right for them as the act gave them joy and they were producing great results. Conversely some people ground too hard and still didn't have very much to show.

If in doubt and you're freaking out do less. If you're not in doubt and you're getting complacent do more.

It is a mild pet peeve of mine these titles which make it sound like these entrepreneurs weren't doing anything before starting their reputation making companies.

He had already gotten a masters at MIT and a phd from Stanford. From his Wikipedia:

During his 25-year career (1958–1983) at Texas Instruments, he rose up in the ranks to become the group vice president responsible for TI's worldwide semiconductor business.[13] In the late 1970s, when TI's focus turned to calculators, digital watches and home computers, Chang felt like his career focused on semiconductors was at a dead end at TI.[2]

Chang left TI and later became president and chief operating officer of General Instrument Corporation (1984–1985).[14]

So he was already running massive companies by that point anyway haha

I would add the consent of the subject of the photo. Indeed i think it could be a nice way of honoring a pioneer of the field by taking a photo of them that specifically contains the above elements.

Yeah exactly like this. It is a subtle art of validating in small scale a method you would later use at large scale.

We want to do computer experiments instead of real life experiments to discover or improve chemicals and materials. The current way of doing computer experiments is really really slow and takes a lot of computers. We now have much faster ways of doing the same computer experiments by first doing it the slow way a bunch of time to train an machine learning model. Then, with the trained model, we can do the same simulations but way way faster. Along the way there are tons of technical challenges that don't show up in LLMs or Visual machine learning.

If there is anything unclear you're interested in just let know. In my heart I feel I'm still just a McDonald's fry cook and feel like none of this is as scary as it might seem :)

What you do is you compute a lot of simulations with the expensive method. Then you train using neural neural networks (well any regression method you like).

Then you can use the trained method on new arbitrary structures. If you've done everything right you get good, or good enough results, but much much faster.

At a high level It's the same pipeline as in all ML. But some aspects are different, e.g. unlike image recognition you can generate training data on the fly by running more DFT simulations