hzay

We can't just look up info using an LLM. We have no clue what its weights and biases are based on, and whatever other layers control the output. Just a total black box. It would be an irreplaceable loss if we lost wikipedia to LLMs.

When I was learning ML, I spent a lot of time using sentence transformers. Seriously underrated. Happy to see it here.

How do you debug it properly? Suppose you see others not have the mysterious difficulties that you have. What if they were simply pretrained - through prior exposure to that material?

How would you even know that this was the case?

I think if you're fortunate enough to really, deeply want something, then you should simply train to become good at it. Don't worry about your natural talents, since those will change.

Personal anecdote. I started learning a rigorous dance in late 20s. No fitness or movement or musical background. Programming/sit on my ass background only.

After 10 years of it, when I try something like tai chi now, the teachers pick out that I'm genuinely "gifted" or "talented". Then I tell them I'm a dancer and they'd be like "oh that explains it".

This happened even 5 years into dance training. I had absolutely no talent for it - I always struggled with mysterious problems others never had. Whether it's postural, rhythmic, musical, whatever. Had it all.

My point is, identity change happens much faster than we imagine, when you go all-in. It doesn't take 50 years. But it's also slower than we imagine. It's not 5 months. You have to understand the timelines of human change.

Of course on day 1, week 1, year 1, even year 3, everything sucks. You can't then write an essay saying "here's my lessons from learning journey". I will believe an essay when the author gave his youth to understanding the nature of talent. Not if he gave it 3 years.

Hey I'm reading that book too! Glad to meet you! I love that book.

I mean, my friend's wife is an eye surgeon. My husband's cousin is a pediatrician, with neonatal specialty. I'll take software engineering.

Lots of happy examples in this thread. Let me add mine.

My 3 year old vastly prefers complex carnatic music to cocomelon (and its ilk). He can listen to a 15 minute, intricate song without losing interest, and will ask for it in a loop. Children can handle a lot more complexity than we generally assume.

Some languages are supposed to be very difficult & mentally taxing to learn, because they have many conjugations. But a native speaker with very low intelligence (however you measure it) has zero trouble conjugating it all correctly.

ITT: ppl saying LLMs are v helpful

The keyword in title is "bullish". It's about the future.

Specifically I think it's about the potential of the transformer architecture & the idea that scaling is all that's needed to get to AGI (however you define AGI).

Companies will keep pumping up LLMs until the day a newcomer puts forward a different type of AI model that will swiftly outperform them.

I started with this 3 part course - https://www.coursera.org/specializations/machine-learning-in.... I think the same course is available at deeplearning.ai as well, I'm not sure, but I found coursera's format of ~5 min videos on the phone app very helpful (with speed-up options). I was a new mother and didn't have continuous hours of time back then. I could watch these videos while brushing, etc. It helped me to not quit. After a point I was hooked & baby also grew up a bit and I gradually acquired more time and energy for learning ML. :)

fastai is also amazing, but it's made of 1.5 hour videos, and is more freeflowing. By the time I even figured out where we stopped last time, my time would sometimes be up. It was very discouraging because of this. But later, once I got a little more time & some basic understanding from Andrew Ng, I was able to attempt fastai.

I learned ML only to satisfy my curiosity, so I don't know if it's useful for interviewing. :)

Now when I read a paper on something unrelated to AI (idk, say progesterone supplements), and they mention a random forest, I know what they're talking about. I understand regression, PCA, clustering, etc. When I trained a few transformer models (not pretrained) on my native language texts, I was shocked by how rapidly they learn connotations. I find transformer-based LLMs to be very useful, yes, but not unsettlingly AGI-like, as I did before learning about them. I understand the usual way of building recommender systems, embeddings and things. Image models like Unets, GANs etc were very cool too, and when your own code produces that magical result, you see the power of pretraining + specialization. So yeah, idk what they do in interviews nowadays but I found my education very fruitful. It was how I felt when I first picked up programming.

Re the age of LLMs, it is precisely because LLMs will be ubiquitous I wanted to know how they work. I felt uncomfortable treating them as black boxes that you don't understand technically. Think about the people who don't know simple things about a web browser, like opening dev tools and printing the auth token or something. It's not great to be in that place.

Not the poster you responded to but I learned quite a bit from kaggle too.

I started from scratch, spent 2-4 hrs per day for 6 months & won a silver in a kaggle NLP competition. Now I use some of it now but not all of it. More than that, I'm quite comfortable with models, understand the costs/benefits/implications etc. I started with Andrew Ng's intro courses, did a bit of fastai, did Karpathy's Zero to Hero fully, all of Kaggle's courses & a few other such things. Kagglers share excellent notebooks and I found them v helpful. Overall I highly recommend this route of learning.

What an incredible waste of words. What is this even based on?

For a start, where I'm from, there is no marker around the age of 40 (or 35 or 45). It's the age when people are extremely busy because of career & kids.

Secondly, culture is made by a lot of people. Not so many in their early 20s, but several from late 20s to 60s.

Thirdly, it is also made for people over 40s. Not perhaps Hollywood movies but how about carnatic music? Concerts in Chennai overflow with people over 50s, and barely anyone in their 20s.

Fourthly, it just sounds like someone had their 40th birthday, felt the usual crisis, and tried to make up some idea to create meaning for themselves. Nothing wrong with it I suppose, except that it has no meaning for others.

Thanks for asking this -- your question made me realize I don't have much experience with pain at all! The answer is pretty run of the mill stuff -- I got my finger stuck in a door-crack-like spot. Ugh.

Childbirth pain is not horrible and sharp like that. It feels a lot more productive, so to speak. It's not right to use the word "pain" to describe both things actually. Without that pain, you'd have to depend upon a nurse to tell you when to push -- there would be a delay between her seeing the monitor and asking you to push. It all seemed very unlikely to work to me (but of course it works). But if you feel the pain yourself, you will know when to push.

Also it comes in waves. After each wave of "pain", you get completely pain-free rest periods. But of course towards the end when the baby actually comes out, the rest periods shorten and the pain periods lengthen so it all kind of rolls into one. Many women are not so lucky as me and have prolonged labour, lasting into days. Mine only lasted a few hours so it was fine.

I completely agree with the other person who replied to you. Personally I said no to anaesthesia because they put that in your spine -- shudder.

Pretty common. I did.

Imagine someone being a brain surgeon also interested in the heart, kidneys and urinary tract, nose-throat-ears, ... that's a Renaissance Doc.

Do you mean that such a Renaissance doc this is not a good thing? I think this is exactly the sort of doctors we need.

Right now, my father has diabetes t2, high bp, a slightly enlarged prostrate gland, a series of UTIs, hernia, skin rashes and probably a few other things. He's fairly rigorous in researching & learning about his conditions. But no doc he consults takes an overall picture of his health. You have nephrologist, urologist, cardiologist and so on. They tend to miss things among themselves until he reminds them.

Yes medicine has advanced greatly but a renaissance doc would be transformative for people like him.

He meant that micromanaging and getting into details is good (not disparaging him here, fwiw I agree with him), but he didn't want to come right out and say it. Also, I think he's not yet sure what else it involves -- like the 100 ppl party thing. He just knows it's not delegation.

My amnesia is somewhat intentional. Like some of the commenters here, I'm doubtful if I should really take it everyday. Just seems excessive? Did it during pregnancy ofc, and it's not like the early days of breastfeeding when you're making almost a liter a day (kiddo is 2.5 so while still BFing, it's not an important source of food), so idk. My grandmother had like 10 kids, breastfed them all and never had bone issues. My mother breastfed us v little (<6 months all together) and got osteoporosis. I'm not at all sure how well-supported this recommendation is.

I try to eat more yogurt/kefir instead.

This is great. My mother had osteoporosis and I hope they come up with a therapy she can use!

However as a currently breastfeeding mother, I'm asked by doctors to take calcium supplements every day (I only remember it once a week or so), and they threaten me with future osteoporosis if I don't take it. But these researchers are saying that breastfeeding mothers' bones aren't affected despite calcium depletion?

Fwiw I've also read research that the bones are indeed affected (as measured by density) but they rebound after you stop breastfeeding. I remember that the most depletion happened in the lumbar region, and that the rebound didn't happen fully if there was "parity" (multiple kids).

So idk. I hope their premise is correct.

Visualizing 3D matrix multiplications, and getting comfortable with it. Then there's basic calculus in understanding gradient descent. Can't think of any other advanced math that was necessary to grok the innermost workings of most models today.

Source: I won a silver medal in a kaggle competition after 6 months of ML self-learning.

The end neither justifies nor undermines the means.

Is it really secret though? It's the first thing you learn when you google "how to take bp". Like I'm not a medical person but when I was pregnant, I was asked to check BP and I asked the doctor "should I like average 3 readings or something?" and she said "just take one, don't stress it".

This is well known re thyroid medication as well. Also re weight, progesterone, a bunch of things.

Nice. I tried visualizing this algorithm here - https://visuallyexplain.pages.dev/kmp/algorithm-(wip) . The sidebar has a bunch of completed algos -- DFS, BFS, binary heap queries, dijkstra's, union find, topological sort, quicksort, etc.

I abandoned this project without completing KMP. If anyone is interested in this sort of algorithm explanations, I'd love to collaborate / finish this project.

Does anyone know if there's something similar for plain old geometry? I need some proofs beyond basic ones (for ex, I need to know how to find out intersection points of 2 rotated ellipses on Cartesian space).

Any parents here who have successfully managed this with their kids? Please share resources or ideas for newer parents.

I so relate to that out of body experience. My story is different though -- 10 years ago, I was driving along the SF-LA scenic route (I forget its name). I'd rarely driven in US at that time and was brand new to the country. Suddenly I felt like I knew the route. I'd seen these underpasses before, surely? It was so surreal. I could even predict some road feature that was going to show up next.

Turns out the game Roadrash that I played as a kid had that route :)

Aswath Damodaran speaks of a life cycle for companies. In his view, companies do decline and shut down after sometime. He makes it seem like a fact of life -- if it's true, then stock markets aren't so terrible after all. The declining companies often pay good (but declining) dividends, and wind up gracefully.

I also have always slept with our child in our bed. This hasn't done much for my sleep.

there was a point where I thought to myself I haven't had more than 2 hours of continious unbroken sleep for few months

Omg same, but for more than 2 years, because I'm a breastfeeding mom. Idk how this is healthy or normal!

I love this story. It's a lot more humble and nuanced than the similar one by Ted Chiang called "Understand".

I get disproportionately angry at even low levels of unwanted noise.

I also suffer from this problem. Is there a name for this?