blackbear_

Having similar thoughts at times. What are you reading nowadays?

The GPT3 paper is a good starting point

Language Models are Few-Shot Learners https://arxiv.org/abs/2005.14165

I also enjoyed the papers for DeepSeek and GLM for an overview of all the tricks you need to make these things work

DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models https://arxiv.org/abs/2512.02556

GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models https://arxiv.org/abs/2508.06471

There are other players in the game: the business and the market.

Good code makes it easier for the business to move fast and stay ahead of the competition while reducing expenses for doing so.

Is it because people genuinely don't care, or because the barrier to become a power user is becoming taller and taller every passing year?

what kind of neural-net architecture and training would allow a model to handle numbers lengths it hasn't been trained on

A recurrent neural network implementing binary addition with carry could do this, and one can derive the correct weights with pen and paper without too much effort.

Whether gradient descent will find them too is another matter entirely

I measure, test, and validate outputs exhaustively.

How do you do this? Do you follow traditional testing practices or do you have novel strategies like agents with separate responsibilities?

There is no escaping the fact that feeding addictions is a great business model.

Luckily our friends overseas have shown us the way of dealing with uncompetitive local industries: tariffs.

Whether the code is AI generated or not is not important, what matters is that it really works.

Sharing code enables others to validate the method on a different dataset.

Even before LLMs came around there were lots of methods that looked good on paper but turned out not to work outside of accepted benchmarks

Isn't it insane that any web page can run a port scan in the first place? Who wants that?

Meanwhile, running opencode in a podman container seems to stop this particular, err, feature.

I had similar thoughts, and my conclusion is that competition is an inherently unstable state of affairs: at some point somebody wins, and they will try very hard to prevent any further competitors from arising.

Indeed, competition is undesirable for all participants involved: everybody wants to win and exploit the rest for their own gain. Note that this is the only way to make competition work and result in its temporary benefits (if nobody wants to win, nobody will compete).

So there must be a system to keep the competition going and preventing the rise of a definitive and exploitative winner, and the existence of this system has to be accepted by the competitors. But why would serious competitors accept a system that prevents them from winning?

But we do ban tools sometimes: you can't bring a knife to a concert, for good reason.

It is both common and uncontroversial to put restrictions on using certain tools in certain situations for safety reasons, especially in public and crowded places: you can't bring a hammer to a concert.

As the provider of a public place, X ought to take certain measures to ensure public safety on its premises. Of course, deciding what is and in not tolerable is the crux of the issue, and is far from trivial.

What other countries could teach the U.S. about bringing down drug prices

A look at how European governments negotiate with pharma companies helps explain why Americans pay more for prescription drugs.

https://news.stanford.edu/stories/2024/11/what-other-countri...

Because US voters prefer the free market as opposed to government regulation and nationalized healthcare.

No, this would crash with numpy arrays, pandas series and such, with a ValueError: The truth value of an array with more than one element is ambiguous.

It is done, in many countries there are legal requirements to report adverse events whenever they are observed upon use

https://en.wikipedia.org/wiki/Pharmacovigilance#Adverse_even...

Generalized Additive Models: An Introduction with R by Simon N. Wood

This book is very strong on the fundamentals, while the R code is minimal and easy to follow.

And so fact checking is back in vogue. Seems pretty biased.

Two thoughts:

how important is computing the exact gradient using calculus

Normally the gradient is computed with a small "minibatch" of examples, meaning that on average over many steps the true gradient is followed, but each step individually never moves exacty along the true gradient. This noisy walk is actually quite beneficial for the final performance of the network https://arxiv.org/abs/2006.15081 , https://arxiv.org/abs/1609.04836 so much so that people started wondering what is the best way to "corrupt" this approximate gradient even more to improve performance https://arxiv.org/abs/2202.02831 (and many other works relating to SGD noise)

vs just knowing the general direction to step

I can't find relevant papers now, but I seem to recall that the Hessian eigenvalues of the loss function decay rather quickly, which means that taking a step in most directions will not change the loss very much. That is to say, you have to know which direction to go quite precisely for an SGD-like method to work. People have been trying to visualize the loss and trajectory taken during optimization https://arxiv.org/pdf/1712.09913 , https://losslandscape.com/

Tailoring the message to the audience is really a fundamental principle of good communication.

Scientists and academics demand an entirely different level of rigor compared to customers of LLM providers.

I really don't know... They made a model to predict the number of views of a video from its metadata, then assumed that model errors correspond to intentional manipulation rather than, you know, metadata not being fully predictive.

GOS allows you to install and use apps from the Play Store and the vast majority of them works flawlessly.

Yes; I wrote linear transformation on purpose not to remain constrained on spatial or geometric interpretations.

The (discrete) Fourier transform is also a linear transformation, which is why the initial effort of thinking abstractly in terms of vector spaces and transformations between them pays lots of dividends when it's time to understand more advanced topics such as the DFT, which is "just" a change of basis.

The fundamental truth is that matrices represent linear transformations, and all of linear algebra is developed in terms of linear transformations rather than just grid of numbers. It all becomes much clearer when you let go of the tabular representation and study the original intentions that motivated the operations you do on matrices.

My appreciation for the subject grew considerably after working through the book "Linear Algebra done right" by Axler https://linear.axler.net

But not all fringe beliefs lead to progress, in fact most of them don't.

I would have expected this to be exceedingly obvious to anybody who ever went to a shopping center, but anyways...

The Global Anti-Aging Products Market size is [worth] USD 47.4 Billion in 2023

Women Dominate Anti-Aging Products Market with Over 80% Share in 2022

https://market.us/report/anti-aging-products-market/

I don't get why invest time in PhD if your work afterwards seems totally unrelated to your expertise

Depending on how the supervision chain is arranged, a PhD can a journey of discovery, of new science but first and foremost of yourself and your interests. It can be very self-directed and the only mandate is to discover something new. For this reason it is common for people to dip their toes in a few distinct but related subfields during those years until they find something that sticks (if at all), and the person that comes out of a PhD can be very different from the person who started it.

Not sure I am following; typical neural network training via stochastic gradient descent does not require Jacobian inversion.

Less popular techniques like normalizing flows do need that but instead of SVD they directly design transformations that are easier to invert.

Strang's Linear algebra and learning from data is extremely practical and focused on ML

https://math.mit.edu/~gs/learningfromdata/

Although if your goal is to learn ML you should probably focus on that first and foremost, then after a while you will see which concepts from linear algebra keep appearing (for example, singular value decomposition, positive definite matrices, etc) and work your way back from there