nborwankar

Not sure if this article provides enough definitive info - happened to see it yesterday.

https://www.newscientist.com/article/mg26335104-500-the-brai...

Seems ironic to be working for a company without integrity, drawing a salary ie your livelihood from such an organization while believing somehow your personal integrity is not already impacted by doing so. The two seem incompatible to an outside observer.

Can you share what kinds of problems were conducive to hyperbolic embeddings in your experience. Also, separately, are you saying companies are using these in practice but don’t talk about them because of the advantage they give? Or am I reading too much into your last sentence.

They have vacuum-cup like pads on their feet?

+1 for Mathematica’s documentation system. It is the best of any software doc, OSS/free or closed/commercial. By far.

This is related to a classic relational database problem called “bill of materials problem” - suggest you Google that. Also look for book on Graphs and networks in SQL by Joe Celko where he goes into modeling graphs in SQL tables. Naive formulations will lead to deeply nested joins which balloon in computational cost.

Gives a whole new meaning to “manufactured consent“.

If it could support the pgvector extension it would be a super fast vector database with all the power of Pg - the relational aspect brings the ability to add and query using rich domain specific metadata usually contained in relational databases.

+1 on fastmail

I didn’t take a year fully off but last year was slow in my consulting work so I did a deep dive on the emerging LLM area.

One recommendation - get a beefy Mac laptop so you can run LLM’s locally. I got an M2 with 96G RAM. Makes a huge difference to your thinking about LLM’s when you can run your own and integrated it with little tasks here and there for experimenting.

Else I find most people think only of centralized closed LLM’s when they think of what’s possible. Severely limiting.

/r/LocalLlama on Reddit is a great community. Other than that check out llama.cpp and ggml.cpp and the whole ecosystem around that.

Cheers and good luck. Ping me on DM if you want more pointers.

Slightly unconnected but I’ve found a rather disturbing “dark pattern” in Amazon sales of food items in CA.

Background: CA requires a warning on food that has some threshold level of carcinogenic or harmful substance - esp lead and arsenic. It’s called CA 65 Warning or a Proposition 65 warning.

I make sure to look for these on the images of labels and ingredients when I buy food especially ground spices off Amazon.

The dark pattern: Label images show no CA 65 warning but when the food turns up it has such a warning! I bought bulk powdered cinnamon with this issue.

Recently I noticed that while Amazon does not show the warning on the label it has an innocuous small print link in your shopping cart that leads to the full text of the warning should you notice it and click on it

This is now in the vicinity of actually increasing the probability of physical harm. Because of not having the warning in the label AND having an almost ignorable warning at checkout when you are more focused on getting things done and move on to the next thing, as opposed to when you are in a more deliberative state while browsing.

Just want to put that out for folks in CA who might care about such things. I’m sure they wouldn’t try shtuff like this in EU.

Programmers Introduction to Mathematics by Jeremy Kun should get you started. Look on Amazon. Then you can decide if you want to do a formal course and which one. I would also strongly suggest you buy yourself a personal copy of Mathematica which will give you a very powerful portal into Mathematics via a notebook interface and an elegant functional programming language. The documentation is probably the best I have ever seen for a software product.

Book overview for Jeremy Kun book from Amazon A Programmer's Introduction to Mathematics uses your familiarity with ideas from programming and software to teach mathematics. You'll learn about the central objects and theorems of mathematics, including graphs, calculus, linear algebra, eigenvalues, optimization, and more. You'll also be immersed in the often unspoken cultural attitudes of mathematics, learning both how to read and write proofs while understanding why mathematics is the way it is. Between each technical chapter is an essay describing a different aspect of mathematical culture, and discussions of the insights and meta-insights that constitute mathematical intuition. As you learn, we'll use new mathematical ideas to create wondrous programs, from cryptographic schemes to neural networks to hyperbolic tessellations. Each chapter also contains a set of exercises that have you actively explore mathematical topics on your own. In short, this book will teach you to engage with mathematics. A Programmer's Introduction to Mathematics is written by Jeremy Kun, who has been writing about math and programming for 10 years on his blog "Math Intersect Programming." As of 2020, he works in datacenter optimization at Google.The second edition includes revisions to most chapters, some reorganized content and rewritten proofs, and the addition of three appendices.

Here’s another one - it’s older but has some interesting charts and graphs.

https://arxiv.org/abs/2303.18223

That’s not quite correct.

Private data especially in the enterprise cannot use public LLM’s like GPT-4 or 5 or N. Use cases needing data privacy have to use an internally implemented LLM application. In Currently, RAG is a concrete and pragmatic enterprise use of LLM’s aside from summarization, which is not amenable to using GPT-4.

GPT-5 may very well be amazing. But unless it runs on-prem it can’t be used in many scenarios because of data privacy.

To the OP - learning how to run LLM’s locally via say Ollama (see ollama.ai) will get you started in a hands on manner. See the /r/LocaLlama subreddit for a very active community around running LLM’s locally.

See Large Language Models - a Survey. Minaee et al https://arxiv.org/abs/2402.06196

If the robots are cloud connected we are doomed the robot apocalypse could be triggered by cyber warfare. Or by malicious human owners. If they are autonomous there better be an emergency “wake word” that instantly disables the robot. In this case I suppose it would be not a wake word but a “sleep word”.

The secular core of Buddhist thought has parallels in the Sankhya system of thought in the set of knowledge systems collectively called Hindu metaphysics.

Buddhism and Hinduism had active and passionately debated exchanges of ideas leading to continuous improvement of each.

Hinduism while considered to be a religion broadly speaking has a philosophical core that is rich and sophisticated in its treatment of the mind and how to observe and tame it. The thick outer layer of colorful belief systems makes it easy to reject it all as nonsense and hard to penetrate to get to the metaphysics of the mind.

The subjects discussed in this newsletter would be some of the first things a neophyte monk in 200 BC India would encounter whether couched in Buddhist or Hindu wrappers.

The question "Can you hear the shape of a drum?" was first posed by mathematician Mark Kac in a paper published in 1966. It was proven in the 80’s that there are manifolds for which you can’t.

“Integer arithmetic is all you need” ? NVIDIA stock arrow up or down?

Re RAG aren’t you ignoring the fact that no one wants to put confidential company data into such LLM’s. Private RAG infrastructure remains a need for the same reason that privacy of data of all sorts remains a need. Huge context solves the problem for large open source context material but that’s only part of the picture.

You do get some money back for a trade-in depending on how old the machine is.

Llama and GPT are auto-regressive decoder only architectures which for pure translation jobs are not the optimal architectures. Training seq2seq models or encoder/decoder models on datasets of sentence pairs designed for translation will likely allow you to use much smaller models. You will not be wasting parameters on general “language understanding” capability that Llama and GPT have if pure translation is all you need. T5 or Flan-T5 might be good starting points.

Got 502 Bad Gateway

Here’s a thread from biology stackexchange about this which also refers to CETI.

https://biology.stackexchange.com/questions/108499/can-whale....

Meta comment about the comments saying “I didn’t find what I was looking for” - core fact is it was using Wikipedia as a knowledge base. So if your topic is not represented well enough in Wikipedia it’s not going to return good results. Secondly, comparing something that runs on your mobile device or laptop to Google is like comparing an apple to a container ship full of oranges.

You don’t have to look far. What happens when everyone can write a letter to the editor and have it instantaneously published? What happens when everyone has a handheld camera and can publish huge amounts of visual crud? Do we really savor a well taken photograph ie form or do we just look for the content. Does art remain art when the form is average and the content is either close to literally presented or just surrealistic? The enshittification of creativity and craftsmanship, perhaps? Mass produced content will undoubtedly lead to enshittification of the platform where these are published.

I have a fundamental disagreement about the premise that abstraction from CPU instructions should be the basis of a language model. I posit that the starting point should be lambda calculus which is formally equivalent to Turing machine based theories. Lambda calculus -> functional languages -> algebraic type aware languages (Haskell?) -> languages used in theorem proving (Agda ?..) is a progression that allows logic and provably correct programs and reasoning about programs. Messy natural language can be used as away to collect requirements by an LLM that can translate them into formally correct tests and specifications which can be used to Instruction-train LLM’s that could (insert frenetic hand waving here) write provably correct (or close) code in say Haskell or Agda. In summary Lambda Calculus is a better starting point because it’s logically closer to formal languages and reasoning.

Shouldn’t this be called Regularized SoftMax? Adding 1 in the denominator looks a lot like a regularization in other ML contexts.

“Everyone is ignorant but in different things” Feynman.

CPU only?