usernametaken29

Every skill humans once bragged about is getting commoditized. Writing, coding, analysis, translation, design, research. A machine does each of them faster, cheaper, and increasingly better.

This is very much up for debate. I also would have liked your article to delve into whether thing’s that bring us humans joy actually need to be faster, and cheaper. Knowing why you do something crucially depends on knowing the thing being done. It’s very hard to cast an ethic judgement over a process you don’t understand. If your argument is that we need to be ethic stewards of machines but need to step aside the doing then your premise might lead you straight into tautology.

Maybe the LLM doomsday won’t be self aware AI but instead everyone will be able to alter anything in any customer service backend in any way they want to the point we bankrupt all the banks and insurance agencies and governments. Yikes

The challenge for enterprises is no longer proving that AI agents can work, it’s making them reliable enough to do high-value work in production.

Is this some sort of joke? Like. That’s the proof. If it can do reliable work then that’s the proof. Contradiction in sentence one. If your pipeline breaks 80% of the time but sometimes the magic token lottery gives you something remotely useful, that’s NOT proof. The ceiling for these companies is so low it’s unbelievable. What happened to products that worked first, and worried about the rest later? Can we have an inspired ad article for that instead?

So essentially you’re telling people: “Don’t use the evil labs they can read your DNA, instead send DNA to our lab and with magic maths we won’t be able to read your DNA but diagnose you!”

Sounds about right to me

This! And even if you could magically have a 1000$ sequencer at home, you still need to correctly extract the blood, and apply the kit. It sounds easy but really isn’t, this isn’t home baking but PCR, where it’s easy to mess up. You need to know when a result is a true positive and whether your sample is useful / correctly collected. Maybe an inspired bioinformatics citizen lab can do it, but it’s bizarre to think the average consumer would do it. It’s not as simple as taking a drop of blood beep boop result like it is for insulin tests and I don’t think it’ll be like that for a very long time to come.

I literally presented you with the solution in the sentence before. Buy a usb c dongle key. You pay 20 euros to NEVER have to remember a single password ever again. Seems like an ok trade off to me. Likewise you can register multiple passkeys for multiple devices, so long as you do it in sequence (first Apple, then Android) etc. Really it takes very little getting used to. Ever tried to fill out a crap password form which wants 7 special characters but no dashes but uppercase but not uppercase Y and so forth.

In genetics the end user is never the consumer. You have to collect blood, prepare it, sequence it. These are all highly specialised steps in which something will and can go wrong, and that’s why typically a hospital carries them out. That’s before even getting the device to do sequencing. The cheapest nanopore comes you at a couple ten thousand euros. Given the premise your work strikes me as oddly theoretical. Who is that mystery home DNA lab that needs decentralised private compute? Also, and funnily enough, for much of the genomic pipelines to run (eg a paternity test or a cancer test), a home computer is sufficient. Again, of course, you would need proper medical training to read, interpret and judge the results, which is why a hospital does it… so what problem exactly are you solving?

I don’t understand this point at all. I think the author has himself confused with the average consumer. For the first time in a decade or so you can buy a PHYSICAL key and use it to sign into websites. I can explain this to any grandma out there. Likewise, I’m an Apple user. Once you’re in Apple universe passkeys are extremely easy. Tap your thumb on the scanner, done. Now we can put on the tinfoil hat and say how this fosters vendor lock in yadda yadda but the last thing I would say is that it has terrible user ergonomics. LOL

I personally know some businesses in which the key differentiator is excellent human customer support. But then again, we’re talking 5 digit software licenses where that matters a lot more then for simple SaaS subscriptions

Wow, putting effort into training material, thoughtfully designing it, and relating the material to the final exam, will increase performance on said exam. So much AI so much wow. Like seriously. Most university statistics courses suck big time, so literally any effort put into them will Improve the field. I’m happy the authors want to improve education but they don’t seem to understand that preparing questionare style material is a confounding factor which could very well explain the better performance too… instead of cramming AI into the next thing. I’m generally opposed to AI on basic textbooks. You don’t want hallucinations imprinted on students who have no idea and can’t judge the quality of the generated text. Some things require effort, reading intro to statistics is one of them, and it’s for a reason, the effort IS the learning

This article is awesome. It should be required reading for all engineers but probably mostly ML researchers. I’ve encountered my fair share of geniuses that are oblivious to the fact that the world is indeed complex and has a near infinite amount of detail. Of course, they’ve been trained as engineers, so they think in models and abstractions, but reality is almost always more complex than what people at desks estimate it to be.

You’re thinking information content in the neurons but not distributions. D(world of text) is much much much larger than D(transformer). Just because you’re passing information through a smaller channel doesn’t mean the original distribution necessarily isn’t smaller. Tokenisers are the numbers but then you have V^n where V is your vocabulary and n is the sequence length as a subset of the whole distribution of possible text. Now you pass that through something with sufficient capacity, which compresses the input distribution (you loose bits), and then recreate from your lossy distribution. AutoEncoders are not trained this way because they simply don’t converge if you try to pass smaller input to produce larger embeddings, but they’re doing the same thing. Another way is to think of it is that LLM layers act like binary classifiers via soft max for the input distribution (there were a slew of papers comparing LLMs to SVMs). Essentially during your training you decide what part of the input distribution is going to be part of the embedding. In my humble opinion, while the training diverges, these models produce the same manifold induced by the original distribution.

The problem really is one has to find out what constitutes the embedding and output if you want to apply f(manifold) but as others have pointed out that’s out of scope for this paper. My only insight here is that it is not surprising at all that one layer, or even a single function pass, suffices to get the desired reinforcement. I don’t actually know if there’s any research that quantifies the “manifold threshold” of each layer to try out this approach but it would be interesting for sure

Now that’s interesting.. what exactly distinguishes latent representations and the manifold? IMHO, those are the same, and you’re constructing a piecewise function of the manifold itself. Decoders also produce manifolds much in the same way, with the distinction being that the encoder isn’t learned but static after initialisation. So fundamentally it is still DOING the same operation.

Kind of. Autoencoders don’t need to have an embedding that’s smaller than the input. Their only requirement is that they compress information and thus create reconstruction loss. Typically however they are not trained this way because they don’t converge.. transformers do the same thing, but they can squeeze much more bits of information through one pass because the way they are designed. This holds true even for decoder only networks because they’re still doing the same thing

If you think about it for some time then you’ll come to realise transformers are autoencoders on steroids. A small input space is expanded onto a big manifold and contracted again. Now, suppose you want to impose a function to regulate the output of an autoencoder. It’s actually pretty obvious that you need exactly one layer to do so… f(manifold).

I agree partially with this. America is a special circumstance because of the way the industrial complex and government contracts are linked, and there’s a lot of blatant corruption. If you look at the European regulation framework I believe it works rather well. You have multiple big companies delivering components for different parts of the airplanes from different countries which are all individually big but not too big to fail because no government is solely reliant on them.. as it should be

Honestly this is a good thing. I can endure buggy software but I don’t want to deal with buggy planes. Regulatory pressure is a market force and a useful one too. There’s a huge difference between ship fast and ship right - the latter one requires deep pockets and willingness to commit to ongoing risk. People always say big Pharma and aviation and such are oligopolies, and that’s bad, but they rarely see the capital intensiveness of the whole process. Some things are slow and deliberate and restricted to big corporate only for good reasons

I was thinking the same. Airflow does exactly the same thing. The only benefit here is that it’s their little workflow engine so they can get all their little edge case accounted for…

Wishful thinking has it that we rally our representatives to let OpenAI and consorts rot. The last thing people should do is bail these delusional people out. Let them have it worse then WeWorks and let’s see if their self crowned AGI can help them out of their misery

Ory Hydra was one of the few tools I remember being actually good and lightweight and useable. Tried setting up and using KeyCloak for a while, absolute nightmare

When the title said PR spam my first thought was the massive amount of scam posts released by Antrophic, OpenAI and a gazillion AI tools that solve all your problems (supposedly). Much to my surprise this was also an ad for an AI tool. Lowkey disappointing

While I think it’s a noble idea I think much more could be achieved with much smaller amounts of money. Actually zero. Regulate sugar, introduce a HIGH sugar tax. Introduce higher nicotine and alcohol tax. Introduce stricter environmental controls for poisonous materials and water and air pollution. All these things cost essentially zero to implement, they even bring in money and all of them are credible ways to significantly reduce health problems world wide. But eh, I’m not part of a lobbying organisation, so what do I know.

Honestly can’t tell the difference between this and a regular day r/dbsucks

However, once again, we are seeing a pattern whereby first, models are helpful to humans. Then, humans are helpful to models. Finally, models are largely able to do things themselves. We have seen this in cybersecurity and now the same dynamics are starting to take shape at the intersection of AI and the physical world.

It’s good they are the one seeing those things because otherwise no one else would have. Now if only seeing things would translate into getting any actual economic value out of them… instead of losing billions. But hey, who am I to do a reality check on this shameless piece of hype.

Isn’t the idea of AI precisely to find universal compression from arbitrary input data, at least with LLMs?

You know 60000 words, that’s not a lot, go back to reading the dictionary

Goes to the about section: an average native speaker knows 35000 words.

Ah yes, the classic British insult, should have known it.

Actually I got the timing wrong, it was the 50ies, but here you go:

https://pmc.ncbi.nlm.nih.gov/articles/PMC1363505/

Or is it literally that the freezing and thawing can't happen evenly when you're too big?

This. You literally can’t evaporate all the thawing agent out of the blood of the organism without substantial burns by sheer volume

I remember that cryogenesis was deemed viable in the 80ies but essentially surface area is your enemy. Anything larger than a cat can’t be resurrected. It’s pretty bizarre really, they froze mice and microwaved them back to life.