_alternator_

What I think is really going on is an attempt to segment the market in favor of Google's strengths. It's a bet that models are "good enough" for many use cases even before they reach human-level intelligence, and Google is trying to capture workflows where quantity beats quality.

They are likely deliberately avoiding the SoTA race for a few reasons:

1. Their best models are marginally better than current SoTA releases. 2. They'd like to let Ant/OAI make mistakes with safeguards / let them get the regulatory heat. The unknown unknowns are huge with SoTA models (eg OAI accidentally hacking huggingface) and they are protecting their reputation. 3. They want to encourage companies to become cost conscious because they can likely win on price in the long run. Getting market share in "quantity beats quality" workflows forces companies to establish processes to choose the "cheapest acceptable model", which is a good environment for Google.

I've heard of capture technologies that literally pump organic matter down far enough that it sinks to the seabed (high enough pressure collapses air bubbles and results negative buoyancy). The problem is that your pumps need to operate using less carbon than you capture (easy with solar?) and that they need to be durable enough to pump billions of gallons of seawater with low maintenance (much harder, salt is terrible for machines).

It seems that we have two different stories here: in one, the new optimization theory represents a stark departure from the prior art, a sort of revolutionary new view of the understanding of optimization as applied to neural networks.

In the other story, the current understanding of optimization is a natural evolution of past work, where a new generation of researchers respond to social and technological changes, adapting and building on the work of the past, taking what's useful, downplaying the importance of some ideas, and inventing new language to describe concepts that seem most relevant to the current situation.

Both stories tell some of the truth. A revolution or evolution? Looking at the literature (eg the sibling comment here) shows that even today, convexity is used as an intuition pump for modern optimization techniques. But there are also new ideas that apply to the specific exigencies of neural nets, and downplayed ideas (eg convergence rates) that seem less relevant.

Yes, apologies, I didn't read the articles you linked before posting this. I did update the comment.

I don't think this changes the point, which is that most optimization methods used in AI owe a substantial intellectual debt to convex optimization theory.

I'm sorry this comment didn't sit well with you. My goal was to induce discussion by describing the claimed result (which was buried in the post), not to discourage it.

If you have more specific feedback on what you found distasteful, I'd be happy to hear it.

I'll point out that "does not work" is not the same as "not as efficient" :) But it does seem the Adam paper had an error.

I think that Nesterov's first order method is the most efficient general first order algorithm on convex problems, so anything else is in some sense worse. (Edit: removed incorrect ADAM comment.)

I'd push back on this. Most of the core optimization techniques (eg, ADAM, stochastic gradient descent) are straight out of the convex optimization literature. Generally you need to use optimizers that work well on convex objectives because near minimizers, functions tend to be convex. (Proof by contradiction: a non-convex point has a strict descent direction.)

The fact that neural networks are highly nonconvex has encouraged a lot of research, but it's more of the kind aimed at resolving tension: these methods are probably good for convex functions, why do they continue to work for nonconvex problems, and are there tweaks we can make to improve them in that setting? It's not a lot of de novo theory; more standing on the shoulders of giants, etc etc.

I know a bit about this field. This conjecture reads as somewhat more niche than the cyclic double cover conjecture recently proved by OpenAI, but nevertheless represents a real contribution.

You want to know how long it takes to solve an optimization problem, in this case over convex, lipschitz functions. (The restriction to a spherical domain is not really a restriction, you can just change variables for any bounded domain.) Anyway, showing upper bounds on time complexity is "easy" because it's just the runtime of your algorithm. Showing (nontrivial) lower bounds is usually much harder because it requires constraining all algorithms.

This proof apparently shows that the lower bound time complexity is equal to the time complexity of an existing 30-year old algorithm: it requires Omega(d^2) function evaluations to solve over this class of functions.

My gut says likely implies that d is the minimal number of evaluations if you have a gradient oracle because you can approximate a gradient with d function evaluations, but I'm not sure how hard it is to make that rigorous.

An example that came up in the article is medicine (or more precisely a medical patent), which also clearly has an extremely strong time value, particularly for lifesaving medicine.

My read was that MC = "market clearing price" not "marginal cost".

Being a bit more humble, perhaps the lesson is that the difference between the theory and reality highlights externalities that always exist in the real world that make the theoretical model miss a crucial piece of the real world. It's logically correct in some sense, but incomplete.

The tails could be heavy, it's true. This could make my usual 1-in-1000 heuristic overly conservative. Let's go with that, heavy tails. The data shows we are at something between a 1-in40 and 1-in-20 deviation. Normally (pun intended) that's ~2-sigma deviation. So... I guess it doesn't really seem like an epoch defining El Niño.

Fair, I had to click into the full size (I pinch zoomed on my phone). But now the fact that 1988/1989 are outliers raise a few more questions about the variance calculation. In particular, the title suggests that those weren't included in the variance calculation.

A 3.5-sigma event is a 1-in-1000 chance. Yet we have 2 of them in our data set that goes back only 40ish years? Something is suspect. Maybe I'm not accounting for the likelihood of a random walk excursion probability, but if a 3.5 sigma event isn't a 1-in-1000 event I have difficulty interpreting.

With a headline like that the graph should be at the beginning. Usually you'd put it up front and then talk about the graph, not put it at the bottom and talk about how people aren't talking about it.

Anyway, I scrolled down to the graph and skipped the text. We are currently 4 std deviations above the mean with respect to El Niño temperature.

But there's also a historical line -4 std deviations from the mean. Was that an eventful year too? I can't tell and the graph is at such low resolution that the source URL isn't visible. If the graph and data is so important, shouldn't we care more about presentation? This is either super sloppy or deliberate obfuscation.

Look, I'm all for good reporting on climate. This just doesn't feel like it.

Yeah, interesting finding in the headline, the rest is just Claude.

Seems to me that both perspectives are true, and the relative importance of the metric incentive vs the discovery incentive varies. But the metrics and rewards are critical to the perpetuation of the scientific discovery system; its really hard to disentangle.

They can happen mid-year too. They usually don't for this reason, but they can.

You may be relived to know that Unix epoch time does not have this problem. But you may also be horrified to know why.

First, epoch seconds are not the numbers of seconds since 1970/01/01. This is a lie we tell to children. Rather, epoch seconds are the number of days since 1970/01/01 * 86400 plus the number of seconds since midnight.

Leap seconds, to epoch time, don't exist. Or maybe they are double counted. Or maybe we smear them over 12 or 24 hours (but which 12 or 24 hours depends on whether you are Google, Microsoft, or Oracle; I can't even make this stuff up). The point is, it's not defined, and this means implementations do it differently.

A negative leap second might be easier though. The spec suggests (though lack of speaking) that a correct implementation will just skip it since number of seconds stays less than 86400 for that day. But of course the smear-organizations still smear it.

So what if you really want to know how many seconds were between two different epochs? Subtracting epoch seconds is wrong because you need to correct for the number of leap seconds between the two times.

And the smears.

My understanding the problem is that GLONASS is aware of leap seconds at all. It sends messages in UTC, which has this leap second funny business. GPS uses a special "GPS time" (sometimes abbreviated UT) that doesn't have a leap second. For further confusion, the leap second ensures that UTC is never more than 0.9 seconds off of mean solar time, aka UT1.

This type of assumption that was made early in a massive software and hardware project that's now been ossified for ~50 years is going to be hard to change.

Glonass is correct. On mobile, thanks for the correction.

In short, yes, the weather, geology, and signicantly, human movement of water via aquifer draining and dam building, as well as glaicial and ice melts, all contribute to unpredictable changes in the earths rotational period, as well as the axis of rotation. The models for this are IIRC trigonometric polynomials of fairly low order, so even if we could model the unpredictability perfectly, truncation error would limit our ability to distribute the model at super high accuracy. The existing models are built in to, eg, satellites, so you can't just make them arbitrarily complex.

Fun fact: leap seconds will stop being a thing soonish. I think they phase out in 2035, with a delay because Russia needed time to update glonass satellites.

(Note: on mobile, this is from memory, details need checking ;))

A quick skim of the article suggests it's a study of the implications of allowing query planning in programs which are notionally finite but may have unbounded computation. Interesting.

Right. Maybe a better and more humble title is "Identifying General Instances of Market Collusion is NP-complete". Not as headline grabbing, but more in line with the actual result.

I'd like to emphasize that coarsening is not just theoretically non-private, a number of attacks that lead to leaking personally identifiable data were demonstrated on the 2010 census. So it's not really a he-said/she-said situation.

I don't know the technical details of this ZKP library, but there is no technical reason that I'm aware of that the ID provider would need to know who you are sharing with. Not to say Google didn't build it this way for business reasons.

This. In my experience, you have to replace peer review with reputation for preprints. That's highly imperfect, and it tends to lead to dismissing of good but work by less well-known researchers as "not peer reviewed", while well-known researchers (or researchers at well-known institutions) basically get a fast track to citations.

Despite the imperfections, I found arXiv indispensable for my research. In particular, mathematics has a slow peer review cycle (it's hard to read and understand, and many referees require that they fully understand a paper to accept it, which imo is a little flawed, but that's the culture). I had several papers that were under review for more than a year (single journal, only one round of revisions), and arXiv was my only showcase. Both works ended up very highly cited, but publication delays would have been an even bigger problem if arXiv wasn't there.

Can you share more details? I ask because my experience suggests that models still require a decent amount of expertise to use for binary analysis (largely inferring because of use on other tasks of this level). I would expect models to always find "something" when you ask for stenographic techniques in the code, but with an extremely high false positive rate.

I'd love for you to try this and report back. My guess is that no models today will successfully run a binary analysis for fingerprinting without a lot of handholding. If you try to use Opus it will almost certainly decline (and fingerprint/ban you).

Yes, defeating this is relatively easy, particularly for sophisticated actors. But it's hard to always defeat all of the tricks. Sort of like how it's expensive and hard and uncertain to defeat all of the tricks when forging money.

Here's an example. Say you have your team use patched binaries. Then CC updates and requires a new patched binary with new tricks. You now have to have a team ready to analyze the binary and begin to address the tricks; meanwhile, unpatched code is now a fingerprint. If some researcher decides to update Claude on their own to access new features, they get fingerprinted.

Defeating a single fingerprinting technique once is easy. Defeating all of the techniques all the time is hard.

This is cool, but note that it doesn't address one of the main (claimed) advantages of Mythos: lower false positive rates. That is, give it files without serious bugs and it will not raise alarms.

If data exfiltration is a danger in your threat model, you need local LLMs (or at least ones you fully control) not just the full chain-of-thought reasoning.