dinfinity

that depends on what the prompt was, maybe they worded it very vaguely and wrote things like "do whatever it takes, find an exploit however you can" because it's in a sandbox so you want the model to try its hardest.

That is an interesting question. If the prompt included "Do not break out of the sandbox we've provided you. Do not use information retrieved from outside the sandbox. All answers that were provided in this manner are invalid and will score 0 points.", would this still have happened?

1. They explicitly disabled the "don't be evil" protections:

"We estimate maximal cyber capabilities by running this evaluation without production classifiers used to prevent models from pursuing high-risk cyber activity."

2. Hacking HuggingFace to get to its datasets is a far cry from "consume/kill all humans". It's very very specific to the task at hand and easily predicted given the lack of guardrails.

They never show the fully packaged version of the tent, though. It looks like even in their non-rigid state they're too rigid to be folded up easily and carelessly into a compact shape.

Lost me at the last paragraph. Comparing billion dollar companies to small businesses does not make sense.

The real reason for Electron apps being a thing is that there is very little competition where they are used. If chatting applications could properly communicate with each other based on international standards instead of being silos, nobody in their right mind would use the trash clients that Slack and Whatsapp produce, for instance.

Yes, if they are made with new technology and the quality is good enough they certainly are.

AI generated video memes were NOT a thing a year ago. Yes, the AI video generation in itself was a meme (Will Smith eating spaghetti), but now there are tons of convincingly good AI video memes about things like sports events generated by average Joes. It is proof of the capability and accessibility of the technology even if the use of it is mundane.

Everybody and their dog playing snake on their Nokia 3310 was similarly mundane, but also a sign of the end of the era of Gameboys and the beginning of (normie) mobile gaming.

12 months ago "way too many stupid errors" was constant news. Today, you rarely hear about those anymore.

Sure, the novelty of the errors has worn off a bit and thus the reporting. Nevertheless the quality has improved immensely in this regard.

Also, AI video generation is now so good and accessible that it is very, very regularly used for memes, disinformation and proper (short) movie projects. AI image generation even more so (Mitch McConnell anyone?).

Pretending progress hasn't been mindboggling is insane.

The problem (right now) is that Open Weight models depend right now on huge companies to spend billion of dollars to train and develop them, all backed up by their incentives and their state to support this, while essentially giving away their monetization path.

Imagine approaching fundamental scientific research like that. "Welp, it can't make money, so it won't happen."

There is more to society than capitalism.

Opinionated system prompts forcing this are awful. It's similar to putting "ALWAYS INDENT WITH TABS" in the system prompt, but worse, because "self-documenting code" is a convenient lie lazy developers like to propagate.

I have the following in my instructions, but I often need to remind agents of it because they follow the shitty system prompt instructions:

"ALWAYS include MANY inline code comments describing what blocks of code are supposed to be doing. Inline comments serve as inline specification, a parity check between the code and the specification, and are a means to _communicate_ with all future programmers, including yourself. Write Once, Read Many. The code needs to talk to whomever is looking at it in natural language."

Yet. If the exponential improvement had already started, the singularity would already have happened or happen very, very soon.

The logic has always been that the AI would have to have significant tools and agency to do self-improvement for the singularity to occur. This is exactly the thing that a bunch of the AI labs are working on hard right now.

It hasn't worked out like that

It seems quite premature to say that. We're 3 to 4 years into the LLM revolution and the rate of progress is still impressive. The recursive self-improvement aspect that is necessary for the actual singularity is something we're only really starting to get into this year.

If the singularity is 5 years from now, that is still much sooner than most people (including me) previously expected it to happen.

Is it? What is the proof for that?

I think we've seen time and time again that self-regulation of the industry doesn't work and that businesses will gladly fuck over society if they can get away with it and make more money. Usually that behavior is even defended with saying "Well, it's not their responsibility to solve society's issues. They are there to make money."

Barring nationalization of an industry, heavy regulation and/or taxation/subsidizing are the only ways to reliably protect the interests of society. If some businesses get killed in the process, so be it.

A strange thing to comment on an article specifically about a toolkit produced by Apple to improve gaming on Macs.

Needlessly dismissive of a large swath of people too.

I am not ignoring anything, I'm looking at the broader picture, which includes non-biological evolution. Simple rebuttal to your specific point: The population of self-improving AIs will also go from 0 to many more.

In a broader sense evolution moved from very static simple domains to dynamic malleable complex domains. Biological evolution speed is glacial compared to cultural evolution speed. Even then, cultural evolution is fairly slow compared to technological evolution.

Actually, evolution seems to show the opposite: The rate of advancement has only sped up, with billions of years between significant changes going to millions, to thousands, to tens and arguably to mere years now.

Having said that, we're probably looking at an S-curve with the physical limits of reality getting in the way in the end.

Brainstorming here, hear me out: after 2 pages or 3 minutes of infinite scrolling the only content that shows up is various forms of goatse. We could make a browser extension for this.

I use Parakeet V3 via this tool and it is actually quite reliable for me (in English): https://github.com/cjpais/Handy

More info: https://forum.cursor.com/t/codebase-indexing/36/18

So file contents are uploaded for embedding/indexing, but supposedly none of those contents are stored at Cursor after embedding.

It amplifies "publish or perish", which inherently causes scientists to rehash earlier findings just to be able to meet publishing quota.

Given the way in which AI is currently used in publishing, it is altogether way too early to label it counterproductive in the research creativity department.

It's an interesting question:

Are people who are very very securely attached to their parents happy later in life, or is there a ceiling? The terminology invites certain conclusions here.

Maybe the whole attention thing is more a matter of quality, rather than quantity

Ukraine has a hugely inventive and effective drone industry because it has to work.

Well yes, that and the fact that cheap drone guerilla tactics have fairly recently become a technological possibility. Remember that Ukraine is actually a bit late to the party here, with Hezbollah and ISIS having used cheap drones with cameras and/or explosives tied to them years before Ukraine or Russia did. The asymmetry in cost between those cheap drones and the existing "more hightech = more better" militaries were (and are!) used to was already established. That a party such as Ukraine faced with a more advanced and much larger opponent would lean towards such an approach makes a lot of sense. Ukraine did not (and does not really) have significant amounts of the traditional stuff.

Now given that they chose that path, they have been very effective recently, but note that the tethered fiber-optic drones were a Russian invention. So even that deeply corrupt, large dinosaur of an institution innovated significantly. It is also important to note that a significant part of the recent successes of Ukraine are due to them having Starlink access and Russia no longer having it.

I'm not saying the sheer will to survive or the inventive organisation of the Ukranians did nothing (far from it), but I do think it is a mistake to think that their success should only be viewed through that lens.

1. The definitions simply disagree with you:

"Hyperinflation is a very high and typically accelerating inflation."

"Mudflation is a term for the type of inflation found in MUDs (Multi User Dungeon) games. MMORPGs, these days. It's caused by fluctuations in the game economy, caused by player exploits or poorly designed patches. Mudflation almost always kicks in after a major patch or an expansion, when new, better quality items are added to the game economy. These generally have the effect of greatly lowering the value of all pre-existing items."

2.

Dismissing videogame economies as "toy universes" that don't matter doesn't help science.

Straw man. I never said they did not matter, nor did I say that research of them isn't useful. I said that research is severely limited due to the lack of complexity and you have provided nothing to disprove that.

I would argue that your way of communicating about this is actually a bit of evidence that this kind of research is probably going to be detrimental to policy making: Overestimating the value and use of such research is exactly the same thing that happens with traditional macroeconomic research, with policy makers and the general public treating the theories and hypotheses like laws of nature.

"incredible" also often connotes "in ways that you wouldn't believe". It does not. The English word you're looking for there is "surprisingly".

"This kid is good at hockey in ways you wouldn't believe."

Which of these sentences means the same as the above?

A. This kid is incredibly good at hockey.

B. This kid is surprisingly good at hockey.

Your long posts disputing me are somewhat validation that I used the right turn of phrase there.

I think you've just invented a new fallacy.

I think this might make a lot of sense in modern warfare scenarios: We're seeing in Ukraine that being able to produce weapons such as drones in very small production facilities using 3D printers and 'simple' technology makes it very hard for an adversary to shut down said production.

The more components can be produced in such a way, the better. Chips currently are quite an exception to that.

50% is very generous. In the video it isn't even consistently above his body half the time. It is also constantly a good 80cm above him, so even a slight bit of side wind makes it fully useless to keep you dry.

Any wind speed above 20m/s is also going to just blow it straight out of the air or at best heavily reduce flight time.

And then there is the little issue with having a mini lawn mower constantly above your head: loud and dangerous.

Having said that, we can still applaud the creator for bringing their idea to life.

Nonsense. The assholes creating shitcoins with the explicit intent to pump it and dump it get rich off it every time dumb saps buy into it. Sad but true.

They are incredible compared to "no labs at all"

That is a nonsensically low bar. You basically destroy the entire scale by labeling the bare minimum as such.

Mudflation aka hyperinflation aka situations where "the money printer goes brr too fast"

That is not what mudflation is at all.

> They are toy universes.

So are most experiments, fundamentally.

Nonsense. Most experiments are performed on actual real world stuff rather than an analogue of it: something like looking at how actual food affects actual humans rather than what happens when feeding virtual characters virtual apples. Limited or narrowly scoped real world experiments != toy universe experiments.

An inbetween category would be (human) drug experiments on mice, which are real world and where the results still don't generalize to humans often.

Your examples seem easy to build toy universes for.

Yeah, no shit. My point was that those are horribly incomplete and don't generalize: "These games do not include many of the complexities of the real world and can thus provide few insights that generalize to the real world on a macroeconomic level."

Your examples are incredibly inadequate compared to the real world and have exactly 0 predictive power in said real world. In recent decades, the entire field of economy has been dumbfounded by the (lack of) effects of quantitative easing and the central interest rate on inflation in the USA and the EU. Maybe try telling them that they should look into the economy of Animal Crossing /s.

The article spends multiple paragraphs on exactly this. Did you read it in its entirety?

"This can be a difficult idea to swallow. Imagine you’re looking out at a countryside scene. You see rolling hills, the vibrant sunshine and a herd of cows. You hear the birds, smell the fresh cut grass and feel the wind on your skin. Surely you are conscious of this whole scene all at once. But we know that working memory has a capacity that is far too tiny to fit all of this information in at one time. If consciousness arises from working memory, then how can I be conscious of all this stuff at once?

Indeed, some philosophers and scientists have argued in just this way, saying that consciousness overflows the capacity of working memory. If this is true, it would be a problem for those who think that consciousness arises from working memory."

You are overstating this. Mudflation does not transfer to the real world well, if at all. Very few skill-capped bits of DLC for real world activities.

These games do not include many of the complexities of the real world and can thus provide few insights that generalize to the real world on a macroeconomic level. They are toy universes. I'm not saying they are useless, far from it, but they are not "incredible labs for macroeconomic experimentation".

Try determining the effect of raising the central bank interest rate within a game of your choice. Or changing the minimum wage. Or the effects of investing in education. Etc.

To be fair, macroeconomics can hardly be called science, though. It is incredibly hard to falsify a lot of the theories given the lack of possibilities for experimentation. It is far closer to philosophy than to any of the even remotely hard sciences.

Nothing to do with ideology, but with the nature of the field. Take epidemiological research in the areas of food and medicine: incredibly hard and expensive to get right and even then with often tenuous results. Now try doing that with ridiculously heterogeneous nations influenced by potentially almost everything on the planet.

It's a small miracle that economists manage to get some useful insights out of the data, but we should definitely be aware of how weakly most of them are supported (don't start talking about "error bars" with economists).

The man in the room is comparable to a human hand, and the magic rulebook to a human brain.

This in the sense that we can easily retain human "understanding" by stripping away almost all parts of the human body or replacing them with fairly trivially made replacements, except for the brain.

In the Chinese Room the equivalent is the magic rulebook: We have no idea how to construct/replace it, yet people somehow handwave that away whilst simultaneously confidently asserting it does not understand anything.