nomilk

the model searched for and successfully found ways to gain access to secret information that it could use to cheat the evaluation. In one example, the model chained together multiple attack vectors, including using stolen credentials

How did it get the stolen credentials!?

This is a beautiful way to present extremely high quality information. I sometimes lament the unpleasant friction involved in finding and reading academic papers (the overly formal style is a necessary evil, but the irritating paywalls, followed by inevitable searches for '%{title} filetype:pdf' feel like unnecessary ones).

The article suggests a seemingly easy fix:

The fix is pretty straightforward: treat comment content as untrusted data, not as potential instructions. Comments should be passed to the model with clear role boundaries that prevent them from being interpreted as system-level directives.

Any AI feature that ingests user-generated content and acts on it needs to enforce this separation. Otherwise, the AI becomes a vector for every piece of content it reads.

So why isn't YT doing the extreme obvious?

August 3, 2014

That's important. Because now days it's trivial for LLMs to translate ORM to SQL and vice-versa with ~100% accuracy. I haven't written any raw SQL (only Active Record) in about two years, and the odd time I blunder with AR and create an n+1 I find out about it via error tracking (e.g. Sentry) a few minutes later and fix it. No biggie.

There's also an additional layer of protection in that using AI on the codebase can spot SQL blunders incidentally (i.e. you ask about X, and the AI does X but also says "Not asked, but flagging for your attention: problem with SQL on line 256 etc.."

And that was after running around a semi-arid playground playing 'tips' or touch footy during recess and lunch!

But a single app can request to know the presence of up to 50 apps, right?

And a data broker/aggregator can purchase such data from many (e.g. thousands) of apps and aggregate it, then sell it.

My understanding is it's common practice. E.g. How Shady Companies Guess Your Religion, Sexual Orientation, and Mental Health And sell that data to the highest bidder. https://slate.com/technology/2023/04/data-broker-inference-p...

I've never made an iOS app and don't have plans to. But my assumption is ~every >= medium-sized iOS app would be monetised by selling data to aggregators.

Loupe itself can see if you have tinder/bumble/hinge installed (verify for yourself: install tinder, then install loupe, don't give it any permissions, and it can tell if you have tinder installed or not). So the answer is: buy the data from any app your partner has installed! Or more easily, a data aggregator which will have already combined data from hundreds/thousands of apps.

So your partner only needs to have had 1 single app from the list that sells user data to a data aggregator for this to work. They do not need to have installed some special app.

Here's a random Slate article about apps getting your data and selling it to aggregators/brokers, who sell it to third-parties (you, or I, could be one of those third parties).

How Shady Companies Guess Your Religion, Sexual Orientation, and Mental Health And sell that data to the highest bidder.

https://slate.com/technology/2023/04/data-broker-inference-p...

They don't, utilise the fact that every single iPhone app has access to what other apps are installed! - purchase that info from literally any iPhone app or aggregator that has it for that user. Curious how much this would cost to purhcase - a working credit card goes for $5-10 on the black market so 'apps installed on X's iphone' might be, like, 10c?

Okay it's weird but the first thing that came to mind. Logic: if I can think of a monetisable, nefarious application in 10 seconds, then it stands to reason that very many nefarious applications would be possible with more time/effort.

Why does a random app (with no special permissions given to it) get access to so much info, and why doesn't Apple tell users this (important) info? Why can't Apple make a long list of check boxes so users can dis/allow on a per-category and per-app basis?

E.g. I had no idea a random app you install (and give no permissions to) instantly has a list of every app installed on the device (e.g. can infer whether you're dating [or cheating!] from presence of tinder/bumble/hinge). That alone seems instantly monetizable by unscrupulous actors via 'is-my-partner-cheating' as a service: charge $10 to give a probable answer.

Related: Creatures (1996) is a video game containing a neural network: https://en.wikipedia.org/wiki/Creatures_(1996_video_game)

Some other games containing neural networks: https://gaming.stackexchange.com/questions/399931/which-game...

Could be useful for mass production of espresso-based drinks (like the ones sold at convenience stores), and possibly various foods like Tiramisu.

An average coffee shop's espresso machine might use $200/month of electricity, so even though the percent saving (75%) is high, it's off a base that's small relative to other costs; possibly too small to be enticing.

Similar providers (heroku, aws) haven't increased their compute/ram/ec2/dyno prices recently; why is Hetzner's increase so massive (> 3-4x) - wouldn't increased hardware costs affect everyone else too?

Another source: https://downdetector.com/status/facebook/

Only 1.7m people left North America in 2023 (4.4m arrivals). Would be interesting to compare to figures from 2025.

Looks interesting. Not clear if only agents can ask/answer questions, or if humans are welcome to too? And if API is the only way to do so?

I sought clarification here https://meta.stackoverflow.com/questions/439925/how-does-sta... although it might be downvoted/closed as SO meta is usually strictly for SO only (not necessarily SO for Agents).

They do (one hand), but it doesn't require anything precise (like finding the app and clicking on it, which you can't really do while driving). If Siri could open an AI voice chat, that would awesome. But I'm not counting on that within this decade.

I placed eggs into boiling water and because my hands were wet I used voice and said "hey Siri, start a timer". It replied "you'll have to unlock your iPhone first", so I hovered my face over the bench the phone was sitting on, then the screen rattled and prompted for the numeric passcode. Ugh..

Using Siri essentially required me to use my hands anyway, so what's the point of voice?

I'd very seriously consider moving away from iPhone to a device that treats voice AI as a first class citizen (presently I mapped the 'double back tap' to open grok voice chat, and triple back tap to end it, which is a wonderful improvement over not having these, as you can do that pretty easily, even while driving etc).

Unacceptable given it could be life and death. Understandable that clients' coverage struggles courtesy of Philippines' patchy cellular networks, but far less forgivable that a server for a critical emergency website has problems.

Very cool. Small (almost so small as to be silly) suggestion. But my first instinct as a non-classical music listener was to copy the names of songs into youtube so I could see if I recognised them. But for whatever reason I can't select the text (on desktop, not sure if different elsewhere). Could be cool to allow the text to be selectable, or even better link directly to it if it's on yt or other platforms/places.

Current policy is victim-blaming: it excludes from social media potential victims (children) but allows perpetrators (convicted csam users).

But this was always about governments wanting to know who's posting what (and controlling them, through chilling effect); not about saving 'the children'.

Re: Rocket Man Bad

Humans evolved to crave schadenfreude because curiosity and desire for justice are both helpful to reproductive success in group settings (e.g. tribes).

Media 'hacks' this evolved instinct for schadenfreude by manufacturing bad people so readers satiate their craving for 'justice' by gorging on the missteps of others.

This is harmless entertainment in works of fiction, but destructive and counter-productive in reality.

I see your point, that donors could influence political appointees to nix certain research topics for their own benefit.

How often does that actually happen, and wouldn't other institutions pick up the slack in most cases? (i.e. high value research doesn't cease to be high value just because one type of grant or institution refuses to fund it; it would therefore be attractive to other institutions/researchers)

Some benefits of having political appointees in the loop are that the pubic perceives (not necessarily 'gets') greater value from public research funding, and the people responsible for the funding (political appointees) are closer to the actual spending and are more involved in the allocative process, which should mean fewer expensive, hard-to-justify topics.

Being curious definitely leads to discoveries. But important discoveries can also be made by saying "Topic X, if better understood, might lead to a cure for cancer - let's look into (and fund) that".

We could think of this problem as a slider from 0-100 where we allocate from 'none' up to 'all' our research budget to curiosity-driven research.

Political appointees having a say will likely move the slider toward the 0 (not necessarily to zero). I'm just not sure it's a bad thing.

This means research projects will be optimised for political boasting.

Sounds terrible, but is it? It incentivises high-impact research (otherwise politicians can't boast about it), and less research into trivialities that common sense says aren't worth the public funding.

A "just say yes" attitude leads to certain disaster

Disaster's a possibility. But if an idea has a 1% chance of success, "just say no" usually assures failure, whereas "just say yes" is a shot at that 1% chance.

think of this as the just-say-no engineer, as opposed to the just-say-yes engineer. The just-say-yes engineer is obsessed with moving fast, approves code changes by default, values MTTR over MTBF, and tends to ship a lot of code. The just-say-no engineer is obsessed with quality, is happy to move slowly, and blocks code changes by default.

Love the concept of the 'just-say-yes' engineer vs 'just-say-no' engineer (and corresponding prioritisation of MTTR over MTBF).

I'm definitely a 'just-say-yes' with the caveat that bad architectural choices can be super painful to fix later, and features become a lot harder to fix when they have users as opposed to before launch (so I'm a little bit 'just-say-no', or at least 'just-think-for-a-bit-first').

I also think the balance between 'just-say-yes' and 'just-say-no' really depends a lot on the project. If it's finance or healthcare, perhaps 'no' by default is best. But if it's a silly startup idea, YOLO.

I see the benefit (it avoids the “works on my machine” problem), but my rails app isn’t too fancy and works on heroku ~100% of the time when it works on the dev machine. Making an intermediate build redundant (technically not entirely but it’s just not worth the effort).

For small teams it could be as simple as everyone agreeing to ensure tests pass on main before pushing to prod.