Malicious npm packages detected across Red Hat Cloud Services

https://github.com/RedHatInsights/javascript-clients/issues/492

by kurmiashish • 2 months ago

775 453 2 months ago

jofzar• 2 months ago

'No Way to Prevent This,' Says Only package manager Where This Regularly Happens

Edit: some people don't understand that it's a defence to https://en.wikipedia.org/wiki/%27No_Way_to_Prevent_This,%27_...

matheusmoreira9• 2 months ago

All programming language package managers are vulnerable. They all have the exact same caveats as the Arch Linux User Repository. There are no trusted maintainers taking responsibility for things. Any random person can make an account and push packages.

the__alchemist• 2 months ago

I think this is a thought-terminating cliche, and false equivalences. Stating "This area where problems occur at a high rate is not a problem, as problems can happen elsewhere too" is a curt dismissal of a valid concern. It implies the course of action, rather than to address a high-problem area, is to ignore any solutions which aren't global, or equate it to lower-incidence areas.

You bring up a good point that this class of problem, or related ones can occur with other package managers. It was frustrating how long it took the Crates.io team (Rust manager) to address name squatting, in what appeared to be a "no perfect solution exists, so we won't act" line of reasoning.

ajross6• 2 months ago

While true, tarring Arch here is a little unfair. AUR isn't enabled by default. It can't even be used via the same package front end, and in fact the "official" usage model requires that you clone the source yourself.

Indeed, AUR is bad as a software distribution mechanism (really it's best understood as a proving ground for baby packages before they get real maintainers and distro blessing), but it's less bad than NPM which puts the malware in the trusted/default/automated path.

CBLT• 2 months ago

Eh, it's worse than that. The GP comment is repeating a joke derived from an Onion headline about gun control. Where the very poignant message is about political will to make change. However, the npm ecosystem is very much willing and has already made several changes. If we're going to engage in discussion instead of meme-posting, the GP should have (imo) included real commentary _in addition to_ the meme they really wanted to post. What is the policy they want? Why do they see the NPM ecosystem as still resistant to change?

throwwwll4• 2 months ago

Nix enters the room.

(Everyone claps.)

myaccountonhn• 2 months ago

It's far far harder to do something exploits like this in elm because effects are tagged. There are solutions out there.

saturn_vk2• 2 months ago

IIRC, go cannot run arbitrary code at build time, so that should not make it vulnerable

chuckadams• 2 months ago

The big attacks of today are spread across several package ecosystems: TrapDoor and Shai-Hulud have been hitting npm, pypi, composer, and crates with the same malware.

throwwwll4• 2 months ago

And all of them "thought" of security as an after-after-after-after-after-thought.

ajross6• 2 months ago

PyPI and Cargo are, 100%, vulnerable to this same class of compromises. That NPM sucks isn't a statement that everyone else doesn't.

kalcode3• 2 months ago

People make this joke often. It's package managers and how loose we are with installing them, not NPM.

Cargo,PyPi,Nuget,PHP has had these recent too.

It's not just only NPM. It's frequently repeated here just cause of the average bias against Node.

But this problem isn't isolated to NPM.

Defletter• 2 months ago

The problem is compounded with NPM though thanks to lifecycle scripts: yes, any and all package managers create a risk of supply-chain attack, but NPM makes it dangerous to merely open a project up in an IDE.

stingraycharles2• 2 months ago

How many package managers allow executing arbitrary code as part of the installation process by default?

latexr2• 2 months ago

It's frequently repeated here just cause of the average bias against Node.

It’s frequently repeated here because NPM is where it keeps happening over and over and over and over and over and over again.

philipwhiuk• 2 months ago

In short, the problem is `npm` not NPM.

Someone12343• 2 months ago

Let me provide context, since a bunch of people responding with "every package manager can be hit!!!" npm, by design, allows all packages to run package supplied arbitrary code as the logged-in user after an update completes.

That's an INSANE default. pnpm, by contrast, allows you to essentially "opt-in" only specific packages that need this (e.g. four out of thirty, in one of our projects). Then tacks on tons of other security settings, like minimum age, no trust downgrade, etc etc.

All attackers can attack packages by updating how a package functions; but npm is particularly problematic as it runs non-sandbox scripts as the calling user. Putting not just your project at risk, but your entire machine/network.

And this stuff has been known about for YEARS, they've taken no action.

dns_snek5• 2 months ago

since a bunch of people responding with "every package manager can be hit!!!" npm, by design, allows all packages to run package supplied arbitrary code as the logged-in user after an update completes.

This is semi-common and in no way unique to NPM.

sigmoid10• 2 months ago

Putting not just your project at risk, but your entire machine/network.

Between average hackers and extortion groups, foreign governments and state sponsored actors and last but not least my own government, I don't think there's much room left for non-compromised supply chains these days. Treat everything that can run foreign code as potentially compromized and keep everything compartmentalized. If you keep your crypto wallets or private banking info on the same machine where you do development, you're asking to get shafted one day. Or if you keep your big corporate github keys on the same machine where you do private weekend projects. It doesn't matter what you use in particular, even if some vectors are currently more popular than others.

bob10292• 2 months ago

Furthering the idea that not all package managers are the same, there are entire cycles of the moon where I don't open nuget once. Some ecosystems simply don't need to vendor out very often, and these are the ones where you generally find the least news like this.

In about 99% of cases, I have the option to pick between Microsoft, a 3rd party or myself. I'm picking that first option every time I can. If M$ can't handle it, I'm hand rolling it.

Dapper remains the only constant 3rd party dependency in my projects. I don't know how much longer this will last with LLM assistance. The frontier models are very good at writing repositories over arbitrary sql schemas with low level primitives now.

Kuinox4• 2 months ago

Mosts packages manager, allow that.

pnpm can still be exposed, afterall the worm simply have to wait you run tests locally.

ImPostingOnHN2• 2 months ago

Nearly every package manager I've ever used had post-install scripts. Most run as root, since that's what usually what the package manager runs as.

It's not unreasonable: you're already installing software, which presents risks. If post-install scripts were not a thing, a payload could still run because you ran the software you installed. Or because the installer added it to auto-run. Or because the installer placed it somewhere where it would be dynamically loaded all the time.

semiquaver• 2 months ago

they've taken no action.

Not running lifecycle scripts by default is eventually going to be the default behavior. Late is worse (edit: I meant better) than not at all. https://github.com/npm/rfcs/pull/868

insanitybit8• 2 months ago

That's an INSANE default.

It's also the standard, and by far it's the contrast to not allow this. pnpm has a massive advantage of being the non-standard package manager, npm does not have that - what do you suggest that npm does?

chrisweekly3• 2 months ago

Yes, this.

Regarding npm CLIENTS, PNPM is fundamentally different from (and superior to) npm or yarn.

Strongest possible recommendation to use pnpm.

It's also a good idea to use a private registry (eg via jfrog), acting as a proxy / pull-through cache, and point trad SAST and maybe AI scanners at it.

But dropping the npm client in favor of pnpm is a no-brainer. Speed, disk space, security, determinism, flexibility, fine-grained control over your dependency graph...

matheusmoreira9• 2 months ago

allows all packages to run package supplied arbitrary code as the logged-in user after an update completes

As opposed to the completely untrusted package supplied arbitrary code that the logged in user executes when they actually use the package immediately after installing it?

Romario77• 2 months ago

I think another thing that affects security is that in javascript culture people often tie to the latest version instead of concrete version.

This makes it so an update to a popular library can compromise a huge number of packages that depend on it.

In Java for example almost all packages specify a concrete version, even if someone compromises the latest the blast radius is usually pretty small.

westoque• 2 months ago

i've been thinking about this as well. but having built a startup, i've learned that users don't care as long as they are given the value and most convenience. they don't really care much at security as much as we do. just look at openclaw? but maybe it's our job to make sure it is taken care of vs assuming the user cares and just make it look seamless.

rixed2• 2 months ago

One hour ago, while looking casually at a package.json, I saw this and was horrified:

  rm -rf pkg/snippets & rmdir pkg\\snippets /s /q & wasm-pack build --target bundler && node prepare-web.js

Looked like a strange mix of unix shell and msdos batch that would, on my box, try to rmdir "/s" and "/q". I asked Claude about this, and he replied something like "Yes that's a standard and clever hack to delete a directory that works both on linux and windows!".

Poor Claude has been trained on so much awful human code that it required several prompts for it to admit that there was indeed a problem.

The industry is the process by which convenient crap like this gets standardized.

PunchyHamster3• 2 months ago

Let me provide context, since a bunch of people responding with "every package manager can be hit!!!" npm, by design, allows all packages to run package supplied arbitrary code as the logged-in user after an update completes.

Many package formats before NPM allowed for it, and frankly, it matters little, because if it can add code to your app it can run malicious code. The fact it executes on package install rather than when dev runs tests or the app matters little, and in general if environment is sandboxes, the package install is also ran in the same sandbox so disallowing it changes little.

so yes, every package manager can be hit, the reason is twofold

* JS is such a lowest common denominator it has that much more clueless users so just by scale every issue will be more common than in other languages

* extreme fragmentation leading to hundreds of packages needed for even small projects, which is again more chances for compromise

tonymet2• 2 months ago

So does dpkg & rpm

tln• 2 months ago

Maybe NPM is scared to break a ton of packages? I also think action from NPM on the repo level is vital

I went through the package.json on my machine - seems like ~400 / 60000 or 0.7% have (pre|post)install. (That's not all of the scripts that run at install)

Seems to me like a backwards compatibility is a non argument since pnpm is popular enough to stand as existence proof that scripts can be, at least, opt-in

IMO - pre- and post- install scripts should just be abolished/deprecated. It should require a special dispensation from npm to even publish one. A better system for binaries (needed by esbuild) is probably needed.

Even saying "just use pnpm" isn't enough, we need to get the developer community to herd immunity and that isn't going to happen on an opt-in basis.

I would love for npm to sandbox as well. But I think the better way forward is just turn off scripts.

pajko• 2 months ago

So do the pre- and post-install scripts of Debian packages. The problem is not this but the lack of verified and controlled release channel.

jmull• 2 months ago

That's an INSANE default.

I agree that not running arbitrary installation scripts is the right default, but it's just an incremental improvement.

The practical difference between code that runs at installation and code that runs when the package is executed is, very typically, a small amount of time.

IMO, the hyperbole here hurts because it distracts from more effective efforts.

halapro• 2 months ago

npm is so bad at everything it's insane.

SIXTEEN YEARS of development and they can't even resolve a tree of dependencies in the correct manner unless you nuke the lockfile and node_modules.

Dependency resolution is literally the number one task and they fail at it. How can you expect them to be good at anything else? Absolute joke.

bakkoting2• 2 months ago

They have taken action as of very recently. The latest version [1] of npm warns when there are install scripts and tells you they will be disabled by default in a future version, with a per-dependency opt in mechanism [2].

[1] https://github.com/npm/cli/releases/tag/v11.16.0

[2] https://github.com/npm/rfcs/pull/868

tardedmeme• 2 months ago

Every package manager, by design, allows arbitrary code execution after the update completes. It is the entire purpose of a package manager. There is no point installing code that does not run.

TZubiri3• 2 months ago

tbf this is happening with a lot of package managers now, including pypi and composer

junon2• 2 months ago

Please stop posting this on every single security incident thread with npm. It was funny once, it's just rehashing the same debate over and over.

da_chicken• 2 months ago

On the other hand, if the same problem keeps happening, it's hard to argue that the problem isn't foundational to the design and that it should be called out until either the problem is fixed or the design abandoned.

chipdale• 2 months ago

Why should they stop? Maybe they want to rehash the issue that's not being adequately addressed. Maybe it's not supposed to be funny.

How do you propose we address this issue? Instead of policing what people say, are you interested in sharing your or someone else ideas?

rectang3• 2 months ago

Opponents of gun control surely feel the same way about the Onion’s story.

nailer4• 2 months ago

I've deleted and am rewriting this, to be more explicit, because HN downmodded the first comment to hell but I know I'm right and the crowd is wrong.

So, explicitly:

- pip

- Cargo

- apt/dpkg

- dnf/yum

- Homebrew

- RubyGems

- Composer (limited)

- Maven

...all allow scripts.

We understand the reference, it's just not correct: most package managers allow scripts, npm is the most successful package manager.

npm shouldn't allow scripts, but exploits happen everywhere.

m4rtink3• 2 months ago

If DNF/RPM is used there will often be a separate distro maintainer that should ideally review any changes coming from the upstream before pulling them into the distribution.

Also not all maintainers always pull in the latest upstream changes, only rebasing to new stable release or when the new features or fixes are actually needed for the distro stack.

Definitely not bulletproof but still IMHO more robust than "Lets just spray latest code from upstream without any review directly to production with a firehose!" that seems to be the norm.

matheusmoreira9• 2 months ago

HN downmodded the first comment to hell but I know I'm right and the crowd is wrong.

Got downvoted for saying it too. Don't let it discourage you.

xienze• 2 months ago

Maven does not run arbitrary scripts just by including a dependency.

krautsauer• 2 months ago

Are scripts even necessary? I don't think e.g. mvn has any form of scripts¹, but if the dependency is compromised, you're likely to execute whatever compromised code is in there the next time you do mvn verify (or whatever). Slightly less wormable maybe, running tests or at least checking whether your thing still runs after upgrading package versions is really common, no?

¹ Annotation processors are a thing and somewhat similar to rust macros in function, but you need to set those up manually for each dependency, iirc.

wang_li• 2 months ago

It's not the package manager, it's the repo and the cryptographic signatures that are trusted by the package manager and the users who choose to point their pacakge managers at those repos. The fundamental problem here is that people's risk assessment is treating a user named devioustiger12345 as having the same situation and story as Microsoft/Apple/Red Hat.

bananamogul• 2 months ago

You didn't include Perl's CPAN, which I think is older (1995) than all of these. And it allows scripts as well.

latexr2• 2 months ago

There’s actually a blog post with that exact title.

https://kevinpatel.xyz/posts/no-way-to-prevent-this/

https://news.ycombinator.com/item?id=48155690

paulddraper4• 2 months ago

Last 30 days:

PyPI, May 11. [1]

Crates.io, May 22 [2]

Composer, May 22 [3]

[1] https://www.tenable.com/blog/mini-shai-hulud-frequently-aske...

[2] https://socket.dev/blog/trapdoor-crypto-stealer-npm-pypi-cra...

[3] https://phoenix.security/laravel-lang-composer-supply-chain-...

tonymet2• 2 months ago

Npm developers can relate to Windows being a target because it’s the most popular package manager.

Why would you target xyz pkg niche manager knowing that only 200 people will install them?

NPM does perform active offline & online vuln scanning on the packages. Everyone can do more, but they are going to be the #1 target.

xena• 2 months ago

Ask and you shall receive: https://xeiaso.net/shitposts/no-way-to-prevent-this/supply-c...

EGreg• 2 months ago

https://qbix.com/blog/2026/04/01/no-way-to-prevent-this-says...

buckle8017• 2 months ago

Redhat's entire reason for existence is to prevent this.

dada216• 2 months ago

not really, no.

rob_c2• 2 months ago

So why else do we pay someone to package and certify/verify open source projects? This is absolutely 90++% of what should be RedHats core day job.

cozzyd5• 2 months ago

did RPM packages get compromised?

dmix2• 2 months ago

Our company uses yarn 4 which has an option to prevent you from installing an npm package for the first number of days of its release. Most of these seem to be caught within that timeframe (1-3 days).

https://gist.github.com/mcollina/b294a6c39ee700d24073c0e5a4e...

phoronixrly3• 2 months ago

What happens when everyone adopts this policy? You just change it to two weeks?

Tomte• 2 months ago

You rely on the security companies scanning the packages.

BoredPositron• 2 months ago

Always one day more than people on HN tell you. If something is compromised you will hear people complaining here that three days is not enough.

JoshTriplett• 2 months ago

A large array of automated and semi-automated security scanners are finding things quickly. The main benefit of waiting before updating is to give those scanners time to work.

blm1262• 2 months ago

The one week cooldown option is not relying on other users to be a canary for you. Its just giving automated scanners a chance to notice. This is the perfect example. I don't think step security found this by accident. They are actively monitoring NPM package releases at some level.

There is something to be said that Microsoft should be scanning packages pre-release. They aren't, though, so for right now there is a ton of value with very little downside if people implement a one week cooldown period.

To answer your question directly, though. If everyone else moves to a one week cooldown, I would absolutely suggest a two week cooldown is a good idea. Being the "slow" moving organization is a good security trade-off so long as you don't take it to extremes and have escape hatches when you actually need to be moving quickly.

bakugo• 2 months ago

This will never happen unless it's made the default. Most people will always stick with the defaults.

Ukv2• 2 months ago

Security scans and authors realizing an unauthorized version was pushed will generally happen regardless of whether regular users updated. Even for compromises that are found by users updating, it'd generally be better to reduce the number of people affected with a slow roll-out rather than everyone jumping on at once.

iwhalen• 2 months ago

uv supports the same for any Python developers out there: https://docs.astral.sh/uv/concepts/resolution/#dependency-co...

darth_avocado4• 2 months ago

And somehow poetry doesn’t in 2026.

KORraN• 2 months ago

pip has recently added a similar option, i.e.

`pip install --uploaded-prior-to P7D pre-commit`

https://pip.pypa.io/en/stable/cli/pip_install/#cmdoption-upl...

no-name-here4• 2 months ago

Sadly I haven't seen that Visual Studio/Rider/dotnet/VS Code have such a feature for the C#/dotnet/nuget ecosystem.

darth_avocado4• 2 months ago

There is something to be said about the need to keep all the packages as the latest and the greatest at all times. Every minor version update doesn’t need to be immediately applied. And maybe high and critical vulnerabilities don’t need to be a minor version upgrade.

Waterluvian• 2 months ago

I’m having a real problem at work with security theatre and the growing push to obsess over numbers of “vulnerabilities” in our projects. And then auto Dependabot PRs that encourage churn to fix issues that if an informed person actually reviews easily concludes it doesn’t affect us in the slightest.

chrisweekly3• 2 months ago

"maybe high and critical vulnerabilities don’t need to be a minor version upgrade"

huh? what do you suggest instead?

mihaelm2• 2 months ago

`pnpm` also has that and it's on by default since `v11`:

https://pnpm.io/settings#minimumreleaseage

vinnymac3• 2 months ago

It’s on by default in yarn 4 too now, but pnpm was the first to market that default minimum gate.

https://github.com/yarnpkg/berry/pull/7135

olejorgenb3• 2 months ago

pnpm also support this

dmix2• 2 months ago

The gist link above covers how to use it in yarn, npm, and pnpm

kylebebak• 2 months ago

npm supports this now as well, with e.g. `min-release-age=7` in `.npmrc`

winrid2• 2 months ago

not if you have internal repos?

bram98• 2 months ago

Why not the other way around, I man by default and can be changed.

0xbadcafebee2• 2 months ago

The package event-stream was compromised and went unnoticed for 60 days: https://medium.com/intrinsic-blog/compromised-npm-package-ev...

The package axios was compromised, and hijacked the author's credentials, so every attempt at a fix was unfixed. https://www.trendmicro.com/en_us/research/26/c/axios-npm-pac...

The xz utility was backdoored for 2 months: https://gigazine.net/gsc_news/en/20240403-timeline-of-xz-ope...

A student researcher took over Python ctx and PHPass package maintainership, pushing out malicious changes, and that took over 7 days to be detected and fixed: https://infosecwriteups.com/how-i-hacked-ctx-and-phpass-modu...

Kaspersky found multiple PyPI packages that had been exploited for more than a year: https://www.kaspersky.com/about/press-releases/kaspersky-unc...

"LoftyLife" packages were exploited for several months: https://securelist.com/lofylife-malicious-npm-packages/10701...

Now that the attack window has changed to 7 days, all new exploits like these will come with time bombs to not trigger until 8 days.

Sayrus• 2 months ago

Now that the attack window has changed to 7 days, all new exploits like these will come with time bombs to not trigger until 8 days.

Many automated scanners use static code analysis rather than run the installation script. Not all of them are caught, but a good part of them are and you'd be saved by a delay.

pixl974• 2 months ago

Instant attacks are much easier and more common than delayed attacks. Security is an onion.

throwawayffffas3• 2 months ago

Ok so no external dependencies, or pull versions from 2021 and apply security patches?

wg02• 2 months ago

If everybody starts to delay for 3 days, wouldn't it be the case that everyone would discover it on the 3rd day?

cortesoft• 2 months ago

Vulnerability scanners and security researchers would be looking those first 3 days

pixl974• 2 months ago

Most attacks are discovered 'pretty quickly' via scanning services and groups that monitor repositories. The problem is even an hour gap could mean tens of thousands of downloads and executions.

freakynit2• 2 months ago

Lol.. yet again npm and install-scripts abuse at play.

Updated:

1. All exploitation techniques used since May 2025: https://npm-supply-chain-attack-techniques.pagey.site/

2. All attacks that happened since May 2025: https://npm-supply-chain-attacks-25-26.pagey.site/

indy• 2 months ago

This is a completely unexpected turn of events that no one could have possibly foreseen.

m3kw9• 2 months ago

At some point, they need a new system for these "packages", you've got to be insane to install any of these right now.

arianvanp• 2 months ago

Given they use nx my bet is on developer laptop compromise through the nx vscode extension that also compromised GitHub engineer's laptop

dist-epoch3• 2 months ago

the security of their packages should not depend on one laptop being compromised

gbuk20132• 2 months ago

I came across this interesting rant the other day: https://github.com/uNetworking/uWebSockets.js/blob/master/mi...

It does make sense that the right way would be to fork every dependency you use and install from your own repo reviewing and merging from upstream as needed. Would be a giant PITA though. :)

twodave• 2 months ago

For smaller shops (by small I mean <1,000 employees) this isn't even tenable. We (engineering team of about 10 people) mitigate what we can via tooling and cooldown periods/minimum release age. This will work as long as these malicious packages remain reasonably detectable. I think that's the proper balance, because we can adjust the # of days we are willing to risk against the SOTA of detection tooling.

dboreham• 2 months ago

I think the general idea that your supply chain should be rooted in source repositories and associated commit hashes is the right one. Tooling can be made to automate the process of putting together a product from those defined sources. Some languages/systems already have some support for this. E.g. Golang and Rust. The concept of a "binary" artifact is really dead now everyone uses git and builds are quick. It lives on in things like npm and docker hub but we don't actually need it.

Zardoz843• 2 months ago

DUB , for D Lang does that.

Cthulhu_• 2 months ago

Nothing that couldn't be automated; in Go land this is (arguably) called vendoring (https://go.dev/ref/mod#vendoring). Good to offload or reduce dependencies on 3rd party dependency hosters, pull a dependency into your own code review tools, and to ensure reproducible builds long term.

gbuk20132• 2 months ago

I mean there’s nothing stopping you from committing node_modules to git (after running something like https://github.com/timoxley/cruft on it) and reviewing code changes on dependency updates.

I even managed to make that part of the workflow on one team I worked with but several other teams since thought it was a crazy idea. :)

Izkata• 2 months ago

It's a generic and very old term for committing dependencies to your repo, it's not go-specific.

Bilal_io• 2 months ago

The problem would be the dependencies of your dependencies, and keep going many levels.

pipo234• 2 months ago

But presumably, you only include dependencies that you trust and those dependencies themselves do their trusting more strictly than you. Trust is built on vetting, signatures and reputation.

That is, at least what we do, in theory. In practice, we cross fingers and let the LLM pick dependencies, are satisfied if it just works and we either update our deps frequently or infrequently.

nafey2• 2 months ago

The problem is that node.js doesn't have a good standard library so one must rely on external packages to build even basic apps.

repelsteeltje• 2 months ago

That might change the odds, but unless you fork diligently (and monkeypatch each and every future vulnerability) you might ship a compromised fork forever.

olejorgenb3• 2 months ago

Except most of the attacks so far has not landed actually source code changes to git IIRC. They have targeting the release files directly.

paulddraper4• 2 months ago

The problem is the volume of dependencies. Most modern JavaScript, Python, Rust, Go, etc. projects have many dozens of transitive dependencies.

mbreese• 2 months ago

Let’s not ignore that dependencies are far more common in JS than any of those other languages. My Go or Python projects generally only include a handful of external packages. Node projects on the other hand…

marcosdumay• 2 months ago

Hum... You should check your JavaScript numbers again.

I have never seen a project that uses npm and has only dozens of dependencies. Normal numbers are in the 10s of thousands (including different versions of some deps).

nullsex3• 2 months ago

So be more selective with your dependencies then. Next.js is usually a mistake.

whynotmaybe• 2 months ago

Shouldn't you also restrict version number for a package not just the latest?

ashishbijlani• 2 months ago

Built Packj [1] to audit dependencies easily from CLI.

1. Packj (https://github.com/ossillate-inc/packj) detects malicious PyPI/NPM/Ruby/PHP/etc. dependencies using behavioral analysis. It uses static+dynamic code analysis to scan for indicators of compromise (e.g., spawning of shell, use of SSH keys, network communication, use of decode+eval, etc). It also checks for several metadata attributes to detect bad actors (e.g., typo squatting).

nullsex3• 2 months ago

Some of us have been doing this for years or decades, including for our desktop OS.

Gentoo graybeards know.

what_hn• 2 months ago

Same actors again?

rvz• 2 months ago

This repository itself had to previously update from the axios supply chain attack [0] (co-authored by Claude lol). But just by looking at the change itself, the package is unpinned and won't solve the problem if another malicious security update happens again.

So if you have an unpinned version of this package and you run 'npm install', you immediately downloaded the compromised version and that's that.

[0] https://github.com/RedHatInsights/javascript-clients/commit/...

general_reveal3• 2 months ago

That’s why I switched to Java.

UqWBcuFx6NV4r• 2 months ago

…. lol

keyle• 2 months ago

    AbstractFinalFactoryShaiHuludSerialisedFactory

general_reveal3• 2 months ago

Yeah but you don’t have to use that I think. I think us Node people can just pretend to write Ecmascript 2 in Java and be fine.

exabrial3• 2 months ago

https://dayssincelastjavascriptframework.com

Rp8yXmdmr• 2 months ago

You are absolutely right. The dangerous part of NPM packages is the post-install script. Therefore moving from JavaScript to Java removes the threat.

OrangeMusic2• 2 months ago

You joke but, yeah, when you think about it, the problem with Javascript is the 'script' part. That's actually correct.

mschuster91• 2 months ago

Meh maven plugins are just as juicy a target as npm is

general_reveal3• 2 months ago

What do u recommend?

exabrial3• 2 months ago

https://github.com/s4u/pgpverify-maven-plugin

If you want paranoid mode, you can verify literally every part of the maven build process.

paulbjensen• 2 months ago

Looks like RedHat got compromised by a Black Hat…

phishin• 2 months ago

Chainguard based images, packages and libraries are first line of defense. Expensive? Yes. Foolproof? No. I think these types services will be mandatory in the near future.

dralley• 2 months ago

How would that help? These are not general purpose, base system libraries, these are libraries specific to a product that uses them. Either you're not using them and hence they would not be installed in the first place, or you're using them because you have the product installed.

Though I would expect that Insights uses RPM packages to ship components and not the public NPM packages.

SSLy• 2 months ago

it wouldn't surprise me if insights was in fact a wrapper around npm install

king_zee• 2 months ago

I've made it a habit now to use the --before=2026-05-30 flag when installing packages, where it'll pick the version released before the date you specify, I usually pick around 5 days ago

sourcegrift• 2 months ago

Yarn 4 can automate this

vinnymac3• 2 months ago

In case others are unaware, you just have to set https://yarnpkg.com/configuration/yarnrc#npmMinimalAgeGate to the value you want. It defaults to 1 day.

nullsex3• 2 months ago

If supply-chain security is a concern yarn is the worst js package manager you can pick. It comes far down their priority list, below "just make things work without need for user input". Whatever you thought you configured will simply be ignored many times and that's considered a feature.

Go look in that projects issue tracker and commit log for changes to relevant configuration and you will know what I mean.

Even yarn 1.22 is a safer choice.

mihaelm2• 2 months ago

I just use `pnpm` and set up a liberal `minimumReleaseAge`: https://pnpm.io/settings#minimumreleaseage

Thankfully, it's on by default since v11.

sync• 2 months ago

If using straight npm (v11.10.0 or higher), you can just add to .npmrc in the project root:

min-release-age=5

t-sauer• 2 months ago

If you use npm 11, you can simplify your workflow by setting min-release-age to 5. https://docs.npmjs.com/cli/v11/using-npm/config#min-release-...

Sudhanshu2310• 2 months ago

We have done the complete analysis and there are 32 packages share the same publishing pipeline. https://safedep.io/redhat-cloud-services-hit-by-mini-shai-hu...

voidUpdate2• 2 months ago

One thing I've never understood is why NPM allows packages to run code immediately after they are installed. What's the use case for that? A package should just be some code you can call on at runtime

tom13373• 2 months ago

Some packages need to build native dependencies. sharp for example needs to build libvips on the system [0] to work

0: https://github.com/lovell/sharp/blob/main/install/build.js

vinnymac3• 2 months ago

I’ve always felt this automation shouldn’t exist at all, but should rather be selectively controlled via a hook. The hooks yarn offers out of the box for example can be used to run any code you need to after install. Putting the project owner in control instead of the dependency.

yread• 2 months ago

Nuget/.NET ecosystem just handles it so much better. Netvips assumes libvips is available and they provide packages for common platforms. No need to waste electricity rebuilding stuff, or install native build chains, build and test deps. Similar for Skia or Sqlite or whatever.

jcupitt4• 2 months ago

sharp does not rebuild libvips, it downloads a pre-compiled libvips for your platform.

https://sharp.pixelplumbing.com/install/#prebuilt-binaries

It can usually also download a precompiled binary for the C++ shim that sits between node and libvips, but if your node / arch / etc. is not supported, it'll compile that (that's what the build.js file you linked does).

mark_l_watson• 2 months ago

I turn off running scripts on installation. So far, no inconveniences.

jcupitt4• 2 months ago

Some packages with native code components (like sharp) will use these hooks to download the correct precompiled native binary for you.

shrikant• 2 months ago

Oooh now I'm wondering if this may have contributed to their Docker image distribution service getting disrupted earlier today... https://status.redhat.com/incidents/jn6r256zc62c

bobkb• 2 months ago

When will npm issues stop ? This has become a big pain !

kitd• 2 months ago

Hmm, same day as RH and IBM announce Project Lightwell to help detect and fix supply chain vulns.

https://www.redhat.com/en/lightwell

jamietanna• 2 months ago

That was a few days ago: https://news.ycombinator.com/item?id=48313577

tetsgima• 2 months ago

man we gotta do smth with preinstall hooks atp

mellosouls• 2 months ago

Should link to the original announcement I think:

https://www.stepsecurity.io/blog/multiple-redhat-cloud-servi...

nailer4• 2 months ago

Yes. Also Red Hat is misspelt in the title.

dist-epoch3• 2 months ago

if RedHat is unable to secure their packages, what can we expect from mere mortals...

cozzyd5• 2 months ago

this looks like another GitHub Actions workflow compromise...

Is it really so hard for people to make releases manually?

Havoc• 2 months ago

The entire ecosystem is cursed

thewebguyd• 2 months ago

Always has been. I remember poking fun at it 15+ years ago (queue the 'MongoDB is web scale' meme video).

kittikitti• 2 months ago

I'm refactoring all my personal and research projects to utilize pure HTML/CSS without any dependency of JavaScript. This was always on the table but the cybersecurity risks from all programming languages and frameworks have increased due to AI.

I know of fundamental issues with JavaScript and see no reason why it's still standard on all web browsers.

MadrasTh0rn2• 2 months ago

Fucking Microsoft

hsibenMohamed• 2 months ago

Salam

bpavuk• 2 months ago

Violence!

grugdev422• 2 months ago

The joke is on you NPM! I only use CDNs for my JS libraries.

iconicBark• 2 months ago

Is this more secure?? I would genuinely love to know

bdcravens2• 2 months ago

Yes, none of npm's lifecycle hooks. You're just pulling bytes over the wire.

n_e• 2 months ago

Yes (assuming they're doing frontend dev and including the resources from the page). The code is fetched and executed from the browser, so It'll have to escape the browser sandbox to do something nefarious.

lostmsu• 2 months ago

Same. I came back to do a little frontend work a couple of years ago and was horrified by the replacement of script tags with subresource integrity with npm and bundlers.

insanitybit8• 2 months ago

Just some suggestions:

1. Dependency cooldowns of 1-2 days seem to be extremely effective without negatively impacting your ability to patch for CVEs.

2. Anywhere you have `npm install` or `npm test` or anything where code executes, that should happen in an environment that has no privileges. In your github actions you can do this semi-straightforwardly by using two separate jobs - one to build the artifacts and test them, another to do any sort of publishing, signing, etc. If you use AI, add a skill / guidance to enforce this pattern.

3. If you use Github Actions, install the latest version of zizmor. It will significantly improve your posture.

(2) means that you are no longer "wormable", which is a massive part of the problem that we have today. (1) gives companies more time to respond to the attacks.

There are some vendors in this space that you can and should evaluate as well.

tmpz22• 2 months ago

anything where code executes

ALL the agentic orchestrators like codex, claude-code, etc. seem to do this by default.

ffemac3• 2 months ago

Exactly, popular AI coding harness (OpenCode/KiloCode) downloads random npm packages in the background without you knowing. What's worse is the devs don't care.

sureshpaulchamy• 2 months ago

With coding agents, npm install can become a side effect of "make the tests pass." That feels like a very different risk model.

pixl974• 2 months ago

install the latest version of zizmor.

What if it gets compromised?

More of a joke. But was funny after saying that new packages should be delayed.

insanitybit8• 2 months ago

lol yeah I thought of that as typing but figured I'd avoid the complexity. "latest version" means, give or take, whichever the latest one was that contained a bunch of new rules around supply chain stuff.

spockz2• 2 months ago

Should we instead of these cooldowns just run builds in isolated contexts?

I’m running a maven proxy locally. All builds happen inside containers. I only use public repos for python, npm, and go. So these builds happen also in containers but don’t need a repository proxy.

insanitybit8• 2 months ago

Should we instead of these cooldowns just run builds in isolated contexts?

I'd suggest both. Cooldown for 1-2 days is very cheap and you likely won't even notice it, so it's quite harmless and from what I've seen even just 24 hours is enough to let security companies pick up malware.

But yeah, isolation is a must-have.

rochak• 2 months ago

If this is what will take for folks to move away from JS ecosystem, I'll take it.

renox• 2 months ago

Bah, I think that these kind of vulnerabilities exist in any "packaging ecosystem" where the base language offer "ambient authorities"(any library can access your filesystem) which is .. all of them! AFAIK only research languages do not provide these ambient authorities :-(

jollyllama• 2 months ago

This x1000. This is the culmination of 15 years of frontend dev culture. Why does RedHat even have an NPM repo?

kogasa240p2• 2 months ago

Seconded

czbond2• 2 months ago

I am not a JS dev, but had to interact with the ecosystem some. It became so bad I won't install anything without it being in a Docker or Podman container.

OrangeMusic2• 2 months ago

I honestly don't understand how you could do JS on the backend in 2026. This language and ecosystem are so bad it's ridiculous. Almost all other options (yes, even PHP) are better.

On the browser at least you have the excuse that there's no other option (hoping Wasm will eventually kill JS for good, but we're not there yet).

anoncow• 2 months ago

This seems to be sinister

lepuski• 2 months ago

Qubes offers the strongest protection against these threats. It's surprising it isn't more commonly adopted.

no-name-here4• 2 months ago

https://qubes-os.org

exabrial3• 2 months ago

NPM broken by design. And the NIH syndrom that runs rampant in the community wont let them do anything simple.

beart2• 2 months ago

I don't follow your second sentence. Doesn't npm have the opposite problem of 'not invented here'? By adopting many external packages rather than developing in-house, npm projects tend to have large, complex dependency trees. It has long been the complaint that packages such as https://www.npmjs.com/package/is-windows create potential vulnerabilities and maintenance headaches, when writing the same piece of code directly is so simple.

TZubiri3• 2 months ago

One common fallacy of the NIH folk is that reinventing X package would take a lot of time.

But first, you will of course not remake every single feature, just the one you need.

And furthermore, when you code just one feature, you don't need to make any abstraction or additional function interfaces. So it's cheaper, and probably better integrated.

Another fallacy is that you'll make bugs and introduce vulnerabilities. Maybe, if you are a bad programmer, but you will also avoid a category of bugs where the vuln is introduced at the boundary of the integration between two different libraries that weren't designed to fit exactly together. (Many such cases)

majorbugger• 2 months ago

I would like to meet the person behind the "postinstall scripts" idea and try to understand how they thought it was a good idea.

MadrasTh0rn2• 2 months ago

Y'know just to talk a little bit

ex-aws-dude• 2 months ago

Has anyone thought of having an agent review all dependency upgrades before upgrading?

I feel like that would at least catch some of these

asxndu• 2 months ago

This outsourcing of thinking is why such issues will continue to happen and possibly accelerate.

How about simply coming up with new standards that emcure such things don't happen at all?

insanitybit8• 2 months ago

Yes, I do this. It absolutely would catch some of these.

Pxtl4• 2 months ago

The combined features that make npm particularly vulnerable:

1) Update by default. Manually updating your package references is annoying and does lead to other security issues as you don't automatically get latest, but it makes this risk much lower.

2) Code executed on install. Statically-typed languages don't run the code until you use them, and that might not happen on the developer machine at all for first run after upgrade, it might be a lower-priv test-server.

3) Culture of many tiny modules (this is good! It's the natural way to fight NIH! Yay modularity!) means many more points-of-failure for security for this kind of attack.

replwoacause• 2 months ago

It's becoming laughable how frequently this is happening. Wow.

_pdp_3• 2 months ago

Why blame on NPM? Would you blame GitLab if an opensource maintainer was hacked and as a result the repo contains malicious changes?

All of these recent incidents is just developers doing stupid things ... like using their compromised devices for making production changes, which is basically a big red flag to begin with.

In fact, the entire situation has been exacerbated by coding agents because now practically everything happens on a single device that touches hundreds of different production systems with full production credentials.

gred2• 2 months ago

Days since last malicious packages in NPM: 0 (evergreen)

Days since last malicious packages in PyPI: 30

Days since last malicious packages in Maven: 120

I'm sure this isn't 100% accurate, and there are probably better metrics (average number of malicious packages per year, average number of developers affected per year, etc) but they aren't as easy as a quick Google News search.

_pdp_3• 2 months ago

Except that the JavaScript / NPM ecosystem is 6-7 times larger than Python and Java / Maven.

https://chatgpt.com/share/6a1da751-0d88-832e-ace7-572bc786e0...

Check the linked resource which has the actual data.

calvinmorrison• 2 months ago

no because I dont ship production software from gitlab, I use upstream maintained packages?

eranation7• 2 months ago

Hope it's ok I hijack this thread again about setting up cooldowns... (copy pasting my last comment when tanstack was compromised):

I know people have opinions about cooldowns, but they would have saved you from axios, tanstack, (+ @redhat-cloud-services) and many other recent npm supply chain attacks. If you have Artifactory / Nexus, you probably already have cooldowns, but it's easy to set up if you don't. Why cooldowns? Most npm (or pypi) compromises were taken down within hours, cooldowns simply mean - ignore any package with release date younger than N days (1 day can work, 3 days is ok, 7 days is a bit of an overkill but works too)

How to set them up?

- use latest pnpm, they added 1 day cooldown by default https://pnpm.io/supply-chain-security

- or if you want a one click fix, use https://depsguard.com (cli that adds cooldowns + other recommended settings to npm, pnpm, yarn, bun, uv, dependabot and, disclaimer: I’m the maintainer)

- or use https://cooldowns.dev which is more focused on, well, cooldowns, with also a script to help set it up locally

All are open source / free.

If you know how to edit your ~/.npmrc etc, you don't really need any of them, but if you have a loved one who just needs a one click fix, these can likely save them from the next attack.

Caveat - if you need to patch a new critical CVE, you need to bypass the cooldown, but each of them have a way to do so (described in detail in depsguard.com / cooldowns.dev) In the past few months, while I don't have hard numbers, it seems more risk has come from Software Supply Chain attacks (malicious versions pushed) than from new zero day CVEs (even in the age of Mythos driven vulnerability discovery)

Normal_gaussian4• 2 months ago

If you know how to edit your ~/.npmrc etc, you don't really need any of them, but if you have a loved one who just needs a one click fix, these can likely save them from the next attack.

This feels like a very very small group of people; and people who really could do with opening the file and adding the line.

eranation7• 2 months ago

I wish that was the case. Asking people to do something simple, doesn't matter how simple it is, depends on how simple they view it. Changing your own car's oil is actually not that hard, once you know how to do it, most people don't even try. Think of QR codes, people hardly used them for many years, because you needed to download an app for it, small step. It only started to catch up when you had it built in the camera app in most providers. In any funnel, each step, no matter how easy, adds friction, remove the friction and you get bigger adoption.

So yes, everyone could open a file and edit it, also everyone could watch a youtube video on how to do X and yet choose to have someone else do it for them :)

pavel_lishin• 2 months ago

A friend of mine has a github repo with references to how to set things up in sane and slightly more secure manner: https://github.com/jordanconway/package-manager-hardening

nicolewhite2• 2 months ago

From that repo:

Exact version pinning — specifying precise versions (1.0.0, ==1.0.0, =1.0.0, = 5.31.0) rather than ranges (^, ~>, >=) in package manifests. Ranges allow any version satisfying the constraint to be resolved at install time; exact pins mean only one version is ever valid.

My understanding is that pinning the dependency within the manifest isn't the mechanism that prevents the version from changing across installs -- it's the lockfile that accomplishes this.

doctorpangloss3• 2 months ago

Caveat - if you need to patch a new critical CVE, you need to bypass the cooldown,

by now, you should have received the feedback about why cooldowns don't make sense and why nobody is adopting them. look, you are writing an expression of the reason why right there.

eranation7• 2 months ago

I don't agree that nobody is adopting them. Can you please elaborate?

- Most companies I know have a 24 hours (at least) cooldown via their Artifactory / Nexus. They have ways to bypass it for urgent CVEs

- pnpm just adopted 24 hours cooldown as default, based on community feedback.

throwawayffffas3• 2 months ago

How often do you update your lockfiles? Where ever I have worked, it's once a year or whenever we get a critical CVE (in which case we only update the offending package and it's dependencies if required). Unless an attack is happening every day the chances of getting hit is slim.

make3• 2 months ago

theoretical question, do cooldowns still work if everyone has them?

ZiiS3• 2 months ago

Less well maybe but yes. Security researchers still proactively test them, and the maintainer has a much better chance of catching it themselves.

chillax• 2 months ago

Companies such as socket and safedep will still scan new packages and alert on malware (if they are able to detect it) so the packages are taken down before they pass your cool down

ffemac3• 2 months ago

No, it will stop working. The whole point of min age is letting someone else taste the food before you, so you are not poisoned. (except maybe scanners but they can't detect everything and the payloads will highly likely to remain dormant when it detected it's within a scanning env).

BTW it will only get much worse because popular AI coding harness (e.g. OpenCode/KiloCode) will just download random npm packages in the background without you knowing. And the devs don't care.

QuiEgo2• 2 months ago

As an embedded dev who’s used to locking to toolchains and deps for years at a time, web dev is a culture shock in so many ways.

smrtinsert• 2 months ago

There's literally no excuse - I'm still baffled.

schonfinkel2• 2 months ago

I once worked in a project that vendored most of its third-party dependencies, it was a culture shock at first, but damn, after a while it was so nice being able to work just by building from local source, with normal tooling like `make`, instead of pulling a shitton of deps from the outside world. Made me realize how much "webdev culture" did a disservice to software engineering as a whole.

k_sze• 2 months ago

Nice work!

Is there a sensible way to add a cooldown to Docker/Podman image pulling?

TacticalCoder2• 2 months ago

Hope it's ok I hijack this thread again about setting up cooldowns...

In addition to cooldowns it'd be nice if more package managers did triage between security fixes and normal releases (bug fix / performance improvement / new functionalities).

It's totally possible to say: "A security fix must only be a security fix and cannot ship any other feature".

Then, for a start, a security fix becomes easier to audit (both by security researchers and by the tools security researchers are using).

And then a cooldown can be used for the regular releases (e.g. non security-related bug fixes, perfs improvements, new functionalities, etc.) but no cooldown (or a much smaller countdown) for security fixes.

Something has to be said about a system like Debian where you can have an extremely stable server and you can configure unattended upgrades to only apply security fixes but nothing else.

Such new package releases are easier to audit by security researchers.

collinmanderson• 2 months ago

no cooldown (or a much smaller countdown) for security fixes.

A supply chain attack would likely able to publish a "security" release just as easily as a normal release, so I don't think that would help much.

naruhodo• 2 months ago

but if you have a loved one who just needs a one click fix, these can likely save them from the next attack.

I'm not letting gam gams anywhere near that shit. She can continue writing her own apps in assembly language - it's good for her brain health!

wg02• 2 months ago

Question - is there no way to catch these criminals?

a13n• 2 months ago

It’s difficult to determine which individuals are involved and even if you manage to do that they almost certainly live in countries without extradition.

bel8• 2 months ago

The XML extension I use in VSCode is by Red Hat.

Oh dear. Here we go again.

czbond2• 2 months ago

Podman? Podman for OSX comes with a login item from "Red Hat, Inc". Anyone know how to check if this subcomponent utilizes these npms?

Escapade5160• 2 months ago

Can someone give a tldr on why this happens so much with npm ? I can't recall seeing this with any other package manager. Is npm just the default used these days and therefore sees this more often?

Surac• 2 months ago

Npm is just borked by design. I hop it will take javascrip with it

kogasa240p2• 2 months ago

Throw the JS ecosystem into the sun at this point.

Noaidi• 2 months ago

Human society, and our technology, is a fragile system built on our hubris, a cheap replacement for the Divine Eye of Providence.

zeraye• 2 months ago

There is some effort to make NPM more secure https://github.com/npm/cli/pull/9360.

rectang3• 2 months ago

About a week ago, I uninstalled Node from my laptop, which felt great. :)

I'm trying to do all work in dev containers (or other sandboxes), limiting the blast radius if I'm unlucky enough to be hit by an exploit. The attackers may get a Claude token, but they won't easily be able to escape the container and scan my home dir.

Cooldowns and allow-listing of installer scripts are good additions to layered security, especially for CI. However, I think the fundamental thing that needs to change is the OS permissions model. The default of trusting third-party software with everything your user has access is no longer workable.

backwardsponcho• 2 months ago

Are you using something like Bubblewrap/Firejail/Flatpak, or what does such a setup look like? I've been entertaining a similar idea for a while but haven't gotten to it

rectang3• 2 months ago

I'm using VSCode dev containers, powered by Podman on a Mac. Most people would probably choose Docker over Podman but I'm weary of Docker and wanted to try something else. I would not consider myself an expert on containers but with the help of Claude I've been able to fight my way through various challenges:

* Persist a volume for Claude so that conversations don't get blown away with every container rebuild. An attacker may still be able to get a Claude token from me, which is something I'd like to tighten up in the future.

* Fix file permissions issues by running rootful inside the container. (The container process still runs on the host as an ordinary user. Since my threat model is "compromised dependency scanning for credentials in project dir and home dir" rather than "attacker escaping the container", I figured that was good enough to get started.)

* Work around architectural availability issues with precompiled PyPI libraries. This I punted on by choosing a different approach and eliminating the problematic dependency (by writing my hobbyist CAD 3d printing stuff using Blender extensions instead of CadQuery). I've gotten the impression that dependency compatibility with a container workflow is an ongoing challenge.

* Run a database in a docker-compose sidecar for integration testing.

For all the projects I'm containerizing I'm the solo dev with full control over the Git repo so I can make the call to add a `.devcontainer/devcontainer.json` config file. I haven't yet explored how to isolate projects I don't control.

mnahkies• 2 months ago

In every of these threads there's a bunch of snarky comments, either acting like this class of attack is exclusive to npm, or that nothing has been done about it. I don't think that's fair.

There's plenty of comments mentioning delay lines, and the other good stuff pnpm (and others) have implemented in response to protect package consumers.

That bit that's getting less conversation is the tools on the package maintainer side:

- MFA for publishing: https://docs.npmjs.com/requiring-2fa-for-package-publishing-...

- trusted publishers, available for about a year: https://docs.npmjs.com/trusted-publishers

And as of recently, staged publishing, essentially combining the best of both those features: https://docs.npmjs.com/staged-publishing

Now you can: - Publish from CI, without static credentials

- AND require a maintainer to approve it using MFA before it actually goes live to the registry

If you want you can still use something like GitHub Actions Environments protection to require multiple approvals, or a time delay, on the CI side.

We need to encourage the community to adopt these publishing protections or this will continue to be an issue.

selckin• 2 months ago

they need to make it required for everyone, and then they'll have done something

ajross6• 2 months ago

IMHO those are both lipstick on a pig solutions. Ultimately all this stuff is just a variation of "make releases harder to publish", which isn't going to do anything but train people to evade them. Notably, neither would have prevented the xz-utils backdoor from reaching package distribution, which remains the gold standard for sophisticated upstream compromise.

The bug here isn't that we need to better authenticate already-trusted upstreams for packages, it's that the upstreams cannot be trusted as the sole source for security at all. Upstreams are a bunch of hackers[1] who aren't really interested in, nor will ever be good at, solid release engineering practices.

But some people are! The solution in the Linux world (and the one that saved us from xz-utils) is that there is a second level of human beings responsible for reviewing, auditing, packaging, and customizing those hacker-generated upstreams for the benefit of their users. These people have different eyes, different consumer requirements and different quality metrics. And they catch bugs and malfesance that the upstreams aren't prepared to do.

NPM (and cargo/PyPI et. al.) continues to think it can short circuit this requirement for human labor. It can't.

[1] In NPM's particular ecosystem, a bunch of web jockeys used to extremely fast release processes, loose compatibility requirements, and extreme reliance on reuse. This really explains why we see this with node packages more than Python or Rust: older and more conservative programmers just don't have as many rakes to step on.

simpaticoder• 2 months ago

The solution in the Linux world ... is that there is a second level of human beings...

AKA "unpaid labor". I don't think that's a good solution, either. Certainly it's only by pure luck that no malefactors have infiltrated the ad hoc, anonymous social proof communities that Linux depends on, and I don't think other systems should emulate it.

The real solution (for Linux too) is a paid package curation service. Or really, a small handful of them competing on price, speed, reliability.

AgentME3• 2 months ago

But there is a second level of people reviewing packages on npm. They're the ones that report issues like the github issue this HN thread is linked to, and they very frequently get malicious npm packages taken down within a day of publishing. The big issue is just that not everyone is using a cooldown to avoid packages less than a day old and so people who install new packages at unlucky times don't get the benefit of that layer of review.

TacticalCoder2• 2 months ago

Notably, neither would have prevented the xz-utils backdoor from reaching package distribution, which remains the gold standard for sophisticated upstream compromise.

Mandating that the final binary is compiled without having any access to any test file though would have prevented the xz-utils backdoor as it was conceived though.

A proper packaging setup would first verify that all the tests are passing and happen in an isolated environment. And that isolated environment either returns which tests failed or gives the green light.

When the greenlight is given (that all tests are passing), another environment should first delete all files related to tests and then build (in a bit-for-bit reproducible way btw and we're basically here already so that's good) the final binary / package.

If you prepare your final package in an environment that has access to test files, there are simply way too many ways obfuscated binary data can be hidden in test cases / test files.

I'm not saying the NSA (sorry, Jia Tan) wouldn't have tried something else but I think we should really move to build/packaging that discards non essential data/files before compiling.

P.S: note that as a side-effect of reproducible builds... If we have reproducible builds and if we add, later on, a builder/packager that discards tests files and ends up with a final package that's not bit-for-bit identical to the package created while having access to the test files during the build, we've just detected a backdoor hidden inside test files (like the XZ utils one). As a really mindboggling food for thoughts: if we were to recompile all the Debian binary packages that are already reproducible today (95% of them), but while discarding all tests files before the build, we may catch other backdoors.

randusername• 2 months ago

My read is that there's a crowd that is unimpressed with mechanistic changes when in their view there is a cultural issue.

From the outside looking in, web dev has this frantic wild west energy to it. Mutability, dynamic typing, standards changing constantly, frameworks changing constantly, continuous delivery, CDNs, live A/B campaigns, large numbers of dependencies, sensitive user data spread out across a lot of infrastructure.

I'm not saying that's an accurate view and I don't think "I told you so" is the right attitude, but I can understand the place it comes from.

michaelt• 2 months ago

- MFA for publishing: https://docs.npmjs.com/requiring-2fa-for-package-publishing-...
- trusted publishers, available for about a year: https://docs.npmjs.com/trusted-publishers

According to [1] "All affected packages were published via GitHub Actions OIDC from the RedHatInsights/javascript-clients repository, indicating the upstream CI/CD pipeline itself was compromised."

So the malicious package would have gotten the happy little green star, with users assured it was "Built and signed with provenance."

[1] https://lwn.net/Articles/1075742/

za3faran• 2 months ago

Why doesn't Java seem to have anything close to this issue? Isn't it a solved problem?

skydhash5• 2 months ago

No deep dependency graph, so easy to audit and inspect. Stable development process, so easier to manage the flow of updates. And I believe a more cautious ecosystem. Not everyone is rushing to create or adopt new libraries, especially for what could have been a single file. So most libraries are about solving a domain, not just one single algorithm.

Same thing with C, Perl, PHP,…

48terry• 2 months ago

In every of these threads there's a bunch of snarky comments, either acting like this class of attack is exclusive to npm, or that nothing has been done about it. I don't think that's fair.

I mean it keeps happening lmao. You can track npm attacks these on a calendar. Someone made a npm parody of the classic "no way to avoid this" The Onion article.

It's great there's work to stop it all but also... it keeps happening. I find it funny in a "here we go again" way.

no-name-here4• 2 months ago

> In every of these threads there's a bunch of snarky comments, either acting like this class of attack is exclusive to npm, or that nothing has been done about it. I don't think that's fair.
… the classic "no way to avoid this" The Onion article

But isn't point of The Onion article that A) the US has >50x as many incidents as the rest of the developed world combined [1], and yet B) acts like there is "no way to avoid this". Does NPM have >50x as many incidents as the rest of established languages combined? Is NPM claiming there is "no way to avoid this" or are they putting in place things like automatic install delays?

While all the major js package managers already support install delays, none of the big local C#/dotnet/nuget apps do (Visual Studio/Rider/nuget/dotnet/VS Code). https://github.com/NuGet/Home/issues/14657

[1] https://edition.cnn.com/2018/05/21/us/school-shooting-us-ver...

joshspankit2• 2 months ago

Devs and other people who have seen behind the scenes at large companies know that most security is at best shaky and mostly hand-waved

It’s not even really the fault of the people who pushed for these setups, it’s a seemingly simple business decision: build it in a way that looks secure, add some black-box process, and tell the overseers that the reason there are no attacks is because it’s bulletproof, and definitely not because no one has really tried

Then, when someone finally turns their attention to you and walks in: fire whoever needs to be fired, patch that specific hole, maybe spend a bunch of money on a different system, assure the overseers that it’s handled, and move on with business as usual

It’s cheaper in the long-run, it makes stockholders happy, it relieves the bosses and their bosses, and for the most part there are “no security holes”.

Until now, of course

joshspankit2• 2 months ago

Downvoters: I’m curious why

numron-dev• 2 months ago

As a dev mostly working on Node. Those are scary title to read

ef2k• 2 months ago

perfect time to drop a reminder: vendoring might seem annoying, but it's way safer than trusting a remote registry.

quantumsicarius• 2 months ago

I also commented on the other issue flagged: https://github.com/RedHatInsights/platform-frontend-ai-toolk...

Also detonated the payload: https://leitwacht.eu/blog/valid-provenance-malicious-package

ffemac3• 2 months ago

It will only get much worse because popular AI coding harness (OpenCode/KiloCode) will just download random npm packages in the background without you knowing. And the devs don't care.

Setting min age is useless if everyone is doing it. The whole point of setting min age is make someone else take the bait before you.

beart2• 2 months ago

It isn't useless. Security researchers are the ones catching a lot of these and they will certainly not wait 3 days to inspect a package.

obsidianbases1• 2 months ago

Cool down sounds good until everyone does it and the issue isn't caught until afterwards

greatgib• 2 months ago

In addition with my usual rant about the current situation with most devs that now want to use dependencies in prod almost the day that they are released, there is something else that I just realized. A big part of the problem might be attributed to Github and the modern CI/CD frameworks.

Before, the source code was located somewhere, and the CI was usually located somewhere else, and slightly unrelated. At first, the CI job was to build ("privately") the artefacts, and they were manually released and deployed by maintainers and owner of software projects.

Then, it became the norm to have the CI located within the VCS file and the VCS located source code controlling the CI. For example having "script"/"description" of the CI actions located within the VCS itself.

Then, Github killed the CI/CD software market by offering "actions" almost for free and totally integrated within Github that was already widely used.

But still, for a long time people were wary to put tokens and security keys in Github and "public" CI/CD jobs and services.

And then, a few years ago, it became the gold standard also... You would look ridiculous to manually sign and deploy new releases with well guarded keys. What is expected from you is to have all your aws, github, ... secret api keys loaded in Github, and have your deployment and infrastructure provisioning automated with ("public") github accounts. All to be deployable in a second of a change being pushed.

So, obviously, the moment a hacker get control of a Github account or Github API keys, it is game over for the entire infrastructure.

Here I only referenced Github, but the bad things that it taught us became the norm and now these patterns are replicated everywhere (Gitlab, ...)

bodash• 2 months ago

GitHub repo (800+ stars) on a list of tips for protecting against npm supply chain attacks: https://github.com/bodadotsh/npm-security-best-practices

TZubiri3• 2 months ago

There's no magical solution, you just have to use (WAY) less dependencies

photon-torpedo• 2 months ago

Red Hat security bulletin on this incident: https://access.redhat.com/security/vulnerabilities/RHSB-2026...

Quote:

The affected packages are frontend libraries that are compiled and bundled into some container images during the Red Hat product build process.

So you might be affected if you deployed any Red Hat container images built after the compromise, I guess. (I doubt anyone besides Red Hat is using the affected packages directly.)

cyphar• 2 months ago

I'll be honest, I have always found the "trusted" publishing concept quite suspect -- the boiled-down argument is that developers are too incompetent to manage their own keys. Yes, there are obviously problems with storing plaintext publishing keys and tokens in $HOME, but that isn't the only solution. You can use hardware keys for signing (I would wager a lot of open source maintainers have at least one Yubikey or NitroKey kicking around -- they used to give them away at conferences) and for upload tokens you can even just password-protect them or require them to log in each time.

The alternative proposal with "trusted" publishing is to grant all of GitHub's automated infrastructure the ultimate authority to publish things on your behalf automatically in a way that it is shown as being more "trusted" than the developer uploading it themselves! I'm honestly surprised that it took this long for supply-chain attackers to start targeting this obvious security hole -- I doubt any user of GitHub would argue that GitHub Actions are a parogon of security, and yet the strongly recommended deployment workflow for several language communities involved bolting it into the core of your release mechanisms! I think the OCID stuff is interesting and provides some nice properties but it doesn't prove that the GHA workload is actually trusted in the sense that "trusted publishing" means.

The arguments for adding speed bumps to auto-applying updates apply just as much to adding speed bumps to doing releases (arguably even more so -- even rapidly evolving libraries don't have hundreds of releases a week). Automating everything to the nth degree just expands the blast radius when stuff like this happens.

I do all runc releases manually and sign them with a PGP key stored in a hardware token that requires a pin for every signature attempt. Yes, it's a bit more cumbersome but it certainly would be harder to supply-chain hundreds of projects of every maintainer would need to manually publish them. For some other projects that need to publish to crates.io or PyPI I was honestly a little dismayed at how prominently they push for "trusted" publishing and how little support there is for making the "dumb" publishing flow more secure (they even visually display such releases as less trusted, and the only way to get more green checkmarks is to let GitHub take the wheel).

Malicious npm packages detected across Red Hat Cloud Services

Related Stories