Two malicious Python libraries caught stealing SSH and GPG keys

https://www.zdnet.com/article/two-malicious-python-libraries-removed-from-pypi/

by choult • 7 years ago

624 317 7 years ago

zdnet.com

Havoc2• 7 years ago

Alarming given that pypi is treated as trusted by many I think (even if it shouldn't)

blub• 7 years ago

What is trusted then, the Anaconda base repo?

Havoc2• 7 years ago

I don't know the answer frankly.

The serious devs in actual dev shops I've asked answered with something like:

We do code reviews on the libraries we use. Basically if someone wants a library they're responsible for checking it

Me: Isn't that a shtload of work with new versions etc

Yeah so we tend to lag behind official versions quite a bit

It sounded like they host local mirrors of some sort with just the vetted code. Though I think vetted here is a quick glance over for shady sht rather than true security vetting

tandav• 7 years ago

bruh moment:

    $ python -m pip list | grep date
    python-dateutil               2.8.0

Yeah this is another library but anyway that was creepy.

mikorym2• 7 years ago

The first is "python3-dateutil," which imitated the popular "dateutil" library. The second is "jeIlyfish" (the first L is an I), which mimicked the "jellyfish" library.

F147H34D• 7 years ago

Wish they listed out how many installs.

medecau2• 7 years ago

No info on installs but some more details on each:

https://libraries.io/pypi/jeIlyfish

https://libraries.io/pypi/python3-dateutil

xtreak29• 7 years ago

At pypistats.org download numbers of the last half year can be found. * python3-dateutil has 271 downloads from non-mirrors in last month[1]

* jeilifish has only 106 downloads from non-mirrors in last month[2]

[1] https://pypistats.org/packages/python3-dateutil

[2] https://pypistats.org/packages/jeilyfish

https://news.ycombinator.com/item?id=21702973

qxnqd• 7 years ago

I don't get it. Who would type "pip install jeilyfish" by mistake?

jagged-chisel• 7 years ago

I can only see this working where someone would copy and paste the package name.

EDIT: another vector I saw mentioned in another comment: you pull in what appears to be a 'valid' dependency, and jeIlyfish is listed as a dependency of that package; looks legit so you proceed.

zbentley3• 7 years ago

Regrettably some people use fonts in their terminals and IDEs which do not make the difference between uppercase I and lowercase l obvious.

dahart2• 7 years ago

You don’t have to type the name of a package to install it. There are GUI package managers that use click to install. See Anaconda Navigator, for example.

steventhedev• 7 years ago

Put yourself in the attackers shoes. Your goal is to spread this to as many machines as possible. The best and easiest way to do that is to add your library as a transitive dependency. What better way to infect people than to get everyone who ran `pip install numpy`? As for getting it in, I'd push it to older projects as part of a "styling cleanup" PR because there's so much noise in the diff anyways. Imagine a PR to a project adding a transitive dependency for python3-dateutil. Most people would merge without looking twice, especially if you add some scary "this deprecates py2 support" to the PR.

abhiTronix• 7 years ago

These people did:

Downloads last day: 13 Downloads last week: 103 Downloads last month: 119

Check https://pypistats.org/packages/jeilyfish you won't believe your own eyes.

maweki• 7 years ago

It isn't helping that the dateutil-library actually is "python-dateutil". This confused me this weekend as I wanted to pip install dateutil which did not work.

campuscodi• 7 years ago

To search your own projects for the malicious libraries:

pip3 freeze | grep -i jeIlyfish

pip3 freeze | grep -i python3-dateutil

vortico• 7 years ago

There are probably several open-source projects that you can download today that include unknown malicious code. But they will be discovered eventually. Proprietary software, on the other hand, can keep malicious code during their entire life of relevancy. And in fact, it's rare for proprietary software to not have malicious code these days, with personal data being sent to servers, ads being delivered, and installers bundling third party products and plugins.

iudqnolq3• 7 years ago

Complete false equivalency. Exploiting user data deliberately, when you technically mention it in a thousand page legal document, is completely different from distributing malware because your dev machines got infected.

Jonnax• 7 years ago

I get the python3-dateutil because you might think it's an updated version of the standard library.

But how does jellyfish with a different char for L work? Someone would need to copy and paste it. But if they go to pypi, it won't have many installs.

Unless they started writing tutorials with:

"okay now just pip install X"

daolf• 7 years ago

If I had to guess, I'd say that the JeIlyfish package is just here to wrap the malicious code since JeIlyfish is a dependancy of the compromised python3-dateutil package.

deaps• 7 years ago

The python3-dateutil listed jeilyfish as a dependency - so I'm guessing that upon installing the fake dateutil, the user would see something like "installing dependency - jeIlyfish" - which, depending on your font, may, at the very least, cause a person to google "python3 what is jellyfish" arriving at a concise description of what jellyfish is from pypi.

Then dateutil calls the code in jeilyfish.

noisem4ker• 7 years ago

Just an idea: in autocomplete scenarios, jeIlyfish comes alphabetically before jellyfish, while looking like it. That could hijack an install. I don't know if and where autocomplete can be used with PIP, though.

mixedmath• 7 years ago

I think the key here is that jeIlyfish had malicious code in it, and the fake python3-dateutil imported jeIlyfish. Even if you were examining the source for malicious code, you might not notice the difference between jellyfish and jeIlyfish, I guess.

xamuel• 7 years ago

But if they go to pypi, it won't have many installs

If people are using number of installs as a safety metric, can't blackhats game that by having a bot install the package many times?

zbentley3• 7 years ago

Copy and pasting, even via word of mouth (rather than, say, a blog post or highly-google-ranked tutorial) is a surprisingly viral propagation vector.

And god help us if a malicious install command was posted, even if only for a few minutes before being edited, to a help or forum site like StackOverflow or Reddit.

dahart2• 7 years ago

PyCharm, Stallion, Anaconda Navigator, etc...

You don’t need to copy and paste it, or go to pypi, since there are commonly used ways to install packages by clicking on the name.

rmbryan• 7 years ago

What's the best information source for me to follow to keep up to date on these kinds of library vulnerabilities? I would make a feed of the homepages for all the libraries I know I use, but that won't help me with the libraries I use without knowing.

_rkz1• 7 years ago

I saw a recorded conference talk where they mentioned Snyk as a way to keep your eye on package vulns, but have never used it myself

medecau2• 7 years ago

"bandit" (available in pypi) is a nice static analysis tool - I don't remember if it is able to recurse into dependencies though

[safety](https://pyup.io/safety/) is a commercial product that monitors your dependencies for this kind of shenanigans

LGTM.com seemed to be working in this area - Semmle was acquired by github/microsoft

JakeMimoni3• 7 years ago

we made a tool that automates that process - https://trustd.dev

it will analyse open-source packages as you install them and tell you of any vulnerabilities before they are even on your system...

meaning it will detect problems in the libraries you aren't thinking about.

frou_dh2• 7 years ago

These two have been caught. How many haven't yet been caught?

Traditional Unix file permissions are pretty much a joke for the way developer computers get used (one user - does everything). Real process sandboxing is needed.

pornel• 7 years ago

And it's not just the developer machine that's at risk. Even if you protect your own system, you will still be shipping malware to your users, who may be vulnerable. And you'll be lending your credibility to the malicious libraries you distribute.

pnutjam2• 7 years ago

Not everything needs to run as root.

frou_dh2• 7 years ago

It's all running as john

OJFord• 7 years ago

    $ pkgman install hip2019pretty-ls
    $ su sandboxeduser prettyls

... nobody does that. Sure daemons might run as different user(s), but on ~100% of developer machines, the logged in user launches a shell as themself, and runs programs in that shell as themself.

swebs2• 7 years ago

Ok, but your private keys are stored in your home directory. No root access required.

megous2• 7 years ago

Anyone here not encrypting their private keys?

Also known_hosts file is a double edged sword. It's pretty sensitive in combination with a private key.

microtonal3• 7 years ago

You shouldn't just encrypt the keys. If malicious code is running on a machine, they could as well eavesdrop on keystrokes. This is even made simpler by the fact that a large part of the demography uses X11, which is inherently insecure since all applications can read key strokes, mouse events, and do screen grabs.

Get a hardware token. If you have a hardware token with a pin, they could extract your PIN by reading keystrokes and use the token on your machine. But once you yank the token out of the USB port, that's the end of it. Even better is a hardware token that requires physical confirmation for operations.

avian• 7 years ago

Modern SSH versions only store hashes of domain names in the known_hosts file exactly for this reason.

megous2• 7 years ago

Hmm, you're right, but HashKnownHosts in openssh defaults to no, still.

ohyeshedid• 7 years ago

I haven't personally seen that enabled by default.

greggman26• 7 years ago

I don't know what the solution is but it feels like this is a much bigger issue and we need some rethinking of how OSes work by default. Apple has taken some steps it seems the last 2 MacOS updates where they block access to certain folders for lots of executables until the user specifically gives that permission. Unfortunately for things like python the permission is granted to the Terminal app so once given, all programs running under the terminal inherit the permissions.

Microsoft has started adding short life VMs. No idea if that's good. Both MS and Apple offer their App stores with more locked down experiences though I'm sad they conflate app security and app markets.

Basically anytime I run any software, everytime I run "make" or "npm install" or "pip install" or download a game on Steam etc I'm having to trust 1000s of strangers they aren't downloading my keys, my photos, my docs, etc...

I think you should be in control of your machine but IMO it's time to default to locked down instead of defaulting to open.

microtonal3• 7 years ago

For this particular case: if you have GnuPG or SSH private keys, do not store them on-disk. Use a hardware token, such as a gnuk token, or a token with a secure element if you also want resistance against physical key exfiltration. A gnuk-based hardware token can be had in a nice format for under 30 Euro [1] or you could buy a STM32F103-based microcontroller for a few bucks and flash gnuk [2] if you like DIY. If you are a company, invest the 50 Euro in hardware tokens for your employees, one compromised SSH private key means that all the machines that use the public key are suspect.

I fully agree. It seems that the UNIX model is not very compatible with the macOS permission system, but one could imagine defining multiple types of shell sessions, each with its own set of permissions.

[1] https://www.nitrokey.com/ [2] https://salsa.debian.org/gnuk-team/gnuk/gnuk

Xylakant5• 7 years ago

yubikeys can store GPG keys and emulate SSH keys via gpg-agent. More exciting: OpenSSL recently added full U2F/Fido support, but it might take a bit until that lands in all distros https://www.undeadly.org/cgi?action=article;sid=201911150648...

gruez4• 7 years ago

I never understood the point of hardware tokens. If the machine it's plugged into it's compromised, the attacker can still use it to sign whatever they want. If the attacker has persistence on your machine, it's as good as stealing the key file. You're basically counting that you can discover the infection before the attacker can use it, or the attacker is unsophisticated so all they do is copy your .asc files (although you could also protect against this by putting your key files in a weird location). Depending on how proficient the attacker is and how vigilant you are, it could be a while before it's discovered. When you do discover it, you still have to revoke/rotate your keys since you have no idea what it could have been used for. So you end up not saving any effort at all.

noodlesUK• 7 years ago

As much as I would like very much to have an open source hardware token, there’s a major downside to the nitrokey start and other STM32F103 gnuk devices. The microcontroller itself is not hardened at all in hardware. It would not be remotely infeasible to extract secrets from its hardware using relatively available tools. This is in contrast to a true smart card or a yubikey, which have tamper evidence.

matheusmoreira• 7 years ago

Hardware tokens usually have three slots for keys. That's enough for a full set of signing, encryption and authentication keys. It is not wise to keep the primary certification key there, though. Hardware tokens can fail or be lost or stolen. Subkeys are expendable but the primary key is important enough for special treatment.

The best way to keep it offline is to make a paper backup with the paperkey tool and store it in a safe:

http://www.jabberwocky.com/software/paperkey/

In addition to this, it's a good idea to QR encode the key. QR codes are quickly and easily restored with a laptop camera and they support even 4096 bit RSA keys.

The zbarcam program can be used for this purpose but current versions have binary data decoding problems and aren't easy to interface with. I sent some patches to that fix these problems but they haven't been reviewed yet:

https://github.com/mchehab/zbar/pull/64

https://github.com/mchehab/zbar/pull/60

pulse7• 7 years ago

Maybe the future is in fully sandboxed/containerized apps, where each action outside of sandbox/container is confirmed by the user.

y04nn• 7 years ago

On freeBSD there is jails since year 2000.

UI_at_80x24• 7 years ago

When Docker was first introduced this is exactly the niche I thought it was going to fill. After I elbows deep into it I saw that wasn't the case.

I still think something like it could evolve to fill that role.

vsareto2• 7 years ago

Alert fatigue -> user accepts everything

_jal2• 7 years ago

That is a super-broken future. I mean, that's working... sort of OK for cell-phone-style use, where the vast amount of use is single-service, casual human-computer interaction.

It is horrible for any reasonable professional workflow. Even if you get permissions lined up correctly once, when you change your workflow, shit breaks. and more subtly, the way permissions accrete through reactive interactive dialogs, folks become less secure over time - past permission "paths" enable capabilities that the user no longer even remembers making.

More formal ways of defining security policy of course work better (I deal with that sort of thing for a living), but expecting normal people to be able to understand how to, e.g., use `restorecon` or the moral equivalent is a nonstarter.

Personally, I'm headed back to Linux when I need to replace the Macbook I'm typing on. I'm no longer the target MacOS user, and I'm not going to rely on a machine I don't control.

XorNot• 7 years ago

This is literally what SELinux does, and has been able to do for years. We don't need "default block lists" - we need solid SELinux policy in all distros.

ownagefool3• 7 years ago

There's a fair amount of other solutions to this too.

Things today:

- Protective monitoring. If it's stealing your key, it's presumably sending it somewhere, so somebody alerting should be going off asking clients are calling something external.

- Peer review. Maybe its about time we question whether we should actually be running so many random modules without actually vendoring and reading the code.

- Subscribing to security lists, so the people who actually do the above stop you from using dirty modules.

- Yubikeys, etc.

Things in the future:

- New home directory model. I believe systemd crew are looking into this, so I imagine it'll be hated, even if it does help solve similar issues.

- Newer distro models such as fedora silverblue where every process is isolated. I believe this currently mounts your home directory though, so maybe wouldn't work.

- New FS layout. Putting .ssh keys in something like .secret/ssh would likely make it easier to not mount secrets into isolated processes and help MACs too.

aledthemathguy• 7 years ago

This is currently only implemented in FreeBSD? No other distro uses this?

geofft4• 7 years ago

"Solid SELinux policy" is really the hard part there.

When I pip install paramiko, I do, in fact, want it to have access to my SSH keys. When I pip install ansible, I want ansible to be able to shell out to OpenSSH to use my keys. If I write custom Python code that calls gpg, I want that custom code to be able to load libraries that I've pip installed without the gpg subprocess being blocked from loading my keys. If I have a backup client in Python, and I tell it to back up my entire home directory, I want it backing up my entire home directory including private keys.

If I wanted an OS where I couldn't install arbitrary code and have it get to all my files for my own good, I'd use iOS. (I do, in fact, use iOS on my phone because I sometimes want this. But when I'm writing Python code, I don't.)

SELinux has been able to solve the problem of "if a policy says X can't get to Y, prevent X from getting to Y" for years. Regular UNIX permissions have been doing the same for decades. (Yes, SELinux and regular UNIX permissions take a different approach / let you write the policy differently, but that's the problem they're fundamentally solving; given a clear description of who to deny access to, deny this access.) Neither SELinux nor UNIX permissions nor anything else has solved the problem of "Actually, in this circumstance I mean for X to get to Y, but in that circumstance I don't, and this is obvious to a human but there's no clear programmable distinction between the cases."

To be clear - I think there is potentially something of a hybrid approach between the status quo and what newer OSes do. For instance, imagine if each virtualenv were its own sandboxed environment (which could be "SELinux context" or could just be "UNIX user account") and so if you're writing code in one project, things you pip install have access to that code but not your whole user account. I'm just saying that SELinux hasn't magically solved this problem because all it provides is tools you could use to solve it, not a solution itself.

codedokode4• 7 years ago

Selinux cannot show a popup asking whether you allow program X to do Y or whether you want to deceive it and provide fake data. So it is completely useless by itself.

LinuxBender• 7 years ago

I would add, SELinux policies distributed with each app, and iptables outbound rules using the owner module, as a partial mitigating control. One shortcoming of iptables / ipsets is the lack of DNS lookups that obey TTL, but that can be solved with some helper code. There may even be a xt module by now that does this, I have not checked.

hordeallergy• 7 years ago

If I were installing a custom kernel would adding selinux limit risk, or would a compromised kernel simply laugh at it?

AdmiralAsshat• 7 years ago

One of the reasons I've stuck with Fedora is the fact that it does have SELinux turned on by default, and takes active steps to make sure that the policies are up to date. It can occasionally suck when something gets borked and one of my programs starts throwing SELinux errors after an update, but I think the security trade-off is worth it.

It becomes much more manageable when you go all-in on SELinux from the outset, rather than trying to bolt it on after-the-fact.

e2le• 7 years ago

I suspect the solution is a combination of selinux (apparmor?) and curated code repositories. I don't think sandboxing alone is a solution as it doesn't alleviate concerns with zero days.

riyakhanna19832• 7 years ago

SandFS could be used for lightweight custom sandboxing $HOME dir. It uses eBPF to let you insert custom checks. Check https://lwn.net/Articles/803890/

whack• 7 years ago

When I first realized that any and all code that I execute, has read/write permissions to most of my filesystem, it blew my mind. The OS grants every process its own unique virtual-memory-space, specifically to prevent malicious/accidental interference with other processes. It seems like the file-system really should operate on a similar principle as well. Every application should run in a sandboxed environment by default, with exceptions being granted by the user for specific applications that actually do need access to the entire file system.

umanwizard• 7 years ago

This is already possible in Linux with mount namespaces, and used by (for example) systemd to block access to /home by services if so configured by the user.

de_watcher• 7 years ago

Maintainers Matter: http://kmkeen.com/maintainers-matter/

asdkhadsj2• 7 years ago

Unfortunately for things like python the permission is granted to the Terminal app so once given, all programs running under the terminal inherit the permissions.

Would Python permissions even be enough though? All it takes is one legitimate Python application wanting your Photos (lets say some Python photo manager app you wrote) and now all Python libraries get access.

Unfortunately, I think Apple's direction may be correct. It feels hugely inconvenient, but an end goal of all processes being signed and explicitly allowed certain things seems useful.

At the very least, Python should perhaps never get access to anything beyond the devving folder. Then any real use of Python applications would have to be properly baked out into processes where the OS can manage permissions for. This goes for all languages, Python was just an example here.

This is all off the top of my head, so I could be way off base. But it seems logical to me in the moment.

narrator• 7 years ago

The no setuid shell scripts rule on Linux was an early attempt to deal with this.

https://unix.stackexchange.com/questions/364/allow-setuid-on...

tomp• 7 years ago

Unfortunately, I think Apple's direction may be correct. It feels hugely inconvenient, but an end goal of all processes being signed and explicitly allowed certain things seems useful.

Apple is operating a racket. There's e.g. no need for signing (and developers buying expensive certificates), you could also have a dedicated "permissions agent" checking executable hashes against an online service, and giving them as many permissions as desired... So users could (1) pick and choose their "permissions providers" (not necessarily Apple, could also be e.g. GNUpermissions or WikiPermissions or whatever), (2) users could modify permissions (e.g. provider defaults to Web Access but I want to deny it for this specific executable), and (3) all (even unsigned) programs could run, but with minimal permissions by default (i.e. sandboxed).

greggman26• 7 years ago

Well it does get worse. Lots of apps/programs run in python. I want `python goodprogram.py` to have access to certain things but not `python badprogram.py`

notatoad• 7 years ago

Wasn't this the promise of containers? "python" should have access to the whole system that it is run on, but instead of running random python scripts you download from the internet on your base system that has all your personal data in it, you run it in a container that only has the specific files that the script needs access to.

equalunique• 7 years ago

Use SELinux

nnx• 7 years ago

https://hacks.mozilla.org/2019/11/announcing-the-bytecode-al... is a step in the right direction.

htns2• 7 years ago

Qubes OS already exists! It's been perfectly usable for more than a decade now. It can't be recommended enough: https://www.qubes-os.org/

Sadly "modern security awareness" doesn't seem to really amount to anything, and existing solutions go unused. %90 of this "malware library" problem too would have been avoided if package repositories just required all packages to be signed with keys on hardware dongles. Ruby and python and some others at least have the excuse of inertia, but it's pretty cynical of everything else to not require signing already, when the added friction is nothing relative to the effort of writing software worth publishing.

fwip• 7 years ago

I don't think that's a good solution.

A) it adds cost to what was previously free for many people

B) it does nothing to deter bad actors

C) many software packages would be abandoned because their dongle was lost or destroyed

tkfu• 7 years ago

%90 of this "malware library" problem too would have been avoided if package repositories just required all packages to be signed with keys on hardware dongles.

I'm with you about requiring signatures, and you can get around the FUD about packages getting abandoned because of developers losing keys by implementing something like TUF[1] (because of delegations in the targets role), but I don't really see how you can enforce dongle usage. That is, how can the repository administrators tell the difference between a signature from a key on a hardware dongle and a signature from a key on somebody's windows laptop? You'd need an IRL auditing process, which just isn't feasible for most open source packages.

[1] https://theupdateframework.github.io/

slavik81• 7 years ago

Why? If I sign a package, that's proof I endorse the contents... but how do you know I'm not a malware author?

omarhaneef• 7 years ago

Some "easy" tools:

1. Ban libraries that have names that might confused with others (so no I where L is etc).

2. Only use libraries that have been around for over 1 year (you're relying on the community to debug them).

I think with these two alone you might get rid of most issues. The problem is that someone might buy an old, trusted name and then inject malicious code. I don't know of a technical way to enforce change of ownership that might be useful and not too much work. Alternatively a nefarious player might release useful code, bide their time to get a reputation, and then inject malicious code.

Short Life VMs are a good idea but sometimes you want them all to talk to each other.

jonathanstrange• 7 years ago

The problems are forks, though. Anybody can create a copy and the problem is that e.g. on Github there are tons of library clones that aren't even marked as fork, because they have been cloned and then imported as a new library for whatever reason by the new maintainer. I've often had substantial troubles finding the original library or the most well-maintained fork.

Large repos like Github should do an automated similarity search and prominently display potential older (as in creation date) versions of the same library even if they are not forked from the original repo directly.

ezzzzz• 7 years ago

Ban libraries that have names that might confused with others (so no I where L is etc).

I agree that this is some obvious low-lying fruit here. Given that the onus is currently on devs here, an actionable solution today is to use a font which makes the differences between i, I, L, 1 easily distinguishable (obviously not fool-proof), or bake a check for malevolently named dependencies into your linter or plugin.

neves• 7 years ago

When I see any software updating its dependencies or an app store I think the security battle is lost. It's very easy for any gov intelligence agency to seize an abandoned software and start to use it to distribute spyware.

dbrgn• 7 years ago

Flatpak offers sandboxing, but I think it's mostly meant for applications, not for development tools (where you often need to access your keys anyways).

simias4• 7 years ago

The problem is convenience vs. security, and we all know that even knowledgeable users will often sacrifice the latter for the former.

Technically on many OSs you already have dozens of ways to achieve what you're saying. You could spawn a VM, use a different user, use some container framework, use SELinux, etc... The problem is that usability is generally terrible. Or maybe not terrible but bad enough that many people will start mashing the "allow" button indiscriminately when they're in a hurry and want the thing to Just Work. There's only one thing I know for sure how to do with SELinux without having to lookup a guide or manual, it's how to disable it.

Look at the modern Linux desktop for instance, you need admin privileges to configure a printer but your sensitive private files are just lying on your home directory with user privileges. That makes no sense.

This is not really a technical problem, that was solved when we invented the MMU, it's an UI problem.

eythian• 7 years ago

Look at the modern Linux desktop for instance, you need admin privileges to configure a printer

Not for an amount of time that is firmly outside "modern."

fauigerzigerk• 7 years ago

I sort of agree, but I think that user interfaces need an underlying conceptual model that is familiar to users separately from the actual UI. Otherwise any purely presentational change implies arbitrary new rules that no one can remember.

One conceptual model that has worked to some degree on mobile platforms is the idea that programs not users have permissions.

But mobile platforms are overdoing it in a way that makes data centric work impractical.

I'm not quite sure how to overcome these issues, but I feel that making the distinction between permissions assigned to users vs permissions assigned to programs more explicit and more visible could take us a step further.

justincredible• 7 years ago

Secure but inconvenient has an immediate and visible downside, whereas insecure convenience has delayed and nebulous downsides. Convenience will always win when appealing to the lowest common denominator, but maybe we can have both.

greggman26• 7 years ago

I agree it can be inconvenient but don't iOS and Android kind of show that for most users the experience can be just fine? For us devs we can still either open our systems or find better ways but I'd prefer the default to be more closed.

I'd also prefer the default to be VM-ish. Right now I have 275 projects in my /home/me/src folder. Every one of them has at least a build process that executes code I have mostly not reviewed. Many of them execute more code after being built. Some of them execute code when they were installed for dependencies. Maybe I'm just not used to it but I'd really like if the process for isolating all 275 of those projects was somehow easier. Or maybe I'd prefer they were all isolated by default and that we could work on ways of making them easy to do the things they really need to do and hard/discouraged to do anything that would also be a security issue.

Maybe people have suggestions. I don't want an actual VM if it means I have to install 275 copies of OSes and 275 sets of semi global dependencies (like I don't want to have to install python 3.8 200 times) but I do want all 275 of those projects not to be able to read my private keys or my photos etc or write outside designated areas

StreamBright2• 7 years ago

Not really. You can have security with any system where security was present in the design phase and not trying to retrofit it into the implementation without giving up much of UX. Non if these operating system we use today were designed with security in mind because at the era they were created there was no need for security, half of the problems we know today simple did not exist. SELinux is great but extremely hard to get it right.

ken• 7 years ago

I don't buy it.

I heard the same complaints when we went from Apple II to Macintosh, from 68K to PPC, from Finder to MultiFinder, from OS 9 to OS X, and so on. At each stage, the CPU and RAM became less of a free-for-all and more of a system where you were only allowed certain operations. (Yeah, protected memory is terrible for hacking! Sorry.)

And yet, somehow, we survived. Security got better, and usability (generally) got much better in other areas to compensate. Life is better all around when you can tell the computer accurately what you mean, instead of relying on your ability to jump across layers willy-nilly.

You're describing possible solutions using current technologies. None of those (except perhaps "users", in some form) are inherent to the design of a security model. We only have them because they were a convenient way to implement our current security model on top of the operating systems we've got now. At some point, we aren't going to be solve all our problems by adding more layers to a 1972 design. I can think of many security models which could offer better usability than SELinux or "mashing the 'allow' button".

Abishek_Muthian• 7 years ago

You can add android to the list, any app which is given storage access can access any of your files there. So, if we had downloaded sensitive.pdf from email and it lies in the downloads; theoretically any of the dozen apps which has storage access permission can siphon it off to their server.

Android Q has 'scoped storage access' to prevent this and individual folder access permission; but is optional and will be enforced only in the next major android version which by android update standards will take another 5-6 years to get widely adopted.

danShumway5• 7 years ago

For a long time, the security model of OSes has been, "only trusted code should run."

That model doesn't work. Code affects too much of our life and is used in too many scenarios for a binary "trusted" boolean to be feasible for most people.

Offices and houses have locks inside of them as well as outside. If I invite someone into my office, they don't immediately have the key to my server room. But (while I agree that software bloat is a problem) you'll still find plenty of people after these issues who argue that having too many packages is the real security issue, and really all this comes down to is vetting our repositories better, or forcing everything to be signed, or whatever.

In reality, the long-term solution is that we have to start taking native sandboxing seriously -- embracing efforts like Wayland, Flatpak, SE Linux, JS Realms, and turning on secure sandboxing systems by default. The problem isn't NPM or PyPi, it's Node/Python sandboxing. The problem isn't allowing arbitrary browser extensions, it's per-domain extension sandboxing. The problem isn't third-party scripts, it's browser fingerprinting.

It is impossible to scale a trustworthy repository to the size of NPM, or PyPi, or the Apple store, or AUR, or the Chrome web store. There is not, and is never going to be, a trustworthy repository of that scale.

In the meantime, because most platforms don't have serious sandboxing controls turned on by default, you just have to reduce your dependencies and install less software. But that band-aide fix will get less and less useful over time, because more and more of your life will depend on software from more and more diverse sources, and it will be impossible for you to vet everything. Being conservative about dependencies and code you run is an extremely temporary fix that is not going to work in the future. But people treat it like it's the obvious solution and that we don't have any need to address the fact that most consumer-grade OSes, platforms, and runtimes are simply crap at security.

tarsinge• 7 years ago

my keys, my photos, my docs

You listed it, the problem is personal data.

If an executable nuke my computer it’s a pain but if they don’t have access to my data it’s not that severe. For a long time we conflated the two, being hacked / getting a virus meant you got downtime, and you were screwed if you didn’t have backup, but it stopped here.

But really what changed in recent years in threat level is how much personal informations are in computers, and classic security models were not thought with that in mind. There is no « personal data » flag, just a permission it belongs to a user. It worked fine for executables and working documents, but personal data are a different thing.

continuational2• 7 years ago

Why is it that we accept that any piece of code can just randomly reach into the file system and connect to any server?

Why isn't the file system an object you pass around instead?

greggman26• 7 years ago

I agree with this as well. I feel like the OS should require apps to specify which domains they'll connect to and they should be allowed to only to connect to those domains. Yes you should be able to specify all domains like if you're making Firefox but otherwise it should be looked on with huge suspicion to list more than a couple. As it is any app/install script/build and scan my local network, find out my SSIDs, tell what devices are connected to my lan, which ports are open, what software is on some of those ports, and of course exploit any known vulnerabilities.

I have a guest WiFi partly for that reason so when guest come over their apps hopefully can't hack stuff on my non-guest network. Unfortunately every new game/app/dep I install on my phone/pc/mac/tv/appletv/ps4/switch can still do that and while I might trust the app devs I don't trust the library devs, especially the analytics libs compiled into every game.

jerf2• 7 years ago

The phrase you're looking for to poke into a search engine is "capability-based security": https://en.wikipedia.org/wiki/Capability-based_security

It's a long, kinda story, which I'm not intimately familiar with, but seems to boil down to, it's more effort than we're willing to spend on rebooting our entire computing infrastructure. (Lots of things that could improve computing have that problem. "Rebooting our entire computing infrastructure" is a boil-the-ocean problem now.)

staticassertion7• 7 years ago

Just have package managers run as a separate user in a separate namespace? I don't get why we let packages do crazy shit on our system as a fully privileged user.

Look to browsers, imo. 'Extensions', which are very analogous to packages, have to package a manifest, users must explicitly ack the permissions, and they have restricted access to the system.

Kalium3• 7 years ago

Most packaging systems are designed with the expectation that they will be installing arbitrary things that can interact with one another in arbitrary ways that should not have to be defined in advance for a given package.

The result has been a very hands-off approach. Package management systems for user-land libraries, like pip and so on, are extra guilty of being hands-off. And who wants their dev tools telling them what they can and can't do?

hatchnyc• 7 years ago

Yeah but not just package manager. As a developer or build machine you are just the first potential target and potentially not even the most attractive one. If you've got a dependency on malicious package that is getting embedded into your code there's near limitless potential downstream harms waiting after the code is deployed.

ozim2• 7 years ago

You know you should not run as root and you should not install all kind of crap on your computer.

Every time you walk on the street you have to trust 1000s of strangers they are not going to murder you or rob you. You also know you should not go into dark alleys alone at night.

This is just how life works.

WorldMaker• 7 years ago

Microsoft has started adding short life VMs. No idea if that's good.

The lightweight sandbox VMs seem to be pretty useful so far. It's still so new that I've not seen it used outside of blog posts yet, but that's the nature of cutting edge features only "just" released. (It was released with 1903 in May, which was delayed several months and a lot of businesses still are hesitant to install it.)

Windows 10 has had Controlled Folder Access (aka Ransomware Protection) available as an option for a while as well, which acts like Apple's system of forcing apps to need additional permissions to user directories. It's not quite as fine-grained as Apple's yet, and it is certainly not on by default.

(I turned it on for interesting paranoia reasons on my gaming desktop, and it's been fascinating to watch what gets blocked. Though my paranoia led me to adding Steam folders to the Controlled Folders list, which has especially made it a permissions whack-a-mole because Game Developers are children and games access random folders all the time, use five EXEs where one probably should have been enough, use random EXEs in TEMP folders to call EXEs in Steam folders, games not installed by Steam sometimes try to access Steam folders, NVidia wants to touch everything, etc.)

Both MS and Apple offer their App stores with more locked down experiences though I'm sad they conflate app security and app markets.

Windows 10 has had sideloading on by default for years now, and supports Win32 apps inside APPX/MSIX. Win32 apps by default aren't nearly as sandboxed sadly (because it is hard to guarantee Win32 apps work as expected when sandboxed), but some sandboxing is better than none. The biggest issue seems a lot less to do with "app market" (you don't have to publish to the Microsoft Store, and MSIX even supports auto-updating for sideloaded apps vaguely "ClickOnce-style") and a lot more to do with "please rebuild your installer to something more modern", which is a hurdle a lot of developers don't want to overcome. (Because installers are terrible and once one is built who wants to rebuild it. Because MSIX support for Windows 7 is still "Beta" with the April end-of-support for Windows 7 looming over everything, and some Enterprises clutching their Windows 7 purses as the new XP...) Who wants extra app security when it means a lot of development effort to repackage your app? (Even if it can be semi-automated from your existing installer.)

icefog3• 7 years ago

OpenBSD has a really novel concept of using unveil and pledge to contain processes into restricted-service operating modes. And being OpenBSD, these security features are actually turned on by default for many binaries. See discussion in https://news.ycombinator.com/item?id=17277067

saagarjha5• 7 years ago

macOS has this already.

gnl• 7 years ago

This is spot-on. In a perfect world every major OS would have proper, granular mandatory access control enabled by default and applications would come with a profile specifying precisely which resources they require – at least regarding the more critical stuff like keys and cookies – with attempts to access anything else triggering an optional notification. Hopefully macOS will become more granular that way and Apple will continue pushing and improving what they began with Catalina.

Meanwhile, in a less than perfect world there's XFENCE [0], previously known as LittleFlocker. It's basically LittleSnitch for files. It was originally developed by Jonathan Zdziarski and later sold to F-Secure.

The challenge is to set it up in such a way that the level of interaction is kept at a minimum while still providing some level of protection.

I might write a detailed blog post / howto about it, but meanwhile here's the TL;DR if someone wants to try this blacklist/greylist approach:

1. Set an 'Allow any app – rwc' rule for /Users to override the default 'Watch – rw' rule there, which would otherwise result in a ton of popups. This does not override the more specific watch rules for some critical resources like loginitems, etc.

2. Add watch rules for additional critical resources, like ~/.gnupg, ~/.ssh, ~/bin, possible password manager directories, Firefox/Chrome directories to prevent cookie extraction, etc.

3. Temporarily add a watch rwc rule for ~/, thus overriding the Allow rule for /Users.

4. Run any network connected software with a potentially large attack surface like browsers, torrent clients, vpn clients, etc. and give them the required permissions to your home directory using the popups. Make sure to put them through their paces in terms of file system access to cover all possible use cases.

5. When they are usable without any more popups, remove the temporary watch rule and add 'Deny rwc to /Users' rules for each one, thus overriding the general /Allow rule we created above. An application-specific watch rule would be nice here instead, but sadly that doesn't seem to be possible – watch rules apply to all applications.

Execute steps 3–5 for any other untrusted software you might want to install/run.

When combined with LittleSnitch to catch possible attempts at data extraction, this reduces the risk of rogue applications extracting/damaging critical data and limits the potential damage of possible RCE vulnerabilities in network connected software. And it does this with a minimum of interaction – after the initial setup phase.

I've been running LittleFlocker/XFENCE for a couple of years now and the setup described above for maybe a year and it works like a charm, currently on Mojave, previously High Sierra, all the way back to Capitan, if memory serves.

A whitelist approach would of course be more secure, but that's way too stressful and distracting for me.

[0] https://community.f-secure.com/t5/Home-Security/XFENCE-beta-...

ris• 7 years ago

The first step is to reject the fashionable "wild-west" style of package repositories and realize that maintained repositories came about for a reason. Then you can worry about rearchitecting your OS for sandboxes up the wazoo.

bdamm2• 7 years ago

Yep, and Apple, Google and Microsoft aren't helping. Particularly bad these days is the idea of granting apps permissions; for example, if I want to deploy a new Google Drive application for some whiz-bang thing, I almost certainly need to "Grant read and write for all documents." Well, I don't want to do that, so I don't use any Google Drive add-ons ever. Is there a way to restrict privileges by folder? Nope! Same is true for Microsoft Office 365. Apple is trying, but still a long way to go.

canada_dry• 7 years ago

Is there a way to restrict privileges by folder? Nope!

Playing devil's advocate... I'm sure most folks here appreciate that providing granular level of permissions/restrictions can often lead to a less secure environment!

Users (of pretty much every skill level) quickly become overwhelmed and give up.

I recall advocating that a large client group manage their own permissions in Sharepoint. It was a painful and futile exercise! Though really not much different than IT admins trying to managing security of a large organization using active directory.

tzs• 7 years ago

You could probably do something with access control lists and setuid/setgid. Take a set of programs that all should have the same set of accessible folders and the same set of inaccessible folders, and make them setuid to the same UID, and then add the appropriate entries for that UID to the access control lists for all those folders.

The usual interfaces for manipulating ACLs are not very friendly so you'd want to have some kind of tool to set this thing up. Managing the ACLs, and managing the program grouping UIDs, and making it so things don't get regularly broken by package managers that don't know about this system is probably going to be challenging.

What I'd like to see is someone make an access control system for Unix based off of the ideas from the old DEC PDP-10 operating system TOPS-10 file daemon system. TOPS-10 originally just had owner/group based access. Later, they added the file daemon access control system. The way that worked is simple:

1. If an access was allowed by the owner/group system, it went through normally.

2. If an access was NOT allowed by the owner/group system, and the new "use file daemon" flag had not been set by the caller, the accesses was denied.

3. If, on the other hand, the "use file daemon" flag was set, the OS sent a message to the file daemon describing the desired access, and asking the file daemon if it should be allowed or not. Whether the OS allowed the access or not was determined by the file daemon.

In what follows I'm probably getting some details wrong, but the general idea is right.

The way file daemon made its decision was by consulting a rules file containing access rules. The rules file was named ACCESS.USR. I don't remember if file daemon looked for that in the same directory as the file it was being asked about, in the home directory of the owner of the file, or what.

The key idea is that the access rules for a file were in ACCESS.USR, NOT in metadata of the file itself. ACCESS.USR allowed for wildcards, so one rule in there could specify the access rights for a whole class of files--including files that did not yet exist.

My recollection is that the rule matching could be based on the name of the file someone is trying to access (full or partial, with wildcards), the user and group that is trying for access (I believe wildcards were allowed here, too), and what program was being used.

So let's say you wrote a game, and it wanted to keep a high score file. You could make an entry in ACCESS.USR that said anyone was allowed write access to SCORES.TXT if they were running EMPIRE.EXE. (Picking EMPIRE for my example in the hopes of summoning Walter Bright, since he probably remembers more details than I do about file daemon). Then just make sure to set the permissions on SCORES.TXT so that access by anyone other than yourself will fail, and make sure that EMPIRE.EXE sets the "use file daemon" flag when it tries to write the score file.

This fits in well with the way users think about things. For example, let's say I've got public and private key files. My public key files have .PUB extensions and my private key files have .KEY extensions. It is easy to put a rule in ACCESS.USR that denies all others access to my .KEY files. If I do something that creates a new .KEY file, I don't have to make sure whatever tool created it set the ACLs...I just have to make sure I use a .KEY extension.

It's a lot easier to develop and maintain a naming convention that reflects my security requirements and stick to that than it is to manage per file ACLs that reflect my requirements.

ufmace• 7 years ago

This sounds kind of like what Microsoft did with UAC in Windows Vista. As I recall, it was a terrible experience and everybody hated it. Seems like the trouble is that the huge installed base of applications have no accommodations for anything like that, which makes it awful to try to impose stricter permission requirements now.

Even imagining how is kind of tough. How do you let some Python scripts, but not others, access certain directories, like the ones with your SSH keys? It would have to be built into Python, which may mean major changes to tons of packages. Ditto Ruby, NodeJS, Perl, PHP, and every other interpreted language out there. And how do you develop compiled applications in such an env? Suppose every new build would have to be signed with the expected permissions. But how would you let one internal package access a dir, but not some other one? More internal permission systems I guess?

IshKebab• 7 years ago

We need library level isolation, not just app level isolation (which desktop OSes don't even bother with).

Fortunately there's some amazing work on supporting library isolation in WASM. There was a really good blog post about it recently.

srcmap• 7 years ago

In the old day, I used ZoneAlarm to alert me / block any app that was trying connect to any networks by default.

Does anyone know similar solution for today's win10, Linux, Android environment?

Win10's firewall is ok for blocking when configure manually. But it lacks the "alert" functionalities on any new app/connections requests.

StreamBright2• 7 years ago

I absolutely agree with you. An OS should be many read only folders like /kernel /etc /bin that can be changed in an update process (that requires reboot) and folder where you give access to specific applications like /bin/ssh ro -> .ssh/id_rsa. Python process should not have read access to this folder by default at all. There are other solutions that you move all the credential access to a service that is accessed on a port by applications using tokens with different privileges. The current implementation of operating systems are pretty horrid, retrofit security all over. I think we are ready for a new OS era.

grumple• 7 years ago

Running everything in containers seems to make sense.

jbergens• 7 years ago

Windows have had locks for specific things for many years. They also created a new model with sandboxed runtimes för Windows Store Apps but those are not very popular.

Yarn, a ja package manager, tried to forbid any code execution during installs. That may be a good idea. At least show a warning or require an extra flag before executing anything.

roryrjb2• 7 years ago

I have been working on a Node.js library (very much work in progress and progressing slowly due to lack of time) that integrates libseccomp to be used programmatically inside Node.js whether at library or application level. For me at the moment it's kind of an experimental idea as it can be quite tricky to get everything right and in order without the kernel killing the process by mistake, but at least I think for some of the cases in the category that this issue is in, it will help mitigate it. I believe libseccomp already has official bindings for Python, as well as third party bindings for other languages, but this kind of work has been very successful in OpenBSD with pledge and I think it has been overlooked in dynamic programming languages and Linux in general.

roryrjb2• 7 years ago

It would be good to get some feedback on this instead of just anonymous downvotes.

I'm thinking the way it would work is that, at the application level, you'd open sockets and other file descriptors and then lock everything down, so if some malicious library tried to read a file or spawn a process the application would be either be killed or return an error, obviously depending on the application logic and whether or not it can handle the error. I would advocate this kind of approach as it doesn't need any external hardening, i.e. you get the benefits whether or not you're inside or outside of a container, whether or not the application is being run on a distro with SELinux or AppArmor, it's basically built in. I may be missing some big thing here but like I say it would be good to get some feedback here.

u801e• 7 years ago

Would sticking to the OS upstream package manager be a safer option compared to installing from pypi directly?

How often does something like this happen with packages in the CentOS, epel, or Debian repositories?

jeremyjh• 7 years ago

I think it would be quite a bit safer and you are already trusting your distribution but those libraries tend be very old/stale versions, and limited selections.

abstractbarista• 7 years ago

Terrifying but understandable. We need better ways to stop this but I'm not sure how...

In the meantime, store your private keys on a security device like a YubiKey. I use it simultaneously for all signing, encrypting, and authenticating (SSH as well as PAM to my workstations).

Be sure to set a strong PIN on the device. And have a backup!

jacquesm• 7 years ago

Package management and curation are the Achilles heel of open source. Abandoned packages, typo and letter substitutions, maliciously crafted pull requests and so on are all going to go up in frequency until the environment is hardened enough that the bulk of these attempts fail. That's a long way to go, and the number of capable maintainers and curators is relatively small.

Some environments (Python, Node) are more susceptible to this sort of trickery than others.

roland35• 7 years ago

From the article... The libraries were "jeIlyfish" (with an upper case I) and "python3-dateutil" (not "dateutil"). Both libraries were close spellings of the real libraries.

The lesson for developers is double check your imports! Spelling does count and there are lots of similarly named libraries (most are not malicious thankfully)

keyP5• 7 years ago

Agreed, although I think there's more than just a comparing spelling issue here. "dateutil" via pip is installed using "pip install python-dateutil". I can easily see someone thinking "python3-dateutil" is simply a Py3 compatible version of "python-dateutil". The "python3-dateutil" module imported "jeIlyfish" so my guess is that the creator banked more on people installing the fake dateutil library than directly downloading jeIlyfish.

bhouston2• 7 years ago

Maybe ash keys and got keys need to be protected from access by anything local unless given permission? Right now they are just files sitting there that can be read by any standard user process right?

saurik• 7 years ago

Ok macOS you can easily throw your SSH keys into the secure Keychain (not that I do this...).

bhouston2• 7 years ago

It should be the default that they are protected though on all os. We should not automatically trust locally installed apps anymore. They should be sandboxed by default like on Android and they should ask for permissions as they need them.

Windows and Linux need to get with the times.

My dev env should be sandboxed like everything else. Git can have ssh permissions but not every random tool from pip or the ceasepool that is npm.

lovelearning2• 7 years ago

"pip" has a usability problem. It should do a lot more at preventing this kind of thing. When using pip, it's not easy to tell information like the release date, how many versions have been released, and so on.

Since such info is available from PyPI API, I wrote my own "pypisearch" script to sort by latest release date and include number of releases to weed out packages that seem useful but are old or rarely released. I should probably integrate PGP signing info too into it.

iudqnolq3• 7 years ago

Is the code public? I'd love to use something like that.

lovelearning2• 7 years ago

It isn't. I'll make it public and announce it here by Friday.

ddevault2• 7 years ago

This is part of why I install my Python dependencies from downstream Linux distro repos. I never use virtualenv. If a distro is missing a package I need, it's a simple process to put it together, and the additional steps and checks built into the process stop close to 100% of these issues. Getting a human here also lets you do things like patch out telemetry or other anti-features.

Software repositories without a human review process are a bloody stupid idea.

cozzyd2• 7 years ago

Indeed. Plus this whole "install the whole python ecosystem for each thing you want to use" is insane.

gdevenyi• 7 years ago

In science this is pretty important so we can control versions and reproduce a result again as well.

pfranz• 7 years ago

I'm a bit torn and often err on the side of installing into --user or virtualenv. What often happened is I would toy around with a project, install a few libraries, then abandon it. Months/years later I would hit an incompatibility and I wouldn't know which packages were necessary for things I use everyday and which were because of that abandoned package. Similarly, when revisiting that abandoned project I wouldn't know what a good known set of libraries were. Installing stuff globally also makes it really difficult to know what dependencies were needed to run it on a different machine.

This wasn't just for Python libraries, but also with OS packages (especially non-sanctioned RPMs). I've been very selective about what I install globally, and usually first reach for a VM or other virtual environment.

One big counter to this is when I want to make it a simple commandline tool to run. I have to enter a Virtualenv just to run my fancier `ls` command?

Carpetsmoker• 7 years ago

Software repositories without a human review process are a bloody stupid idea.

What review process? Do people actually review/audit the code? Usually they don't. All it tells you that at least one person though it looked "okay enough" to package it, based on unclear criteria. It's most certainly not a "review process".

scbrg• 7 years ago

Which, honestly, would have been quite sufficient in this case.

I can't remember ever having seen anything like this in Debian, for instance.

arijun• 7 years ago

In the end I think the solution to these issues will be something like what's promised by the Bytecode Alliance[0]. The idea is you give each package its own WASM sandbox with granular control over its permissions.

That solution also has the benefit of allowing you to call a package from any language from your language of choice.

I highly recommnend reading their the article introducing the idea, its very convincing:

[0] https://bytecodealliance.org/articles/announcing-the-bytecod...

ryukafalz2• 7 years ago

I agree with this. They don’t mention this explicitly in the article, but it has a capability-based security model, which is something I think we desperately need in our OSes. (They do link to a paper about it that mentions this.)

There are a few other such systems that look interesting; Agoric is working on one for JavaScript, Google has a kernel patch set that adds capability support to Linux, and Christopher Lemmer Webber is working on a similar system on top of Racket called Spritely Goblins. I’m excited about all of them though, because it feels like this kind of security model is starting to gain public awareness!

kasperni• 7 years ago

There is also https://xtclang.blogspot.com/

Java, has started going down this route with the new module system and Lookup objects, however, this is mainly for restricted field/method/constructor access. I do hope we will see something similar for File and Network I/O (Random memory access is less of an issue in Java)

I do think we are going to see a lot more of this in the future.

gnufx2• 7 years ago

Hear, hear on capability systems, but they seem of limited use confined to specific language implementations, as opposed to the whole system. I wonder what's the Google kernel patch, and how it compares with Capsicum. It's rather tragic to gain public awareness so long after KeyKOS et al...

Nokinside• 7 years ago

"jeIlyfish" (with an upper case I) and "python3-dateutil" (not "dateutil").

Libraries should take lessons from writing safety critical code. If you identify libraries visually by name, the main problems are:

* easily misread characters like 1 (one) and l (lower case L), 0 and O, 2 and Z, 5 and S, or n and h.

* identifier names that differ only by on or few characters, especially if they are long.

It's possible to enforce a set rules that make identifier names are visually distinguishable and string distance measure to check all new libraries that are added against old names.

cstuder• 7 years ago

Ironically jellyfish is the one library which can help you with that task: doing approximate and phonetic matching of strings.

keyP5• 7 years ago

There's a bit more to it as "dateutil" is actually installed via "pip install python-dateutil", not simply "pip install dateutil". If someone was to see "python3-dateutil", there's every chance they think it's the same module but with Python3 compatibility.

JakeMimoni3• 7 years ago

We just recently launched a free tool that helps python developers prevent exactly these type of issues.

Feel free to check it out: https://trustd.dev

we work preventatively, so as you download packages, the tool will analyse it and tell you of any issues found.

We’re looking to collaborate with devs to work out what features should be next.

andybak• 7 years ago

First question is what the hell does this mean:

As you install open-source packages, trustd will scan them and provide you with instant feedback on any problems.

What kind of scanning? Algorithmic? Based on human review? If we're outsourcing trust to you, I'd want to know a lot more.

And "we use Slack instead of a dashboard" doesn't sound terribly appealing. I'd want a dashboard and a range of notification options (for me email > Slack. Others may differ)

JakeMimoni3• 7 years ago

might not have explained it the best I could have haha..

It means as you pull packages in from NPM et al, the analysis goes to work, telling you of any known vulnerabilities, or any license in-compliance.

With regards to Slack, we are hearing that a lot, it isn't the best mechanism for providing this feedback, and we are working on alternatives now, including email.

Happy to answer any more questions on here or reach out jake@418sec.com

swiley2• 7 years ago

Don’t install packages with large numbers of dependancies (for me, this is more than 2.)

Don’t install packages you haven’t at least been to the website for and preferably couldn’t build yourself.

Libraries and package managers help us work together, they’re not excuses for not thinking.

johnday• 7 years ago

Placing the burden of responsibility for security on the end user is not the way to go about this - at least, not if you want people to actually use your product / language.

Package managers do not only exist as a convenience. They should also provide guarantees about their packages, or at the very least some level of moderation.

swiley2• 7 years ago

You could make that argument about repositories not package managers (which are just software.)

Some do! (Main OS repos (not community ones) do.) But the reality is that this takes man hours, so repositories have been set up without those guarantees in the name of efficiency and expectation of responsibility.

I don’t think these are built because people “want users for their language” but because they were needed. This whole “you shouldn’t do that because it scares users away” thing tends to result in terrible software IMO.

objectified2• 7 years ago

This will keep happening, and not only will SSH And GPG keys be the target, but any interesting data will be stolen.

And the problem is much larger than these typosquatting attacks. Abandoned Github projects taken over my malicious users, rogue Maven/npm/PyPI/what have you repositories, hacked accounts on any website that is used for distributing programs, feature branches in open source projecs that are automatically built on CI servers in side corporate networks, the possibilities to grab data and send it to somewhere on the internet are endless.

One security measure that somehow grew out of fashion over the last years, is at least on application servers, to disallow any outgoing network traffic, especially to the internet (at least any cloud environment I see nowadays allows it by default). This would largely prevent these sorts of attacks from being able to actually send anything out, but also prevent XXE attacks from happening, prevent reverse connections to an attacker host from being set up, make SSRF attacks harder to verify, and so on.

I strongly recommend whitelisting only the network traffic that your application actually needs.

pouta• 7 years ago

How would this work for a public facing API? Or an API that serves a SPA?

I'm interested in this approach

objectified2• 7 years ago

I'm not sure I understand your question correctly, but I'm talking specifically about outbound network traffic. Your API's application servers (where such evil libraries could be deployed) should not be able to have any network connectivity towards the internet. So on that server, you should not be able to do even `curl www.google.com` for example.

zbentley3• 7 years ago

objectified has it already, but to reiterate: you can block outbound traffic initiated on a host without blocking outbound traffic that is a response to externally initiated traffic. This is, for example, what haproxy, iptables, and AWS security group outbound rules do.

I'm deliberately avoiding the term "connection" above because new UDP-first protocols require slightly different handling to determine who initiated what, but most routing/firewall software can deny-initiated-outbound for those protocols as well.

novaRom• 7 years ago

Is it possible to find out where that IP is located?

keyP5• 7 years ago

One thing I've always thought would be a good idea is a tool (either local or part of the pip/other packet manager download process) that greps and prints out all URLs and IP addresses within the code, including common encodings. Additionally, any lines that uses any transfer protocols (like HTTP requests) should be highlighted too as IP/urls can be encoded. Any HTTP request, for example, to suspiciously encoded URLs could raise flags.

The official library itself could have a "urls" file which has a list of urls that are expected and so anything that doesn't match can be questioned.

Whilst this won't solve the issue 100%, it raises the difficulty barrier to implement outgoing network calls.

jstanley2• 7 years ago

it raises the difficulty barrier to implement outgoing network calls.

Not very much. You just obfuscate your code until this tool doesn't notice anything untoward, and then upload it.

keyP5• 7 years ago

Highly obfuscated code would raise suspicions, especially in similar cases found in NPM packages.

E.g. in Python, obfuscators I've come across tend to replace characters with non-Latin unicode chars, which should raise flags when found in a predominatenly latin based source code.

sciurus• 7 years ago

Does anyone know where this was officially announced? I don't see anything at https://mail.python.org/archives/list/security-announce@pyth...

whb07• 7 years ago

The solution to this is WASM. Where you compile a cross platform library like this, and due to the sandboxing it isn't able to arbitrarily access memory outside its package.

samwillis• 7 years ago

Does anyone know of a way of running something like “little snitch” on Heroku so that you can whitelist outgoing connections?

That seems like a possible way of mitigating this type of issue.

wglb2• 7 years ago

This shows the inadequacy of thinking “open source makes all bugs shallow”.

jolmg3• 7 years ago

On the contrary, this wouldn't even need to be hidden if it was closed source. It was caught because it's open source.

wglb2• 7 years ago

Possibly, but some of the solutions proposed, e.g., monitoring of network activity, would work either way.

It concerns me that one of these sat out there for a year.

OrangeTux• 7 years ago

At pypistats.org download numbers of the last half year can be found.

* python3-dateutil has 271 downloads from non-mirrors in last month[1]

* jeilifish has only 106 downloads from non-mirrors in last month[2]

[1] https://pypistats.org/packages/python3-dateutil

[2] https://pypistats.org/packages/jeilyfish

riyakhanna19832• 7 years ago

Im assuming that by "only" you mean there's limited impact. However, if the malicious package steals user keys, the harm can spread to the packages that may have received way more downloads.

burrnii• 7 years ago

Best practice: Don't use pip search && pip install. Search for the project site and copy & paste the pip install instructions.

yingw787• 7 years ago

Does PyPI offer package notarization and make that observable in the lockfile or the installation logs? Or offer optimized SEO for notarized packages over those not notarized in package search? If that’s not there, and I don’t see it as part of the PyPA roadmap, it might be a good first step to take.

kardos• 7 years ago

Could Python be retrofitted with import flags, much like the openbsd pledge? Eg you could import a library but not permit file io or network access -- that would snipe these kind of attacks, and it would be a reasonable restriction for many libraries

giis• 7 years ago

Article points out Gitlab account olgired2017 (https://gitlab.com/olgired2017). I think it will be interesting if GitLab shares his/her list active session details like ipaddress, browser, date and time from their site logs (profile/active_sessions).

saagarjha5• 7 years ago

I would consider this to be a huge breach of privacy.

tanderson922• 7 years ago

I wonder if the users who were affected by this malware think something similar.

jmstfv• 7 years ago

Another insidious exploit is to hijack the maintainer's package manager account and push the code directly there, bypassing the repository altogether. It doesn't rely on you installing a new package since the hijacked package in question is already a dependency of yours.

I got into a habit of checking both the package manager and the project's repository before updating any given dependency.

https://news.ycombinator.com/item?id=20377136

codedokode4• 7 years ago

Why should Python program have access to SSH keys? Popular Linux distributions, unlike proprietary systems like Android or iOs, cannot protect user's data from malicious programs run by the user.

Also, in popular Linux distributions programs can read unique hardware identifiers like MAC address or HDD serial number, read browser's history and cookies. These valuable data are not protected by Linux.

3fe9a03ccd14ca5• 7 years ago

Mac has started prompting users when programs access the file system. I hope they implement a HUGE warning is that directory is ~/.ssh

These valuable data are not protected by Linux.

That’s extremely scary to me.

NieDzejkob• 7 years ago

Android isn't proprietary.

On the other hand, Windows and macOS, which, as far as I am aware, also have this problem, are proprietary.

Hence, I don't see why an argument like "unlike proprietary systems" is justified.

pjmlp2• 7 years ago

Windows and macOS have been pushing for sandboxes for quite a while as well, exactly to prevent this kind of behavior.

Currently macOS is more agressive than Windows on this area, with Apple now requiring notarization for all software, which you can still bypass, but need to explicitly allow it.

acdha• 7 years ago

If you want a concrete hardening step to avoid this attack, try using a hardware PIV/CAC device (e.g. a Yubikey) as the only copy of your private keys.

This is very easy to setup on MacOS High Sierra or later (https://support.apple.com/en-us/HT208372):

1. Generate the key: https://developers.yubico.com/yubico-piv-tool/Actions/key_ge...

2. Use "ssh-keygen -D /usr/lib/ssh-keychain.dylib" to extract the public key fingerprint to put in your authorizes keys list.

3. Add this line to your SSH config file to tell the client to attempt to login using the key on your device: “PKCS11Provider=/usr/lib/ssh-keychain.dylib“

On Windows, Putty-CAC supports this and can reportedly be used with Git: https://piv.idmanagement.gov/engineering/ssh/#ssh-using-putt...

leovander• 7 years ago

Some of the permission comments reminded me of deno[0].

[0] https://deno.land/

tus882• 7 years ago

You know allowing people to upload libraries with highly conflicting names to existing ones is almost reckless.

There is a bunch of stuff PyPi could do short of full curation that would make this much harder.

A better solution is for languages to provide a kind of "module sandboxing" where modules need to declare the capabilities they need, and the runtime prevents them from accessing anything else.

In this case, the module would have needed to request the ability to make an outgoing TCP/IP connection - and that request should raise red flags for a date parsing utility.

rob-olmos• 7 years ago

Sandboxing in the language runtime is an interesting idea. I wonder if that could be offloaded to the OS/kernel instead so that every language doesn't have to reinvent the wheel. Eventually then hardware could handle it to reduce overhead.

Eg, declare parts of the compiled/interpreted code with specific privileges. Kind of like how memory regions can be NX.

JackRabbitSlim• 7 years ago

MyPy requests permission to write data to the hard drive. MyPy writes malicious payload to a file and then execs it. MyPy never wrote a damn thing to a socket, netcat did through a bash/ash shell. (Windows named pipes may be a little trickier to work with but can yield similar results)

Sandboxing libraries specifically seems like a fools errand.

tus882• 7 years ago

Writing to HDD would be considered a suspicious permission for most modules. Like opening network socket or patching another module's namespace.

I don't know exactly how it would work. Maybe modules that request these permissions get extra scrutiny, or users can specify "levels" that different modules can run at. If you specify MyPy to run at the least-priv level, it will fail to install/load if it requests greater capabilities.

I can't really think of any other way around the problem. It is a problem for all other scripting languages, especially nodeJS.

jxramos• 7 years ago

I wonder if employing some string metric threshold would make sense. It would be interesting to measure the dispersion of names across pypi along some chosen metric. https://en.wikipedia.org/wiki/String_metric

panarky• 7 years ago

Original source: https://github.com/dateutil/dateutil/issues/984

maxander2• 7 years ago

So they caught the guy using a cliched I-vs-l typosquatting scheme and lazily writing the malicious code in Python; we can presume they haven't caught the guy who took the trouble to put their malicious code in a pre-compiled C extension.

Reminds me of the fraudulent scientific papers that get caught using really dumb fakes (e.g., microscopy pictures that are copies of one another re-zoomed and rotated); we catch the dumb ones, but presumably not all malicious actors are dumb, so there must be a lot more fraudulent work out there.

For that matter, isn't there a reasonable systematic way to catch out typosquatters simply based on text analysis? Any library name that's a short edit distance from a popular library should have been carefully reviewed from the start; there's no excuse for "jeilyfish" to have lasted more than a couple of days.

commandersaki• 7 years ago

For that matter, isn't there a reasonable systematic way to catch out typosquatters simply based on text analysis?

You could probably write one using Python & jellyfish.

CGamesPlay• 7 years ago

I'm extremely disappointed to see that you didn't suggest using jeIlyfish instead.

maxander2• 7 years ago

From jellyfish’s pypi page: “ a library for doing approximate and phonetic matching of strings.”

Huh! Hadn’t realized. That adds an ironic spin to the whole thing.

Animats• 7 years ago

Can the person responsible be found?

techntoke• 7 years ago

This is why the Node-ecosystem is plagued, because many apps will require hundreds of unvetted libraries.

stelonix• 7 years ago

I don't see a solution other than every application being launched in a separate container, exposing only what the user explicitly gives access to, similar to how mobiles applications require permissions. We have the technology, eg. cgroups & unshare on Linux; what's missing is something that plumbs all these brittle pieces into a secure application launcher made for desktop (rather than server/cloud) usage.

dpc_pw• 7 years ago

Ideally programming languages should include capability-based access control, so that a random library that is supposed to do X, can't do Y.

Until then, we need to vet our dependencies. Check out https://github.com/crev-dev/cargo-crev/tree/master/cargo-cre... for a distributed review system we're working on.

johnklos• 7 years ago

The server to which these libraries upload stolen keys, 68.183.212.246, is with Digital Ocean. As of Thu Dec 5 12:50:10 UTC 2019, it's still up with ssh open.

Nice job, Digital Ocean!

neop1x• 7 years ago

I am curious when the hackers will start stealing ~/.kube/config. In the default Kubernetes install on-premise, a token or admin certificate is just laying there, unprotected and adding a passphrase to the cert is not supported. Some are using an oauth identity provider or other mechanisms but that unnecessarily complicates the setup and smaller k8s clusters could be stolen this way...

Two malicious Python libraries caught stealing SSH and GPG keys

Related Stories