Linux 5.13 Reverts and Fixes the Problematic University of Minnesota Patches

https://www.phoronix.com/scan.php?page=news_item&px=Linux-5.13-UMN-Fixes

by varbhat • 5 years ago

262 111 5 years ago

dwheeler• 5 years ago

Just to clarify, all the proposals that were intentionally vulnerable and were really vulnerabilities were not accepted in the first place. However, that event triggered review of all University of Minnesota proposals, and that's what's being discussed here.

Glavnokoman3• 5 years ago

"ALL the proposals that were intentionally vulnerable and were really vulnerabilities were not accepted" The thing is there is no way you can actually know that. So this is not some kind of revenge. This is rather a valid precaution.

Foxboron7• 5 years ago

The thing is there is no way you can actually know that.

We do know since the researchers have told the linux community, the university and IEEE what those patches where. Please do not spread further misinformation about the case.

Please read the IEEE statement and the full Linux TAB review.

https://www.ieee-security.org/TC/SP2021/downloads/2021_PC_St...

https://lkml.org/lkml/2021/5/5/1244

foldr• 5 years ago

As usual, The Onion anticipated this: https://youtu.be/FpN_RjIaVw8

wrs• 5 years ago

Just from a code quality process standpoint, that’s an interesting result. Now I’m wondering what would happen if you picked a set of 150 random kernel patches and told 80 reviewers to re-review them assuming they could be malicious. I bet you’d find quite a few fixes.

microtherion• 5 years ago

I was wondering the same thing, and it appears it would be worth the effort, but getting enough high quality reviewers would be a problem.

Maybe the NCAA could organize competitive code reviewing leagues? I bet you would get e.g. a highly motivated Caltech team reviewing USC contributed patches, and vice versa.

bombcar• 5 years ago

I'd suggest it sounds like a perfect project for some university students, but that may not go over very well right now.

admax88q• 5 years ago

Probably, but the fact that maintainers are stretched thin when it comes to time for code review is not really news.

bluGill• 5 years ago

That is the openBSD model. Or at least it was 10 years ago when I last looked into it.

burnished• 5 years ago

This would be an interesting study.

dec0dedab0de4• 5 years ago

What is really sad, is that this could be a good pen-test for the kernel. Especially the idea of introducing bugs that are only vulnerabilities when they all come in together.

If only they had contacted the Linux Foundation ahead of time to get permission, and set up terms, like a real pen-test. Then work could be done on detecting, and preventing these sorts of attacks, maybe resulting in a system that could help everyone. At the very least a database for security researchers that are registered, but unknown to maintainers, where they could store hashes of bad commits. Then a check if any of those commits made it through. I know the kernel makes use of rebasing, so that might not be the best approach technically, but something like that. To ease the pain of wasting developers time, sponsors could put up money that the maintainer gets if they catch it, and maybe a smaller amount if they have to revert it.

EDIT: if the Linux Foundation said no, they could have tried another large open source project with a governing body, Apache, Python, Postgres, Firefox, etc. It wouldn't have been as flashy and high profile, but it would have still been the same research, and odds are you would find at least one project willing to participate.

mimimi31• 5 years ago

Why would the Linux Foundation get to decide on if those researchers are allowed to experiment on and waste the time of volunteer developers?

hinkley2• 5 years ago

The Linux foundation is in a position to ensure that Linux users don’t accidentally get experimented on when the PRs are approved.

dec0dedab0de4• 5 years ago

Why would the Linux Foundation get to decide on if those researchers are allowed to experiment on and waste the time of volunteer developers?

Granted, In the case of linux, it might make more sense if it were the combination of the Linux Foundation & Linus. It's their project, they can subject their volunteers to any tests they want. It may drive away some volunteers, but wether to take that risk or not, is up to the project to decide. For something as big as the kernel they may decide to get permission from the individual maintainers, maybe even limit the research to only the subsystems that agree to participate.

In any case, the point I'm trying to make, is that this kind of testing may be beneficial, if the project is aware of it, and agrees to it. Who gets to decide on behalf of the project, would depend on the hierarchy of each project.

jfrunyon• 5 years ago

Presumably they would do so after consultation with kernel developers, like uhh... themselves.

ret2plt• 5 years ago

I think it depends on how you frame it. If the Linux Foundation thinks this kind of research would generate useful information for the kernel project, then the developers' time wouldn't be wasted, just used in a different, yet productive, way. I concede that this is not an easy question, because the developers may have different opinions about the usefulness of this exercise, but at the end of the day, maintainers can run their projects how they see fit.

coldpie5• 5 years ago

that this could be a good pen-test for the kernel

Not really. Everyone knows this flaw exists, the interesting part is how to fix it. Did you read the "suggestions" the researchers made in their paper[1]? They're clueless.

It's like pointing out that buildings can be robbed by breaking their windows. No shit. What do you want to do about it?

[1] https://twitter.com/SarahJamieLewis/status/13848800341465743...

MaxBarraclough2• 5 years ago

Agreed. As I mentioned in an earlier thread on this scandal, [0] we already know that security bugs can make their way into the kernel.

[0] https://news.ycombinator.com/item?id=26888129

dec0dedab0de4• 5 years ago

Not really. Everyone knows this flaw exists, the interesting part is how to fix it. Did you read the "suggestions" the researchers made in their paper[1]? They're clueless

Fair enough. The kernel maintainers are probably much more aware of this than the average open source project. Maybe for some projects it would change their mindset from knowing that this could be happening, to knowing that this will be happening.

Maybe it's a pipe dream, but I have a feeling it could lead to discussions of "what could we have done to catch this automatically," which in turn would lead to better static analysis tools.

Edit: It would be about as useful as pen testing that includes social engineering. That is to say, everyone knows there are dishonest people, but they may not be aware of some of the techniques they use.

908B64B197• 5 years ago

How else were they supposed to get that much exposure for their tier 2 CS program?

teknopaul3• 5 years ago

This. and more so, they should be charged for the broken windows. dev hours times 80.

willis936• 5 years ago

The stated motivations of the researchers make it seem like they are not interested in advancing the security of open source software, but rather undermining it.

squarefoot• 5 years ago

Then "on behalf of whom" should be the next question.

guilhas• 5 years ago

Maybe it is more productive to just do the opposite. When vulnerabilities are found investigate the committer(s) for possible links, and quality of previous work

greenwich263• 5 years ago

The guilt by association here is now approaching Biblical scales. Like the guy in the comments who wants to close down the entire Computer Science and Electrical Engineering departments at UMN, which probably employ/educate the best part of a thousand people. Ha!

In general, the fury and seethe which this experiment inspired is amazing. IMO the real disgrace is not the experiment itself, but the response. The kernel developers need to stop being martyrs and playing blame games. They need to be rational and take responsibility for improving their own procedures. Because, if they didn't already have them, governments now have entire departments studying how to use deliberate vulnerabilities in open source projects, for military intelligence and other purposes. And they will not be deterred by the continued public flogging of the University of Minnesota.

belval2• 5 years ago

Most maintainers are volunteer, meaning that they spend their evenings, weekends and holidays developing the kernel because they like it, it gives them a sense of worth and fulfillment that can be hard to come across.

I don't know if you've ever talked to one, but they take a real pride in their work, most make a pitiful salary in comparison to FAANG levels but they still do it because it is full of interesting challenges you can't find anywhere else.

UMN broke that trust. No one is asking the departements to close least of all the maintainers, but you have to understand that kernel devs are not a faceless machine. They spend their limited time on something that is used to make a prodigious amount of money, while never really getting any. In that context they are more than within their rights to be livid.

nitrogen2• 5 years ago

In that context they are more than within their rights to be livid.

They do, and to a very largr extent the same exact overreaction happens in private orgs and we just get to see it when it's the Linux kernel. But at the end of the day, there was an overreaction, and it was public. Both the UMN and the kernel maintainers need to step up and make improvements now, and move forward with level heads.

Have any of the kernel maintainers acknowledged the primary concern behind the misguided research? That is, (quoting parent comment) "governments now have entire departments studying how to use deliberate vulnerabilities in open source projects"?

greenwich263• 5 years ago

I don't dispute that the researchers were dishonest and broke the trust of the kernel maintainers. So they can be a bit peeved off at UMN. But is this sort of thing not the reality of their operation? That liars exist in the world? I don't really see what the volunteer part has to do with it. If anything it means they have less to complain about. I mean...most soldiers are volunteers, which would make it even more absurd, if someone signed up to be a soldier, and then whined and complained when enemy soldiers shot at him. It seems equally obvious to me, that an open network should be assumed to have malicious actors on it, as that a battlefield should be assumed to have enemies on it. Obviously you don't have to like the enemies.

asddubs• 5 years ago

I don't think it's really fair to pin some random person's bad take on the kernel developers. disproportionate calls for mob justice have always and will always be popular, especially on the internet where it's very easy to leave casual comments without thinking about it too hard (and you can never really tell someones age)

bronson3• 5 years ago

They lied in their abstract [0], they lied to their IRB, and Kangjie Lu's aspirations have been wasting a lot of Linux maintainer time.

The real disgrace was the experiment. The response was fairly natural for humans with imperfect information being experimented on.

Please be more specific. How do you think the Linux kernel maintainers should improve their procedures to prevent clownshows like this in the future? No points for mentioning things that they already do or assuming infinite maintainer hours.

[0] https://twitter.com/SarahJamieLewis/status/13848760502079406...

greenwich263• 5 years ago

I just don't understand why open source software can continue with the assumption that every single contributor is honest. Imagine if this philosophy was applied to, say, cryptography or network security. What would be the state of encryption algorithms and key exchange protocols and so on, if their developers had a meltdown at the mere suggestion of there existing a liar?

Since the Linux kernel is installed on many millions of computers, it is obviously pretty important that it doesn't have bugs in it. Certainly not malicious bugs. And if all it takes is a couple grad students and an assistant prof to get them in...well, that reflects very poorly on the state of kernel maintenance, to me. Which seems far more important and deserving of attention, than endlessly arraigning three clueless guys at some university.

I'm not in a position to be more specific about what should be fixed. But, what would your answer be to your query? Apparently, do nothing, and assume that everyone in the world is honest, while writing self-indulgent "public letters" about it? How is that going to help when the CCP tries to insert surveillance into the kernel? Or when Russian hackers try to get exploits and ransomware in there?

balozi• 5 years ago

The moral of this story is this: whatever you do, don't be the dweeb that gets their code closely reviewed by the kernel maintainers. (You are actually better off having it reviewed tho)

AtNightWeCode• 5 years ago

Maybe the people reviewing the changes should be unaware of who made them. I am biased and for sure look more closely at some developers pull requests than others.

63• 5 years ago

For the sake of spotting malicious actors, wouldn't it make more sense for reviewers to be aware, so they can focus their attention on patches from untrusted sources?

teknopaul3• 5 years ago

honestly fsck these "pen testers", spend some time trying to make things better rather than trying to break shit and finger pointing.

Starting to feel like pen testing is a broken profession.

kzrdude• 5 years ago

These were not pen testers by profession.

devit• 5 years ago

So what are we doing about all the maintainers that initially approved these commits that were later found to be incorrect?

hda2• 5 years ago

the same thing we do to a carpenter whose door get broken into in a robbery: nothing unless they were inexcusably incompetent or complicit.

google234123• 5 years ago

Isn't the moral of the story here that it's probably trival for organizations like the FSB/NSA/Chinese equivalent to get malicious patches accepted into Linux.

maaand• 5 years ago

They already have malicious backdoors at bios level ref: intel-me

pumaontheprowl• 5 years ago

Not after Linux banned University of Minnesota from submitting patches. We're safe now.

floor_• 5 years ago

Anyone else notice the site linked has the most crazy racist wigged out psychos on the comments/forums? Yikes.

Linux 5.13 Reverts and Fixes the Problematic University of Minnesota Patches

Related Stories