AI, Encryption, and the Sins of the 90s

https://www.ndss-symposium.org/ndss2024/keynote-meredith-whittaker/

by keepamovin • 2 years ago

49 27 2 years ago

We want e2ee. But we also recognize that e2ee is not going to deploy itself, and that the business incentives in place currently do not allow for the kind of broad privacy protections I believe we need.

When the crypto wars of the 90s came around, I used to think technologies like e2ee and PKE would be something useful for a better world. Now it seems to me that with every new technology like Signal always has an opposing force against it (corporate control of the internet) that makes the system always slightly net negative in terms of benefits to humanity. I think we should scrap the whole thing and start over.

pixl97• 2 years ago

Scrap billions in infrastructure and software. You can see why this won't happen, right?

vouaobrasil3• 2 years ago

What I can see is that it won't be an immediate action by the current power structure. But what I can also see is that it is leading to a corporate-controlled oligopoly that is also fundamentally unstable. So while no one in today's big-tech society will scrap the internet, it may also come down regardless due our inability to make wise societal decisions that transcend micro-moves restricted to the insane, current economic options.

0xDEAFBEAD4• 2 years ago

There's no point in starting over from scratch if you can't explain how things would go differently. And if you can explain how things would go differently, it's worth considering whether that could be a patch on the existing system, instead of paying the cost to rebuild from scratch and possibly introduce a new set of problems.

I suspect many who advocate for burn-it-all-down utopianism are not interested in solving problems for their own sake. It's the same impulse that inspires engineers to rewrite perfectly good software just because they don't like some aesthetic details of the code.

That's fine if you do it on your own time. Go ahead, create a small-scale utopia and see if it works as well as predicted. Even if it fails, it can serve as a valuable data point for the rest of us.

vouaobrasil3• 2 years ago

I suspect many who advocate for burn-it-all-down utopianism are not interested in solving problems for their own sake.

At least for me, the internet is both necessary and a horrible experience. So actually it is motivated from a personal perspective to have something better.

masfuerte• 2 years ago

Counterpoint:

https://blog.cr.yp.to/20241028-surveillance.html

0xDEAFBEAD4• 2 years ago

I really miss this quality and style of internet argumentation. Wish we saw it more nowadays.

dannyobrien• 2 years ago

If I can point to another example that demonstrates that this can still be done: Christine Lemmer-Webber, one of the editors of the ActivityPub standard, and Bryan Newbold, who works on BlueSky's AT Protocol discussing at length the philosophy, terminology and the pros and cons of the various social media protocols emerging at the moment:

[1] https://dustycloud.org/blog/how-decentralized-is-bluesky/

[2] https://whtwnd.com/bnewbold.net/3lbvbtqrg5t2t

[3] https://dustycloud.org/blog/re-re-bluesky-decentralization/

saurik• 2 years ago

The closing point made in the final paragraphs of this counterargument--about how this false dichotomy is being presented as a tradeoff that inherently would influence future action away from defending against governments--can also be directed at Moxie's talk a number of years ago that made it seem as if working on decentralized systems is making the world worse somehow... I guess this form of narrative is popular at Signal?

0xDEAFBEAD4• 2 years ago

For people concerned about mass surveillance, what is your biggest worry?

Worry #1: People who wish to avoid mass surveillance aren't able to do so, because the tools available for fighting it (such as Tor, Signal, etc.) aren't sufficiently powerful

Worry #2: Although the tools for avoiding mass surveillance are powerful, not enough people are adopting them

Worry #3: Something else

I have my own answer, but I'm curious what others will say.

lubujackson• 2 years ago

To me, it is a boiling frog situation where each incremental loss of privacy is no big deal and nobody is actually looking at my data in particular so I don't care at all about that. And I don't care about my own privacy beyond any average person and I don't use any tools to hide my identity or whatever.

The worry is not a personal one, or even a systemic one, but a concern over the general direction of data availability and societal fragility.

Take, for example, the drone scare in NJ right now. The problem isn't what the drones are doing but the potential they have to do any number of harmful or invasive things. Right now I can go to Costco and buy a drone with an HD camera and hover it outside your bedroom window. Or have it sprinkle anthrax on your head when you walk outside your apartment.

The problem is that technological advancement far outpaces our ability to reason or control its usage effectively. Regulation lags misuse and eventually something (drones, nanobots, whatever) is going to lead to a massive and irreversible calamity before we change our starry-eyed rush to embrace the "new".

Sometimes I think the Amish have the right approach, though a little extreme. They aren't 100% anti-technology but meet every year to vote on if something new should be adopted, only after considering all negatives and secondary effects.

I am a realist, though, so I just live my life and brace for the eventual impact.

AnimalMuppet• 2 years ago

The problem is that technological advancement far outpaces our ability to reason or control its usage effectively. Regulation lags misuse and eventually something (drones, nanobots, whatever) is going to lead to a massive and irreversible calamity before we change our starry-eyed rush to embrace the "new".

To paraphrase Lord Of The Rings, those who have no drones can still die by them. As long as drones are being made in China or Iran or anywhere in the world, a terrorist/non-state actor/motivated assassin/special ops unit can get them and use them against you. Banning them in the US may slow this down, slightly, but it won't prevent it.

int_19h• 2 years ago

The tools, even if made illegal (which I think is inevitable long term) will still be there.

My worry is that avoiding mass surveillance will require the level of disengagement from society that is too extreme to be sustainable for the vast majority of people concerned about said surveillance.

And I would further argue that this is already the case.

unethical_ban• 2 years ago

All.

And I'm worried about AI making it easier for governments to basically have infinite, "intelligent" eyes and ears on every camera and conversation being recorded. There won't be such a thing as being unnoticed.

evanjrowley• 2 years ago

In terms of tools, my outlook is optimistic. I feel like worries #1 and #2 being mitigated by tools that enable E2EE in spaces where it was not available before. Examples include CryptPad[0] for office productivity, Ente[1] for photos, and Joplin[3] as a full-featured notes app supporting encryption. In the category of common everyday tools, I'd like to see more E2EE options for managing bookmarks across browsers.

Additionally, I would like to see more E2EE applications competing with popular SaaS offerings. It scares me to think of the potential damage that could come from a breach of ServiceNow, Atlassian, and any other SaaS where businesses expect to store private information. Given the US governments' proclivity to declare economic sectors as "critical infrastructure" plus recent incidents[3][4] affecting major cloud services, we can only expect increased levels of scrutiny over SaaS security.

Living in the US, my primary worry about mass surveillance is less about the technology or the adpotion. I'm fortunate to live in a place where there is a low risk of violence due to authoritarian use of mass surveillance. My primary concern is the effects surveillance has on our collective decision making. The idea that we have less free will because surveillance keeps powerful instututions one step ahead of individuals is the topmost concern for me. I.e., the ending message of Metal Gear Solid 2[5].

[0] https://cryptpad.fr/

[1] https://ente.io/

[2] https://joplinapp.org/help/apps/sync/e2ee/

[3] https://www.npr.org/2023/07/12/1187208383/china-hack-us-gove...

[4] https://arstechnica.com/security/2024/01/microsoft-network-b...

[5] https://www.youtube.com/watch?v=jIYBod0ge3Y

BriggyDwiggs42• 2 years ago

Im worried that unpopular governments can entrench power using it, as is attempted in china.

kleiba3• 2 years ago

I would bet that almost everyone here values convenience higher than privacy - that is, privacy is generally seen as something very valuable, but in actuality, using plastic cards for all payments or point cards at the supermarket are just the two easiest examples of how we happily allow corporations to profile us, just because it offers convenience.

n4r92• 2 years ago

Although this is true, using cash is getting much harder over time. Here in London it's almost impossible to stick to cash as ATMs disappear and services stop accepting cash payments. To travel on a bus now you either have to use your plastic card or top up an Oyster card in advance. Some councils have removed their street parking meters so that you either use a parking app to pay by card, or you have to find and visit a Paypoint convenience store to pay (I have no idea how to do this!). Most of the self-checkout machines in my local Aldi are card-only. If you do try to pay with cash, you might struggle getting a lot of places to break a note as they're just not used to it anymore.

kleiba3• 2 years ago

you either have to use your plastic card or top up an Oyster card in advance

My point, convenience over privacy.

0cf8612b2e1e• 2 years ago

Not that I disagree with your point, but with credit card rewards, it is actually cheaper to use the card. So there is frequently a financial incentive as well. You can still find a few oddball stores who will give you a cash discount relative to the credit price, but those are not common.

kleiba3• 2 years ago

That's the whole reward card model. As the old adage goes: if it is free, you're the product. So, then I should extend my point to "convenience or money over privacy".

motohagiography• 2 years ago

Whittaker and Signal are only going to be able to hold up that umbrella from mass interception for only so long, and we need some fresh thinking in new directions, as there is no "governments vs. corporations" dichotomy anymore.

Instead of providing legal e2ee rights or protections for oligarchic platforms, breaking their hold on software distribution will do much more to ensure human growth and progress.

The big thing that happened since the 90s crypto wars was Snowden's "turnkey totalitarianism" prediction arrived and almost prevailed. The irony was the only thing that stopped it was the basic weaknesses of encryption and security on mobile phones (e.g. Android's fragmentation and some IC encryption sabotage) which made strong digital identity non-viable for policymakers to deploy at scale during the pandemic (newbs BTFO'd, lol. u kno who u are).

I was surprised as anyone, and my mind has changed about whether we should really want secure unhackable devices given who we've seen can organize to use them against us. The only thing that saved humanity from that was the crappy mobile device security that we in the privacy field had been trying to improve. I'm glad we failed. Not disagreeing with Whittaker, but we need some new thinking as the past does not resemble our present at all.

cancerhacker• 2 years ago

Interesting if possibly somewhat off topic (to this submission) - a discussion [1] about DES key length and some of the thoughts given to it in a taped meeting from 1976 -

[1] http://www.toad.com/des-stanford-meeting.html

AI, Encryption, and the Sins of the 90s

Related Stories