An update on our security incident

https://blog.twitter.com/en_us/topics/company/2020/an-update-on-our-security-incident.html

by pseudolus • 6 years ago

228 232 6 years ago

blog.twitter.com

dboreham• 6 years ago

No employee should have the power to subvert 130 high value accounts in a short time period.

djsumdog• 6 years ago

Why do we even have 'high value' accounts on a centralized platform?

Why isn't there a whitehouse.gov ActivityPub instance that no single admin can censor or subvert?

m4634• 6 years ago

We will do that now. We will start the competitive bidding process, and we expect the RFP paperwork to be returned by October, 2021. After that, if there are no injunctions filed because of the bidding process, preliminary design documents will start being created. Preliminary design review will occur August 2022. ...

vaxman• 6 years ago

This. Just as surely as email, texts and tweets are now admissible in contract law and as we can sign documents online that are binding, we need a national realtime information-streaming infrastructure --text and URLs today, multimedia in the future...all over the (hopefully) secure domestic 5G network. Set the standard, setup a court system for inevitable violators, regulate the gateways allow everyone to connect.

brown9-22• 6 years ago

Because the general public has no clue what ActivityPub is or the desire to learn how to consume feeds from dozen of instances when they could just download a single app onto their smartphone where all the celebrities already are.

riffic2• 6 years ago

Why isn't there a whitehouse.gov ActivityPub instance that no single admin can censor or subvert?

Legislation is sorely needed for public institutions to make public announcement messages (microblog posts, or "tweets") using publicly managed and controlled infrastructure, contributing back to the commons[1].

Stop building into broken commercial services, the standards exist today to rebuild a commons-oriented Internet.

[1] https://en.wikipedia.org/wiki/Commons#Digital_commons

0xUser• 6 years ago

Source (with more details): https://blog.twitter.com/en_us/topics/company/2020/an-update...

The social engineering that occurred on July 15, 2020, targeted a small number of employees through a phone spear phishing attack. A successful attack required the attackers to obtain access to both our internal network as well as specific employee credentials that granted them access to our internal support tools. Not all of the employees that were initially targeted had permissions to use account management tools, but the attackers used their credentials to access our internal systems and gain information about our processes. This knowledge then enabled them to target additional employees who did have access to our account support tools. Using the credentials of employees with access to these tools, the attackers targeted 130 Twitter accounts, ultimately Tweeting from 45, accessing the DM inbox of 36, and downloading the Twitter Data of 7.

nodesocket• 6 years ago

Why are internal employee tools publically accessible? Minimum they should require VPN access, but really go further with Zero Trust.

bawolff6• 6 years ago

The blog post is vauge but definitely implies that a vpn was in place.

whoisjuan3• 6 years ago

AFIK, in a Zero Trust Architecture a VPN is considered a perimeter and therefore it becomes a vector of attack to access systems of authoritative decision.

Many security researchers have already established that the benefits of a VPN especially in the modern distributed world are marginal at best.

Basically, yes a VPN makes you a tiny bit safer but it also adds a lot of networking complexity and adds more friction to the job of your employees. It also becomes an attack vector for malicious parties, since once they get VPN access they can theoretically access at least the first layer of protected resources.

So in layman's terms an attacker just needs to phish for VPN credentials, maybe steal an OTP token and they will have access to a non-trivial amount of network protected resources.

On the other hand if every service you use has its own authentication then the attacker needs to target each service and to know what services to attack they need knowledge that is possibly contained in another system that also requires authentication and is definitely not guaranteed for the attacker that all the systems will have the same password and/or have 2FA disabled.

Honestly, in my opinion VPNs are just an excuse to monitor traffic. This is a bit of cynical take, but I'm convinced that companies that use VPNs are more interested in seeing what goes in and out their network than in protecting their resources.

Thorrez36• 6 years ago

What if the attackers phish the VPN credentials too? Does Zero Trust imply phishing-resistant credentials? What Twitter needed was phishing-resistant credentials (security keys, aka U2F).

ThA0x2• 6 years ago

Another instance where zero-trust networking has utterly failed.

Security comes in layers. That first layer of requiring a VPN can stop many types of attacks from happening.

Next layer is requiring MFA for VPN access. Then for admin access, require MFA only from approved devices on the domain.

Large banks and the DoD have been doing this for years.

The "fail often and fail fast" crew are always reinventing the wheel after bad experiences. I honestly feel sorry for them.

ehsankia• 6 years ago

Were the spear fishing attacked also used to get 2FA, or did these accounts not have 2FA? Would hardware based 2FA not have stopped this?

zamalek• 6 years ago

I haven't seen a form of phishing that hardware 2FA doesn't stop. Yes, it would have.

technion• 6 years ago

Without details it's hard to say. They talk about "phone spear phishing". It's within the realm of that description to say they were hit the same way someone I know recently was - someone called and said "Can you just install Teamviewer on your desktop for me and give access to a logged on session".

lighthazard• 6 years ago

Maybe? I imagine a spear fishing attack could entail the target is sent to a false panel to login where it sends a true 2FA request. The target then freely gives this 2FA code to the attacker.

bananabreakfast• 6 years ago

It says right there in the linked page that the phishing attack was able to manipulate multiple employees into giving access past their 2FA.

andrew_• 6 years ago

I'd like to know more about these tools. That there's at least one which can bypass a user's 2FA settings without notification suggests that there are additional tools in the same vein.

spullara• 6 years ago

Every network has to have tools to do that. How else will they enforce the laws they are required to enforce?

ma2rten• 6 years ago

Google requires its employees to use a security key for access to all internal systems including admin tools, source code and email. Every since google started enforcing this policy the number of successful phishing attacks has gone down to basically zero.

Thorrez36• 6 years ago

without notification

Are you sure there wasn't a notification?

dang• 6 years ago

Ok, we'll change to that from https://www.reuters.com/article/us-twitter-cyber/twitter-say.... Thanks!

harikb• 6 years ago

ultimately Tweeting from 45

for a moment I thought it read `tweeting from 45's`

DoofusOfDeath• 6 years ago

Nah, you're thinking of putting a 33 RPM disc on the phonograph, then setting the playback speed to 45 RPM.

Hands down the easiest way to make Shaun Cassidy sound like one of the Chipmunks.

dpweb• 6 years ago

No disrespect to those challenged with protecting such a huge target, but why do admin tools even have these capabilities? I could see needing to disable a user account or change some attributes, but why would an admin ever need to tweet from it? There shouldn't be tools with God privileges even for admins. Not surprising human error was involved in a breach this huge. So, how many people had access to this tool? Is there a killswitch for the tool itself available to very few, really very few, persons? edit: I dont know if the tool can tweet but surprised 2FA can be stripped without a human being confirming (ie.. the acct owner's social media person), especially for famous people.

yipbub• 6 years ago

They probably had permissions to change/reset the access credentials which can be used to gain access as a the user.

Thorrez36• 6 years ago

I doubt the admin tweeted from the tool. The admin changed the email on the account, then did a password reset, then logged in as the account then tweeted.

dlkmp• 6 years ago

Did they tweet directly from the admin tools? My impression was that they used the admin tools to reset the password and then take over the account, ultimately tweeting like any normal user would do.

erk__• 6 years ago

It is my understanding that they used the tools to update the email of the account, then reset the password to log into and make a new password such they could log in and tweet. Do you have any source that says that they could use the support tools to tweet directly from?

wraithy• 6 years ago

Do we know for sure that the admin tools can do all this? My understanding was that the tools enabled password resets, which allowed the attackers to tweet from the accounts themselves.

For 45 of those accounts, the attackers were able to initiate a password reset, login to the account, and send Tweets.

geocar2• 6 years ago

You have a need for a tool allows you to see the UI “as the user does” so you can respond to support requests and maybe someone thinks it’s easier to just copy the cookie.

This isn’t the right way to do it, but given they work at Twitter I could imagine this isn’t the first big mistake they’ve made.

extrapickles• 6 years ago

The admin tool was used to change the email on an account, then the attacker reset the password and got full access to the account. Apparently having 2FA enabled did not stop this attack (admin tool probably had the power to strip 2FA from accounts).

So while the tool did not directly have the ability to tweet, it effectively did.

ameliaquining• 6 years ago

My uninformed guess is that there isn't a "tweet as this user" button (because obviously there's no legitimate use case for that), but there is a "change this user's email address" button (because you might need to do that in order to help someone who's locked out of their account), and if you can do that you can take over someone's account. Obviously something like this would be detected quickly, which makes it less scary in some ways than a "tweet as this user" button, but of course this particular attack did not seek to evade detection once it was launched.

Of course, some of the targeted users presumably had 2FA enabled. How to do account recovery with 2FA in a consumer context is a complicated problem and I'm not aware of any good answers, but there's certainly an argument that the protections in place there weren't adequate and I wouldn't be surprised to see them changing soon.

I would also hope that rank-and-file support staff can't change users' email addresses, and the attackers had to spear-phish one of a smallish number of people whom more complicated account-recovery cases are escalated to. But who knows if that's how it works.

f2000• 6 years ago

I've coded/supported a number of admin tools for a number of large companies (banks, telcos, etc) so I'll take a stab at "why" First, god mode can be implemented cheap and fast. God mode also makes day to day support easy as admins/support staff can do just about everything fast and easy - so they typically love it and will often fight to keep it. More than once I have had managers turn down proposals to tighten up security because of the cost and false beliefs that because the tool is behind a firewall, better security is not needed. Taking a schedule and/or budget hit to implement tighter security is not going to get them a promotion or bonus. Sad but true. In the vast majority of cases I've seen, paying for security improvements becomes incentivized for managers after their company gets burned by a breach.

Also, admin tools are often "afterthoughts", there is usually a motley collection of them, and often considered as an expense/cost to be minimized and not a revenue generating asset that gets more budget and attention.

throwaway345346• 6 years ago

Internal admin tools grow over time. They don't spring fully-formed into the modern company with all the correct access controls and auditing they ought to require at that point. They carry a lot of cruft from things that were needed early on and aren't needed anymore.

Furthermore, they're a classic cost center, not a lot of love or budget goes into reducing their tech debt, or bulwarking them up against a sophisticated adversary. Red teaming yourself full time is expensive and not profitable. What's the worst that happens from a breach like that? Well, Equifax is still going strong!

I recall being party to an amusing conversation at a major network services provider at a team meeting for people with access to such tools, to the effect of:

- Alright, we're modifying <internal tool A> to lock down access to accounts related to <major political figure>. You will no longer be able to use <internal tool A> on <accounts>, only select supervisors will have that access.

%%% ah, okay, that makes sense

# uh, hey, regarding <internal tool B>, which allows us to look up <thing that would provide equivalent access to internal tool A>? does that still work the same?

- Uh, yeah, it does.

# okay?

%%% ... silence ...

- Alright, next item!

To the best of my knowledge, that was never addressed. <internal tool A> has audit logs. <internal tool B> doesn't.

onefuncman2• 6 years ago

"Tweet As User" is the basis for almost all social media management tools and Twitter still doesn't have very fine-grained permissions grants. https://developer.twitter.com/en/docs/basics/apps/guides/app...

We had a running gag in our social media startup of tweeting "poop" from people who left their phones/computers unlocked ... someone did it to an employee that was logged in as a customer (corporate brand) context and that was the end of that 'joke'.

tallanvor• 6 years ago

Admin tools having the capability to change email and 2FA settings is a necessity, but Twitter clearly needs to greatly increase the security.

I don't know what all Twitter uses, but I know that many companies have various methods of authentication depending on how much damage can be done:

- Logging on using a username/password and 2FA is enough for some activities.

- More sensitive operations have to be done on hardware that has a certificate installed and backed by something like Windows Hello.

- Even more sensitive operations require a JIT account and a certificate stored on a separate hardware key such as a yubikey.

- Very sensitive work gets done on a secure device that is very locked down and can detect changes to the hardware that may suggest tampering.

- Some stuff simply isn't allowed to be done remotely, even with the above restrictions.

Obviously not every company needs such a complex setup, but for someone as high profile as Twitter, you'd expect more thought to be put into this.

MoZeu• 6 years ago

It is inexcusable that Twitter is employing people who are susceptible to social engineering attacks like this. This is simple training and seriousness.

t-writescode• 6 years ago

It just takes one mistake to be spearfished.

mikehollinger2• 6 years ago

I will freely admit that I fell for a phishing campaign. I’d just bought something on eBay (this was a while ago). I got an email about something in my account later that day that made it through my spam filters. I clicked on it, signed in, and then realized I’d done the deed. Nothing happened or was lost, but yes - it just takes one quick mistake.

function_seven• 6 years ago

All companies employ people who are vulnerable to social engineering tactics.

All of them.

bawolff6• 6 years ago

Training that is notorious for being ineffective in practise and usually more about box ticking.

Assuming that none of your employees fall for phising, much less targeted phising, is woefully unrealistic. Especially at twitter's scale.

Assuming humans won't do stupid things 100% of the time is never an effective security control.

andykx2• 6 years ago

This is an excessively pessimistic take on security training. How many spear phishing attempts have been thwarted because the employee knew better?

It’s not a solution to the problem, but it certainly helps.

idoh• 6 years ago

Where I work there is training software that is somewhat effective at preventing phising - it actually sends out phising emails itself. Then employees who fall for it are given extra training (in a no fault sort of way).

jacquesm2• 6 years ago

You too could be social engineered. The worlds foremost security specialists are not immune, good chance that there is some social engineering vector that would work on you.

Admitting that to yourself is a huge step forward in being able to detect it. Believing yourself immune increases your chances of being spearfished.

Thorrez36• 6 years ago

The worlds foremost security specialists are not immune

I bet there are some that are immune. But yes, 99% of employees can be phished.

spike021• 6 years ago

You say this like human beings can be perfect.

Nobody is perfect.

Everyone is vulnerable given time/effort.

patagurbon• 6 years ago

Are account support tools available off premises? I know nothing about security for big companies like Twitter but it seems like tools that enable you to post from any verified user (outside of Trump, someone here once mentioned he had additional account controls) should only be accessible from secure offices regardless of individual credentials.

bawolff6• 6 years ago

That would probably not work well during the pandemic...

TwoBit2• 6 years ago

It could with an appropriately secured private VPN.

iamleppert• 6 years ago

They should require hardware security devices (dongles). Really Twitter should be ashamed of their poor internal security.

sushshshsh• 6 years ago

Dongles are rare here in the US. But I know that bloomberg uses them. I was shocked when I learned that retail banks in Singapore give everyone dongles to log in. In the US that's tyranny Lol

Thorrez36• 6 years ago

Rare for employees or rare for consumers? Companies can push much higher security onto employees than onto consumers.

TwoBit2• 6 years ago

rare? I worked at Facebook and every employee needs one.

marcan_42• 6 years ago

We're talking about companies, not users. Competent companies can and absolutely do require dongles (or equivalently trustable corporate hardware) to log in to their systems.

Osiris• 6 years ago

I work for a crypto currency company and it was the first time in my career that I was issued a YubiKey (I once had an RSA 2fa token for vpn access). It took some getting used to but now I just keep in on my keychain and I always have it with me. I need it for SSO, git, VPN, and basically all internal services.

They aren't sufficient by themselves however, they don't protect from is malicious internal employees.

FireBeyond• 6 years ago

When I worked at MSFT in 2006, remote access required a smart card plus credentials.

Thorrez36• 6 years ago

Note that not all hardware security devices are safe. U2F security devices are safe against phishing; OTP security devices are not safe against phishing.

Closi2• 6 years ago

U2F are safe against simple phishing, but sophisticated attacks can include 2-factor input.

qzio• 6 years ago

what about a phishing attack that involves the attacker talking the target through a login, which also involves the u2f part? and then obtaining the session token from memory/disk instead of username/password

thdrdt• 6 years ago

It depends on the attack.

For example if a Twitter user was logged in with a dongle but the attacker had access via social engeneered remote desktop access a dongle still could mean access to private data.

But yes. As far as I know Google and Facebook require them. Also Google sometimes require permission of an other co-worker to access data.

ngomez• 6 years ago

if a Twitter user was logged in with a dongle but the attacker had access via social engeneered remote desktop access a dongle still could mean access to private data

It depends on the dongle. YubiKeys and similar devices require the user to physically touch/tap it to enable U2F auth, and it automatically powers down after a timeout to prevent remote desktop attacks.

I would hope Twitter already had this kind of setup, but their blog posts about this are all targeted at a more general audience, so I doubt we'll get that kind of detail anytime soon.

ridaj• 6 years ago

Freaking Twitter needs a serious auth infra upgrade. Unless phishers hijacked employee devices, they accessed the tools remotely, meaning there's no form of client authentication?? Something like U2F which by now is pretty old seems like it would prevent this kind of attack

zobzu• 6 years ago

what if.. they used the authentication?

But yes u2f/webauthn would probably prevent this.

That said keep in mind they also do PR/damage control so we only know what they tell us. For all we know maybe they have u2f and an employee still did bad stuff while a phone was somehow involved. Or whatever else.

reportgunner3• 6 years ago

Yes they gained working credentials from an internal slack channel by reading them in plaintext.

https://www.dailymail.co.uk/news/article-8536603/British-tee...

woadwarrior01• 6 years ago

Jack Dorsey's twitter account getting hacked via a SIM swap attack last year[1], should've been a wake up call for them.

The recent hack just shows how lackadisical their attitude to security is.

I used to work at FB some years ago and they had U2F for everyone, even back then. Also, regular phishing test drills and red-team exercises.

[1] https://www.wired.com/story/jack-dorsey-twitter-hacked/

badrabbit10• 6 years ago

As someone who works to stop these, the most frustrating part is how even infosec people thik enough training or $vendor's email security solution will stop this. It's like boy scouts that think they will stop navy seals. There is too much focus on entry point of an attack,especially by news media.

iKevinShah2• 6 years ago

Hey,

Any good literature which you'd recommend to read to avoid something like this?

sudhirj• 6 years ago

I keep trying to read and understand the attacks that have all happened so far... after you understand the vectors that are currently in use, hopefully it'll stoke your imagination to see how you might attack other systems and make up new vectors. All the current attack write-ups will usually link to vulnerabilities as well. See HackerOne disclosures, Google Project Zero or the "How I hacked X" disclosures you see on HN.

I don't think published textbooks are very useful — attackers also have access to them, and if the attack has been written down it'll likely be encoded into a firewall software or security process rulebook already (though it might still work for smaller companies lagging behind on the curve).

badrabbit10• 6 years ago

That's the thing, you can't,not with the way current tech is. But you can read up about having good monitoring/detection and hardening on your endpoints.

Microsoft for example recommenda privileged access workstations. If twitter's employees used a separate set of credentials and workstations for privileged twitter moderation than their regular account/machine used for email and day to day stuff I bet the attack wolf have failed.

varenc• 6 years ago

Use U2F for 2FA. If Twitter had all their employees using U2F keys it's very unlikely they'd be phished.

With U2F it's impossible to "enter" a 2FA code on the wrong domain, making you immune to phishing attacks by most definitions. This Kerbs article from awhile back says that Google had zero phishing incidents after making this switch: https://krebsonsecurity.com/2018/07/google-security-keys-neu...

EE84M3i2• 6 years ago

What infosec people think $vendor email security solutions are going to solve phishing attacks? I was under the impression that the people that buy those solutions (like many security solutions) are primarily non-infosec people that want to paper over their real problem without fixing it.

Granted, there is a place for some of these things temporarily while working to fix the actual problem, but that's a mitigation, not a solution.

badrabbit10• 6 years ago

Nope, seasoned pros I respect think trainig+$vendor is good enough. If it isn't, blame the user or the vendor!

There are shops where the goal is to have someone to blame when you get owned and there are rare shops where the goal is to do it right to catch/stop bad guys even if it means you get blamed (because management understand security is not absolute)

staticassertion4• 6 years ago

FWIW email security training is something you'll probably be forced to provide, to some degree, as a matter of compliance. It's another case of compliance wasting time by driving companies to do security work that isn't meaningful.

thaumasiotes• 6 years ago

Speaking from experience at a major infosec company, the impression I got internally was "we offer phishing tests, but we don't recommend them, because phishing succeeds 100% of the time".

So I'm confused by the idea "even infosec people think training will stop this".

Thorrez36• 6 years ago

I think badrabbit should have said "even some infosec people think training will stop this".

As-is the grammar is ambiguous whether badrabbit meant "some" or "all".

TheAdamAndChe• 6 years ago

I disagree. There are services that regularly send fake phishing emails on a regular basis. If they click a link or fail to flag enough emails, their boss gets notified that more training is necessary.

At the bank that I see this used at, the employees are far less trusting of emails and such.

Training works if it's done right.

badrabbit10• 6 years ago

Maybe infosec company is different than infosec at $bigcorp. I worked at a few places and the sentiment is the same.

Thorrez36• 6 years ago

There is too much focus on entry point of an attack,especially by news media.

That depends on what you mean by "entry point". If you define the entry point as a person, then yes don't focus on that. But if you define the entry point as phishable credentials, then focusing on that is good, it will prompt companies to switch to phishing-resistant credentials (U2F security keys).

badrabbit10• 6 years ago

Even u2f isn't fool proof (exploitation and cookie theft techniques). Execution,privesc,lateral movement are things focus should be on. You can't control the facg that people need to use email and they will for for a phish,but you can control your authentication system, alerting system,etc...

snoshy• 6 years ago

Frankly, I love Twitter. They've always seemed like the "scrappy" big player in the tech space that's played just the right side of the norm in many cases. An appropriately filtered experience on Twitter as a user is actually quite fun.

That said, much of this is atrocious news. For all of their engineering prowess, seeing poor opsec failures combined with the lack of basic security principles like "containment of blast radius" and "fast response to critical failures" is not something you can easily forgive at this scale.

But who am I kidding... it would be especially rich if some of these takeovers were enabled by simjacking-like attacks. Not that long ago, the only two-factor auth mechanism that worked for me was SMS.

iKevinShah2• 6 years ago

Sorry if I am taking this on a tanget, but in one of the HN threads regarding this exact security incident, it was recommended that this is why something called as "Blast Radius" needs to be implemented.

Anyone here with any literature / sessions one could go through for a good gist of things with respect to Blast Radius?

AnonC2• 6 years ago

I did a quick search on Algolia for HN (hn.algolia.com) and found several comments in this thread and threads related to the recent Cloudflare incident. This could be a starting point in your quest. [1] I don’t have a blog post or article or documentation though.

[1] https://hn.algolia.com/?dateRange=pastYear&page=0&prefix=tru...

ak217• 6 years ago

"Blast radius" is a general term for the worst case impact of a specific type of breach of a given system.

The recommendation you read was probably about limiting the blast radius. It's a general security best practice, and you implement it through techniques like federating (compartmentalizing) services away from each other, limited lifetime credentials, attribution, SSO for single point of control for invalidation of credentials, principle of least access (PoLA), privilege separation with role-based access control (RBAC), session logging/audit logging, etc. Most importantly the underlying system needs to have a well-defined and pentested authentication/authorization architecture. The hallmark of systems that limit the blast radius is that they have well-defined limits on how much they trust each other.

OWASP (https://owasp.org/) is a great starting point for reading about this stuff.

ngcc_hk• 6 years ago

What they need for you is some Network ai (dark something) to see change in user behaviour eg access accounts and honey pot accounts that any internal staff would raise alert immediately

praveen9920• 6 years ago

We will be slower to respond to account support needs

So, they limited the account support access to smaller trusted team. This I can understand but it might also cause the delay in identifying such attack, if it happens today.

IncRnd3• 6 years ago

Account access is one thing, yes, and a hardware 2FA key can help with that.

But - what is the reason to allow support personnel to pose as specific users and send tweets from their accounts? There is more than a security issue here. There is a complete security breakdown.

Thorrez36• 6 years ago

I don't believe there was a tool allowing support personnel to pose as users. I believe the tool allows support personnel to reset emails on accounts. Then the attackers used did password resets on the accounts then logged into the accounts and tweeted.

IncRnd3• 6 years ago

I didn't see that in the page for this article. But, that's a good point.

The specific text is, "Using the credentials of employees with access to these tools, the attackers targeted 130 Twitter accounts, ultimately Tweeting from 45, accessing the DM inbox of 36, and downloading the Twitter Data of 7."

So, this doesn't say what actually happened. If it was employees posing as users in order to post, that is an permission which should not be granted. If it was as you suggest, a password reset, then there is a separate issue with 2fa that would be expected on these accounts.

Either way, there are serious security issue. This is similar to Oracle calling itself "Unbreakable" and then getting broken. If Twitter cannot safeguard against so many accounts getting injected with tweets, then something is broken with Twitter's security model.

heinrichhartman• 6 years ago

What I find most problematic about the attac is the incident response by Twitter.

As people pointed out here, hijacking Twitter accounts can lead to big stock market crashes, mass panics ("bomb found at XXX") and maybe even military escalations.

Under this circumstances, leaving a platform with an unknown number of compromised accounts online, seems irresponsible to me. In such a case you must stop the bleeding ASAP, either by locking up "important" accounts (what they eventually did, after a few hours!) or taking the site offline.

Next time this happens, we might not be so lucky.

bzb3• 6 years ago

No. We need more things like these to happen so people lose trust on everything they read on the stupid internet.

This was good. I hope it keeps happening.

bananaface4• 6 years ago

If they'd taken the site offline then the event probably would have generated more publicity.

staticassertion4• 6 years ago

At the same time, it's hard to take down a site that has the type of impact that you're referring to.

thereyougo• 6 years ago

We’re acutely aware of our responsibilities to the people who use our service and to society more generally. We’re embarrassed, we’re disappointed, and more than anything, we’re sorry. We know that we must work to regain your trust, and we will support all efforts to bring the perpetrators to justice. We hope that our openness and transparency throughout this process, and the steps and work we will take to safeguard against other attacks in the future, will be the start of making this right.

It's not easy to say those words about your own company.

I really believe them.

bananaface4• 6 years ago

I think it's a mistake to personify organisations like this.

baby-yoda• 6 years ago

Agree. That reads like pure PR-corporatespeak to me - oops sorry, wont happen again, business as usual.

“They” only “care” to the extent it materially affects the business.

bananaface4• 6 years ago

The spirit of Mitnick never dies.

jacquesm2• 6 years ago

Why would it have died? He's very much alive.

bananaface4• 6 years ago

He's not hacking.

toyg3• 6 years ago

the attackers targeted 130 Twitter accounts, ultimately Tweeting from 45, accessing the DM inbox of 36, and downloading the Twitter Data of 7

So much effort for so little gain... With proper preparation (i.e. a simple app ready to download everything from an account), they could have made out with the whole data of 130 accounts, silently, before tweeting the hopeless scam message. Instead, this seems like a mostly-manual effort, done in haste.

Just dumb thieves pulling off the scam of their life, or cover for a targeted attack of one or two of those 7 they actually siphoned properly? I hope US authorities will figure stuff out beyond the usual "it was a Chinese/Russian/Eastern European gang", which is just code for "fuck knows".

ACS_Solver2• 6 years ago

I imagine that the limited damage the attackers did will help them evade being caught. Ultimately, they just stole some 100k$ in Bitcoin and that's it, so investigations by the FBI et al. will probably not go to great lengths to locate the hackers. On the other hand, attempts to blackmail someone like Tim Cook or Elon Musk would probably be riskier. For the same reason, I suspect the attackers may have intentionally avoided breaching or tweeting from Trump's account, too much risk of attracting serious FBI and Secret Service attention.

As it is, the only people who actually lost something are a bunch of Bitcoin owners that got scammed, and it's just not a big deal.

toyg3• 6 years ago

In the great scheme of things, knowing what Trump (or any other US politician) says to other people is worth relatively little.

Having insider knowledge of what certain billionaires say to whom, though, can be immensely valuable. It might well be that a few of them will lit a fire under FBI and friends. We'll see.

vsareto• 6 years ago

I wish I knew why security does this character assassination routine with low-skill hacks. They clearly got in, bumbled attempt or not. Is it somehow helping everyone to know they fucked up? Whose expectations are we changing here?

spenczar5• 6 years ago

Who is “security” here? In my experience, if you ask most working security professionals, they would agree that social engineering is by far their largest vulnerability. All the cryptography stuff is fun and cool but really the work is all about preventing phishing, so I don’t know any working security engineers who would call this a low-skill hack.

_notreallyme_• 6 years ago

Well, I see 2 reasons here: 1/ when an attack is deemed a low-profile one, it is a way to say that the targeted company might be to blame, at least in part, for not having adequate security protocols; 2/ in this instance, some people might be frustrated that the DM of high profile people won't leak on the internet because of a lack of preparation from the attackers.

Most of the time, the "character assassination routine" is to emphasize the first point.

gcpwnd• 6 years ago

Slightly OT: I use private tabs and write my banks URL by hand and log in via 2FA. Are there cases or have there been attempts to poison the users address bar history with malicious URLs for fishing?

info781• 6 years ago

You should need a PIV card to log into in the admin tools.

exabrial• 6 years ago

This is one of the better rcas, but I'd wish they'd punish the details of the fishing attacks so the rest of us could learn.

ben5092• 6 years ago

We have zero tolerance for misuse of credentials or tools, actively monitor for misuse, regularly audit permissions, and take immediate action if anyone accesses account information without a valid business reason.

Okay, so who has been fired?

That's what "zero tolerance" means: no excuses, not even "someone tricked me." And no punishment but the maximum.

Anything less would involve some degree of tolerance, and when you say "zero" that means no tolerance whatsoever.

It's obviously stupid to manage any organization that way, of course. It's a fatuous, dishonest phrase.

So stop talking about "zero tolerance" since all it means is "we make hyperbolic claims that we have no intention of living up to."

Shivetya• 6 years ago

Well besides claims of restricting access to the tools they obviously need to further have restricted access to certain user accounts. We have support persons whose account support is locked from superior accounts or special users and this seems like something Twitter should be doing.

Perhaps have another layer where each use of specific functions require unlocking through an incident management tool?

ytdytvhxgydvhh• 6 years ago

It’s like when motorcyclists say “safety first” about wearing a helmet and other protective gear. If they really put safety first, they’d choose a safer form of transportation. They mean “given that I’m going to engage in this risky activity, I’m going to try to make this activity as safe as possible”.

In this case “zero tolerance” is short for something like, “except for understandable slip-ups that aren’t fully your fault, we’re not going to tolerate any slip-ups”.

NotSammyHagar• 6 years ago

I think you can honestly say you follow safety first in terms of what is available to safely ride your bike.

Just like when I used to rock climb, I felt like we were basically following safe practices - but there was no one to adjudicate them, probably far less testing of various practices with stats than bikes. Also, where we climbed there wasn't expected rockfall. I had barely heard of that being an issue, and we never wore helmets. Later on I realized that was something I might have missed out on. And then the next step was "what other safety practices was I unaware of" ;-)

And of course I get that rock climbing is much more dangerous than hiking.

fulafel• 6 years ago

It's pretty meaningless on motorcycle too. Even aviation with their culture and checklists isn't that.

iandev• 6 years ago

It’s possible to have zero tolerance and not fire anyone here. My understanding is that no employee misused their credentials or tools. The attackers misused them. I suppose you could argue that accidentally exposing credentials is misusing them, but I don’t think that’s what Twitter means there.

kerng2• 6 years ago

Maybe attackers/phishers are the same people who's accounts were used? Easy bypass.

qzx_pierri• 6 years ago

I don't really trust Twitter at all. When you consider how they shape public opinion on so many things, it gets scary.

They have been caught banning people based on their political stances, and refuse to remove the algorithmic timeline sorting method which is designed to strip adolescents (and easily persuaded adults) of their critical thinking skills.

acoard• 6 years ago

That's what "zero tolerance" means: no excuses, not even "someone tricked me." That doesn't necessarily follow, as it depends on exactly what they have zero tolerance for. They say they have zero tolerance for "misuse of credentials." Misuse conceivably may not include insecure storage of credentials or accidentally exposing them, but only actively using them, eg logging in and using them for an inappropriate purpose.

I'm not trying to split hairs or be a Twitter apologist here but there is a meaningful distinction here. Intentional misuse of credentials is ultimately subordination (which is immediately fireable in most situations), whereas accidental exposure is a mistake. Twitter is effectively reinforcing that employees are forbidden to puruse private data. They are not making the point that they will fire anyone accidentally involved in a security breach.

ben5092• 6 years ago

That doesn't necessarily follow, as it depends on exactly what they have zero tolerance for.

It does depend on that, you're right. "Zero tolerance" sounds so clear, it even has a number in there! But, nevertheless, one can rationalize just about any outcome by invoking it.

Specifically, any administrator who hasn't worked out a detailed meaning will have to crystalize their understanding when it comes time to apply the idea. This process of rationalizing will be different depending on the situation and their biases.

The supposedly clear policy becomes capricious or arbitrary. And if it's not arbitrary because they have some actual doctrine that can be consistently applied, then it would make more sense to use that doctrine.

I'm not trying to split hairs or be a Twitter apologist here...

Splitting hairs is the raison d'etre of this site.

I'm not annoyed at Twitter specifically as they're hardly the inventors of the phrase. My issue is with concept itself, and the broader mindset that you see in legal concepts like strict liability.

Intentional misuse of credentials is ultimately [in]subordination...

Well, intentional is your head-canon since they didn't use that word. But intent is useful to the discussion; let me explain why I don't think zero tolerance allows for intent and other mitigating factors.

The point of tolerance is that some harm is done, and the injured party is going to limit their response to it.

Law typically breaks it out as the action that caused the harm, the intent to cause that harm, and the certainty of your knowledge of the facts.

As soon as you bring intent into the equation, you're willing to tolerate a great deal of harm. Someone can get hurt in a car crash, and if it's clearly an accident, the injured party is generally not going to hold a grudge.

If there's sufficient uncertainty, we aren't even sure we can direct our response to the harm at the correct party. Then we're stuck tolerating it, or taking it out on some scapegoat. And I'd even argue that the fact that we inevitably have to tolerate some harm makes the concept of zero tolerance fundamentally contradictory.

Tolerance is what civilized people do in response to real life situations, and when they don't you get feuding and war. This isn't a new problem, the point of "an eye for an eye" in Mosaic law was to limit vengeance and vigilanteism with a doctrine of proportionality. Not surprisingly, people still didn't get it, which was why Christ revised it to "turn the other cheek."

vangelis• 6 years ago

Intent matters here. They ultimately fell prey to social engineering, they didn't give out their credentials or do this themselves.

kerng2• 6 years ago

Indeed... one can imagine scenarios were maybe attackers/phishers are the same people who's accounts are being used? This seems like the easiest way to get away with misusing your access. Just send yourself a phishing mail....

kube-system• 6 years ago

'Zero tolerance' doesn't in itself imply anything qualitative about the the actions that are taken as a result of that intolerance.

'Zero tolerance' only means that some action would happen without any subjective application of the rule.

dirtylowprofile• 6 years ago

They were hacked by kids on Discord.

vinniejames• 6 years ago

Whats the final determination around the rumors employees were payed for access to their creds?

gregors• 6 years ago

Social engineering will never be stopped, people want to be helpful. And generally speaking the cost for stopping it at non-secure businesses is going to be too high until a security incident happens.

Phishing email attacks? Why do employees have business emails at all?

Phishing phone attacks? Why would employees have phones with external access?

Front of the house (dealing with users) should probably be disconnected from back the of the house (admin access).

Before you know it you're in DoD or Bank territory. No Wifi allowed etc, where's your badge buddy?!

Things get complex quickly with security.

tialaramex7• 6 years ago

Social engineering will never be stopped, people want to be helpful.

They really do. And so you should design security systems with the assumption that your employees will actively undermine security "to be helpful" to adversaries.

And generally speaking the cost for stopping it at non-secure businesses is going to be too high until a security incident happens.

The cost for Yubico's "Security Key" is $20 and there is a volume discount. You should buy each employee a key, and if there's no secure means by which they can be re-authorised when they inevitably lose it, a second one to keep safely for that case.

The attackers correctly anticipated that while "Can you get me Jenny in user assistance's phone number?" is just being helpful, "Can you disable Elon Musk's 2FA and give me control over his account?" is a bit... obvious. So they got themselves credentials to do that stuff. But there is no need for Twitter employees to be able to give away those credentials.

An update on our security incident

Related Stories