Hacker confirms access through infostealer infection [withdrawn]

https://www.hudsonrock.com/blog/snowflake-massive-breach-access-through-infostealer-infection

by zbangrec • 2 years ago

680 210 2 years ago

hudsonrock.com

beretguy• 2 years ago

The data from these companies was put up for sale on the Russian-speaking cybercrime forum

Just russia being russia, as usual.

KoftaBob• 2 years ago

Your daily evidence that modern Russia is essentially just an organized crime ring with oil reserves and nukes.

pizzafeelsright• 2 years ago

That's just "government"

cabirum2• 2 years ago

Yes, any country that is not your ally or vassal is an "organized crime ring". It's safer to be with oil and nukes, than without them you know.

shrubble2• 2 years ago

I don't think naming things that apply to USA also, is quite the win you think it is.

mc32• 2 years ago

People in ex-USSR also speak/write Russian, so while I have no doubt Russians frequent the forum, lots of others from their sphere of influence also visit that forum. Don't assume everyone on HN is a 5-eyes resident.

cabirum2• 2 years ago

The screenshot shows the post in English. The website domain is Indian. The seller contact xmpp.cn in China. But no, we'll keep blaming Russia for everything.

WhackyIdeas3• 2 years ago

I agree. Every single security thing that happens is put on Russia.

Typical propaganda.

If we were pre-Ukraine, it’d be China getting all the blame (like it used to be).

I’m not pro-Russia, I’m just anti-bullshit.

NameError• 2 years ago

This article is claiming that the Ticketmaster breach from a few days ago was actually a much broader hack affecting 400+ companies, all through a Snowflake employee's stolen credentials. This seems like a pretty big story that's only being reported on hudsonrock.com now.

I haven't heard of Hudson Rock before, does anyone know if they are a reputable source?

proactivesvcs• 2 years ago

BBC News report of a substantial hack of Santander bank; linked to Snowflake. https://www.bbc.co.uk/news/articles/c6ppv06e3n8o

dralley2• 2 years ago

BBC just linked back to Hudson Rock's allegation FWIW, they don't have any independent confirmation

WhackyIdeas3• 2 years ago

Great, so these companies do not give a flying fuck about their customer data in making sure the data stored at cloud storage companies are end to end encrypted.

To think these random cloud storage companies can access your bank information is utterly shocking.

ransom1538• 2 years ago

Snowflake employees need time to sell off all their shares. This news will hurt the SNOW stock price big.

richbell• 2 years ago

I haven't heard of Hudson Rock before, does anyone know if they are a reputable source?

I first learned of Hudson Rock after their "CEO" started spamming every security-related subreddit with low-effort blogspam over a period of months alleging numerous breaches. They've had several accounts banned, both by Reddit moderators and administrators.

Personally, I would no consider them a reputable or reliable source.

abadpoli2• 2 years ago

Why in the world would obtaining a Snowflake employee’s credentials allow you to then obtain Snowflake’s customers’ data? Doesn’t this imply that people working at Snowflake can see all of the data that I put in it?

Admittedly I don’t have much experience with Snowflake, but as a baseline I expect better from a “cloud storage giant”.

i_k_k• 2 years ago

There’s something missing here.

From what the Hudson Rock article shows, they were able to use an SE’s creds to access their demo account. This is not a customer account and shouldn’t (but of course could) contain sensitive info. It’s not clear to me how this snowballed into a larger breach.

Perhaps customers had granted this SE access to their accounts and the data within. Or perhaps there’s a deeper hack. But this isn’t clear to me from what I’ve read.

p0seidon10• 2 years ago

There may have been administrative access that was not properly secured.

gregates3• 2 years ago

I was just going to post the same thing. The files that they show in the screenshots are things like PROGRESSIVE_BID_CHANGE_202405271129.csv. Looks suspiciously like the Snowflake Sales Engineer's data for their job role closing a deal with Progressive, not Progressive's own data. And there's no reason to think that a SE would have broad access to customer data. There may be some overlap, but I doubt it contains sensitive customer data owned by Progressive.

coredog64• 2 years ago

Something similar happened at a previous employer. Contractor was hired to do a big data PoC, and they managed to cajole access to a prod data dump for a more impactful demo.

They then managed to load all this PII data into an ElasticSearch instance that was open to the internet and was discovered by threat actors.

I wouldn’t be surprised to find that something similar happened here, where an unscrubbed prod dataset was shared for a better demo.

FromOmelas3• 2 years ago

No, that sounds about right. This is a new, agile, cloud-first company that grew very quickly and has faced significant turnover. You don't get such growth by doing everything right.

Looking at linked-in, the unlucky employee could be someone in a sales role, with only 7 months of tenure. Every company has a few sysadmins with a scary amount of reach, but that's not what happened here.

Edit: A ServiceNow access request flow with poor internal controls would explain it.

Keyframe2• 2 years ago

(new) sales person with an uber account that has access to carte blanche customer data. This is not only a disaster, if true, but also violates probably every certification under the sun, if they had any at all. Reminder Snowflake is a couple of sales persons from Oracle and a techie.

DowagerDave2• 2 years ago

> This is a new, agile, cloud-first company that grew very quickly and has faced significant turnover.

This is not really true of Snowflake, which is not some 2-person YOLO startup, and it's also pretty irrelevant as the weakest link is often a single employee regardless of the size or industry of the company. In my experience the support and security is way better than average - example: as a client of both Snowflake and Sisense, Snowflake reached out to me about the Sisense breach before Sisense did.

c0njecture5• 2 years ago

It's not a new tiny company. It's about 12 years old with 7000 employees. They know they are dead if they are not hot on security, so at the moment I would take this story with a big pinch of salt. Quite possible certain customer configurations have been attacked, but that is a different thing.

WalterSobchak• 2 years ago

IOCs: https://community.snowflake.com/s/article/Communication-ID-0...

skilled3• 2 years ago

At least based on the wording of the perpetrator, Snowflake really did have the system designed in a way where a single administrator account gives you carte blanche to everything.

On may 31st, Snowflake released a statement in which they claim that they are investigating an industry-wide identity-based attacks that have impacted “some” of their customers.

https://community.snowflake.com/s/question/0D5VI00000Emyl00A...

This gives credibility to both Ticketmaster and Santander stories.

Wow! Could be one of the biggest dumps of all time if the threat actors did everything correctly?

tomashertus• 2 years ago

If the threat actor has played it right, there is a high possibility that this will be the largest data breach in history.

p0seidon10• 2 years ago

So the account was without 2FA protection?

c0njecture5• 2 years ago

There's no evidence of that at all. The screenshot shows a few Snowflake professional services demo accounts only. These are accounts used by the sales engineer to demo features to customers.

It's possible the attacker was able to deduce some information about certain customers, but they would not then be able to connect to those accounts to extract data as those accounts should not be accessible from the public Internet at all, and should require corporate authentication.

nattaylor• 2 years ago

In my experience with Snowflake support about a year ago, an administrator of the customer's account had to explicitly grant access to Snowflake in order for the Snowflake team to see or do anything -- and if I recall correctly the access had an expiry.

skywhopper• 2 years ago

That’s if they were following procedure. But on these internal systems, there are often hacks around the procedure that folks with the right mindset can easily find.

holmesworcester• 2 years ago

...All with just one successful malware-as-a-service attack against the right employee. (Lumma was the malware-as-a-service used.)

Interestingly, there's another adjacent story on the frontpage about Pegasus being used against NGOs in Eastern Europe. The principle of least authority is important, but also device security is important!

https://news.ycombinator.com/item?id=40535912

gregates3• 2 years ago

That's what the article implies, but I think it's overblown. They provide enough information (unfortunately) to identify the employee whose credentials were stolen, and she's a Sales Engineer. The data seems to have come from her own Snowflake account, which was used to build demos for customers or prospective customers. It's quite possible that those customers granted her access to some of their actual data, which was used in those demos, but it's a far cry from unfettered access to the customer's Snowflake database itself. It's also quite possible that the hacker exfiltrated fake-but-realistic data used for demo purposes and doesn't know the difference.

hn_throwaway_993• 2 years ago

They provide enough information (unfortunately) to identify the employee whose credentials were stolen, and she's a Sales Engineer.

I'm not previously familiar with Hudson Rock, nor how "standard" disclosures around this work, but identifying the breached employee felt like an extremely shitty move to me. If a single infected laptop of a sales engineer (i.e. not even an admin with extensive access rights) resulted in a breach this large, the root cause problem is not the sales engineer - and I'd note that Hudson Rock says as much in their article.

hangtime793• 2 years ago

This is the problem of Hudson Rock making conjecture or trying to be authoritative when tbey don't know the process.

One SE is working on many accounts. Snowflake SEs don't build within the client environment typically. They set up a demo account like you or I do with the $400 in credits. SEs are constantly starting these. Why? They expire after the fact. The SE builds in the created demo account and shows the client. After 30 days Snowflake locks the account (no credit card) and subsequently drops the demo instance and data.

For an SE to do the work the customer can do one or more of the following: The customer's SF instance shares data to that demo instance created by the SE AND/OR the customer has given access to that Snowflake SE through SSO.

Either way, this is more of orgs not being restrictive in their security posture. There is nothing novel about this exploit other than they found an SE who was working very hard and clients who had not properly scoped the security permissions of an employee/contractoe/guest.

elvisizer3• 2 years ago

yeah, the perpetrator is wrong: nothing in production was accessed, no customer data either. totally being mischaracterized in the hudson post.

bbayles2• 2 years ago

The screenshots of the chat logs are really something. This firm claims to be in communication with the actual criminal, and the actual criminal says that using their firm would have helped prevent the breach.

I have updated my sense of the firm's trustworthiness accordingly.

chairmanwow1• 2 years ago

in that you trust them less?

itscrush• 2 years ago

Absolutely the case for me. I don't give Snowflake much here, but Hudson Rock sells this exact type of "protection" and so far including BBC, no other independent verification?

This from the GP's link does it: “should have bought protection from Hudson Rock could have saved them this one”

deinonychus• 2 years ago

That particular exchange is bizarre and cartoonish. I don’t know what to make of it.

“should have bought protection from Hudson Rock could have saved them this one”

“yes i agree it wouldve helped for sure”

JohnMakin• 2 years ago

seems like a shameless marketing plug to me

gregates3• 2 years ago

This is just pure speculation, but it kind of looks like the hacker was being ignored by Snowflake, so they somehow got in touch with Hudson Rock and offered them this promotional opportunity (to break the news, more than the throwaway line in the article) with the goal of retaliating against Snowflake for failing to pay the ransom. And Hudson Rock agreed to play along and hype up the story, presenting it as a bigger breach than it really was. One wonders whether Hudson Rock was the first they went to, or just the first to take them up on the offer.

bbayles2• 2 years ago

It's also possible that the firm is being trolled by the "threat actor."

adtac• 2 years ago

It's a common euphemism in ransomware and protection rackets in general. One of my favourites is the message the akira group leaves in infected machines that goes something like:

    Congratulations, you have passed a surprise information
    security audit and become a victim of ransomware.

    [...]

    We offer:

    1) full decryption assistance
    2) evidence of data removal
    3) security report on vulnerabilities we found
    4) guarantees not to publish or sell your data
    5) guarantees not to attack you in the future

They're just a security consulting company that you didn't know you had on payroll!

Btw I looked at what they provide as evidence of data removal (2) and it's literally just the stdout of `rm -vrf data` lol. I mean, I get that it's impossible to provide evidence of absence, plus the victims have no leverage anyway, but I dig the theatrics.

bawolff2• 2 years ago

Sounds like implied extortion to me.

elvisizer3• 2 years ago

they're also totally wrong about what they had access to . . .

i_play_stax• 2 years ago

The screenshots make this feel entirely fabricated or entirely marketing motivated.

If it's somehow real, omitting the hudsonrock message would be good sense.

I have mentally black listed this company.

RcouF1uZ4gsC• 2 years ago

From their marketing page:

Snowflake’s single platform eliminates data silos

I guess so, especially now.

yodon• 2 years ago

From the official Snowflake response[0]:

We believe this is the result of ongoing industry-wide, identity-based attacks with the intent to obtain customer data. Research indicates that these types of attacks are performed with our customers’ user credentials that were exposed through unrelated cyber threat activity. To date, we do not believe this activity is caused by any vulnerability, misconfiguration, or malicious activity within the Snowflake product.

[0]https://community.snowflake.com/s/question/0D5VI00000Emyl00A...

BillFranklin2• 2 years ago

It sounds like they found a way to bypass MFA on snowflake (because snowflake didn’t expire session cookies), and stole an employees credential, obtained via a “Lumma-type Infostealer” which I guess is just a key logger in browser extensions and fake versions of software…

DowagerDave2• 2 years ago

something doesn't add up, because I don't see how this extrapolates from stealing privileged Snowflake employee credentials. How does that become a keylogger on a client's computer?

BillFranklin2• 2 years ago

Yeah it is a bit muddled honestly. I had to read it a couple times and I still don’t completely get what happened:

1. Employee installs a key logger

2. Snowflake does not expire session cookies

3. Malware steals their session cookie and password, so can bypass employee MFA/okta

4. ???

5. Somehow this one employee has admin access to 4000 snowflake instances

siliconc0w• 2 years ago

Protecting customer data from compromised insiders can be pretty hard. They often need the access to do their jobs. Still, in this case it's was far too easy - just one session cookie and a password shouldn't itself by sufficient to compromise all your customers.

BlackjackCF2• 2 years ago

Was their intent to dox the employee while discussing this beach? They show the employee’s username, which is easily Googleable.

boringg• 2 years ago

Yeah that seems super sus to me as well. Super unprofessional.

owlninja• 2 years ago

As is the plug of 'should have bought protection from hudson rock'

briffle• 2 years ago

But hey, they blot out the name of the alleged attacker in chat, because of privacy...

dgellow• 2 years ago

And IP

lazystar• 2 years ago

the username is probably the only proof they have that they were talking to an actual member of the hacker group.

Bluestein• 2 years ago

To put it bluntly, a single credential resulted in the exfiltration of potentially hundreds of companies that stored their data using Snowflake, with the threat actor himself suggesting 400 companies are impacted

You only have to "fail" once, as they say.-

DebtDeflation• 2 years ago

Imagine having your enterprise data warehouse stolen.

redwood2• 2 years ago

It's unclear if it's customer metadata or real customer data data?

teej5• 2 years ago

Santander claims:

"Following an investigation, we have now confirmed that certain information relating to customers of Santander Chile, Spain and Uruguay, as well as all current and some former Santander employees of the group had been accessed. Customer data in all other Santander markets and businesses are not affected."

https://www.santander.com/en/stories/statement

redwood2• 2 years ago

But this does not mention Snowflake so it's on 100% clear it's the same root cause. And if it is the root cause being discussed elsewhere here namely that a customer engineer with partial data for demos... could that really have had access to this level of data?

rdtsc3• 2 years ago

Snowflake says it's not their fault, it's customer's fault apparently:

https://community.snowflake.com/s/question/0D5VI00000Emyl00A...

If https://www.hudsonrock.com/blog/snowflake-massive-breach-acc... has not completely fabricated the story, Snowflake are tiny bit less than truthful.

Research indicates that these types of attacks are performed with our customers’ user credentials that were exposed through unrelated cyber threat activity. To date, we do not believe this activity is caused by any vulnerability, misconfiguration, or malicious activity within the Snowflake product. Throughout the course of our ongoing investigation, we have promptly informed the limited number of customers who we believe may have been impacted.

They are way too wishy-washy:

* "Research indicates": Their own research or just general security research out there.

* "...do not believe this activity is caused by any vulnerability, misconfiguration, or malicious activity within the Snowflake product": I think they know exactly how it happened. They enumerate all possible scenarios and methods it didn't happen: misconfiguration, vulnerability, malicious activity within the product. But skillfully skip the explanation for how it actually happened.

They were contacted by the bad actor if hudsonrock is to be believed, so they probably had a good idea how it happened.

ActionHank• 2 years ago

Or they had no idea and assumed that customers had a breach, so blamed them without doing a proper in-depth analysis.

The hacker alleges that they weren't even expiring refresh tokens, that is pretty huge if true, it's just a massive, glaring issue.

rdtsc3• 2 years ago

I think it's unlikely:

* If Hudson Rock is to be believed, with the chat screenshot, Snowflake was notified and asked for a ransom. It would seem odd that all their 400 customers were all hacked randomly at the same time, by only one hacker group just based on those customers' own bad credentials

* They enumerate all the possible ways they were not hacked, and seems to skips the exact one way they were hacked. It's like saying: "It wasn't A, C, D, E, or F" as the causes. Hmm, they skipped B it seems, I wonder why...

mahogany• 2 years ago

Don’t they hint at how it happened in what you quoted?

performed with our customers’ user credentials that were exposed through unrelated cyber threat activity

Unless they are just making this up, they seem to think that credentials were obtained elsewhere. They also say at the end that they told customers to review their account settings. So they are directing the blame away from themselves.

You’re right that this seems to conflict with Hudson Rock. Unfortunately our sources are the company that allegedly was hacked and a company that shamelessly doxed an employee in the course of using this event to promote their product. I think we’ll need to wait for more details.

rdtsc3• 2 years ago

You’re right that this seems to conflict with Hudson Rock. Unfortunately our sources are the company that allegedly was hacked and a company that shamelessly doxed an employee in the course of using this event to promote their product. I think we’ll need to wait for more details.

Fair point. Doxing the employee is shitty, no doubt. Makes Hudson Rock look bad as well. But at the same time, I was giving them some credibility as it would seems if they completely fabricated the screenshots, they might as well file for bankruptcy, given the market they seem to operate in.

jupp0r2• 2 years ago

It's surprising that SnowFlake didn't pay the $20M ransom. Seems like a no-brainer compared to the reputational damage this would cause.

pluto_modadic2• 2 years ago

I honestly don't understand why someone would pay a ransom. They just hacked you. Do you think they'd actually follow through with keeping quiet?

teej5• 2 years ago

The query that Snowflake provides to identify privileged users is wrong.

I wrote a better version here: https://gist.github.com/titan-teej/924dcb42604a98b90d6419262...

TL;DR they don't check for users who can transitively gain admin access.

aurum• 2 years ago

The article doesn't seem very consistent with the headline of "hundreds of breached customers"

1. The password for lift/okta is only allowing access to a servicenow portal and not customer accounts, so the refresh token issue seems restricted to the servicenow portal and unrelated to any actual customer data being exposed from customer Snowflake accounts

2. The screenshot with 10 corporate accounts compromised shows 4 different Snowflake account credentials (one of which appears to be a personal demo account) so that might explain up to 3 customers being compromised but there's no details showing other customers being compromised.

Assuming all of the SE's credentials were compromised for all of the customers they were working with, we can probably say the total customers compromised would be in the low double digits (each customer account would have had to provision access to the SE individually)

Big leap to say that literally the entirety of Snowflake's customer base is compromised from a "refresh token issue" (in the internal Okta portal) that isn't even linked to any customer Snowflake account

matsur• 2 years ago

Very possible there were creds etc accessible in Servicenow that could have been used to move laterally from there. Conjecture, obviously.

shrubble2• 2 years ago

Without knowing exactly how the compromised account is set up, and what access is granted, it may be difficult to say. At "security focused large telecom" I am aware of, you would be surprised what level of tech has access to what (though of course all access is logged).

c0njecture5• 2 years ago

Snowflake internal staff do not have access to read customer data, unless a customer grants it. Customers can use their own KMS to generate table keys.

Snowflake has a lot of security features. But still, customers may well misconfigure their own Snowflake accounts and therefore be vulnerable.

A well configured Snowflake account:

- does not allow any access from the public Internet. Network policies set by the customer should restrict access to corporate networks only. - does not allow authentication unless with MFA or via corporate IDP / SAML - has dynamic masking / tokenisation

Snowflake seems to have most of the Fortune 500 as customers. If Snowflake itself was somehow penetrated and all controls circumvented, it would certainly be huge and you'd be reading about a lot more than Santander and Ticketmaster.

At this point it seems more like the "AWS Hack" that affected CapOne back in the day (that was CapOne's fault, not AWS!).

foobiekr2• 2 years ago

"Snowflake internal staff do not have access to read customer data"

Do engineers in Snowflake have access to production systems?

c0njecture5• 2 years ago

They don't have customer key access and can't assume customer identity but ultimately yes, via a multi-eye approval process there is access to the prod infra - but this is extremely tightly secured, and not something a phishing attack on a single sales engineer could ever achieve.

Many enterprise customers additionally use standard third party crypto libraries to tokenise and/or encrypt sensitive fields before storage in any warehouse/database such as Snowflake or Redshift.

This is a similar principle to using client-side encryption for S3. The infra provider (AWS in that case) can never read the data.

teej5• 2 years ago

Snowflake internal staff do not have access to read customer data

By default, no. But it is standard operating procedure for sales engineers to request and be given access to customer data so they can build demos.

c0njecture5• 2 years ago

It is not standard operating procedure, and demos wouldn't be done with production data. In fact, most enterprises would have contracts in place with Snowflake that explictly state that Snowflake staff can not be granted access to their Snowflake accounts (this is actually the default in the Snowflake enterprise professional services contract now).

You can understand it: Snowflake lawyers are naturally reluctant to have their staff be granted access to any customer's accounts.

It is however quite common to have Snowflake PS guys have limited access to customer dev environments.

However, such access, even in dev, should always be network restricted.

philipwhiuk• 2 years ago

How did Snowflake not have 2FA?

hangtime793• 2 years ago

I don't work for Snowflake but I spend a lot of time working with them and their SE organisation.

When working and building demos with clients, SEs create demonstration environments on the same $400 Snowflake demo accounts anyone can. To build demos the client would grant access to that SE. The SE would take some of the data to the demo environment and then work on it. This is further confirmed by the name of the environment Hudson Rock just published.

As far as I can tell, this is a process issue of clients not expiring an ID of someone who they were sharing data with and a threat actor swiping credentials. There is nothing novel about this as there is no exploit.

Also congrats Hudson Rock you just outed a person who was taken due to having malware on their computer. This is no different then if you gave a contractor credentials and they had those swiped. Dicks.

jupp0r2• 2 years ago

Agreed, mentioning the login name of the compromised account seems really unprofessional and unnecessary.

BlackjackCF2• 2 years ago

This was a really effective anti-ad for Hudson Rock.

waihtis2• 2 years ago

The whole blog post reeks of extreme self-congratulation and youre right, a total scum move to expose the victim. Altogether very weak performance from Hudson Rock.

p0seidon10• 2 years ago

It seems somehow like an Indiehacker page, probably taken too lightheartedly. Additionally, the related pages have no real contact details.

ExoticPearTree2• 2 years ago

Snowflake’s customers are the victim, not Snowflake.

abadpoli2• 2 years ago

Just because there isn’t a “novel exploit” doesn’t mean this isn’t a big deal.

Snowflake is susceptible to their SE’s having credentials stolen. These credentials can bypass MFA. And per the article, they have no expiry. That’s strikes one, two, and three.

Snowflake’s security practices lead to a situation where their customers are either required, or at minimum encouraged, to share access to broad datasets with Snowflake employees. That’s strike four.

Yes, there is also issue here that the customers are responsible themselves for not granting too broad access, and that’s on them. But it’s also on Snowflake for not having a better system that doesn’t require this access, or at minimum not having better oversight and control over this transitive access.

Once these accounts are granted access to a customer’s data, they aren’t “demo accounts” anymore. They’re real accounts, with real, very valuable data, and they should be treated as such.

Edit to add: it is worth noting that Snowflake claims the demo account did not have access to customer data and wasn’t the source of the leak, which is in contradiction with what the attackers claim.

bawolff2• 2 years ago

Indeed, the less novel the exploit the more embarassing it is.

Data stolen because of some crazy multi-exploit zero day chain. Well that is understandable, i don't blame the company.

Data stolen because no 2FA support? In 2024 that is just embarassing.

hangtime793• 2 years ago

Having spent a lot of time in the Snowflake ecosystem, a few things to understand.

Snowflake is both a platform and a database. Customers can implement any level of security within the system. For example, when Snowflake introduced OAuth functionality, they put a lot of pressure for us to implement it in our tool. We're small enough and they are a significant partner so we priortised and got it into our platform so that a customer CAN implement it if they want too.

The keyword here is CAN. As a platform with database functionality, Snowflake allows customers to implement any level of security for access. So each instance of Snowflake https://XX123456.us-east-1.snowflakecomputing.com/ security is independent. So let's answer some questions

* "Was this a production breach", No, this was not a production breach. This is not when hackers went into the Azure and were able to elevate credentials and see all accounts.

* "But doesn't Snowflake use MFA", yes and on INTERNAL systems and the ability to connect to the PROD environment of SF it would absolutely have that. However, every customer has the ability to configure a user's credentials for access in their instance https://XX123456.us-east-1.snowflakecomputing.com/. Likely, the SE was granted access to the client's instance through the use of a user name and password.

* "But why not use MFA then Snowflake", this comes back to Okta. You have to create a trust relationship between providers. That's easy when you are only talking about your own employees. They all sit in your same directory. However, to set up a separate trust relationship for another domain, well now we have to get the security team of the client involved. All this so that someone can take a little bit of data and move it into another environment in order to do a demonstration. Not likely.

* "Well that just sounds stupid, my application has SSO enabled." True, but as also a database - Snowflake has to account for entire ecosystem that don't even support SSO-OAuth. For example, two major products that do not support SSO connections to Snowflake: Azure Machine Learning and Zapier. Both only support user name / password connections to Snowflake. So if you were an SE trying to show the value of Snowflake and its integration with AzureML; the ONLY way you could do that would be through user name / password functionality.

The issue in all of this is we are so used to thinking of SaaS as just an application. Databases have to support all sorts of application weirdness. Here is my gut thought on how this went down.

* The SE was working with multiple clients. These clients in an effort to save time and not wanting to have their security team involved and try and create a trust relationship with Snowflake went ahead and created a user name and password for the SE into the client environment vs setting up SSO. Again, this is ONLY controlled by the client.

* The SE at some point had active access to multiple instances of Snowflake to their various clients; with various levels of access depending upon the skill and fastidiousness of the client when they created the account for the Snowflake SE into their account. Again, how the SE is connecting to Snowflake is the same way any other user and application are doing so. I know this for a fact as SEs have to create their own throwaway demo instances https://XX123456.us-east-1.snowflakecomputing.com/. SEs don't have access into the backend of Snowflake.

* At some point this SE had malware installed on to their computer. At this point, this person was cooked and any customer who had given them access to the environment.

As for Hudson Rocks, they can go f themselves for doxing the SE. It was quite trivial to find who this person was and based on the Snowflake announcement they no longer have a job. That person was having an awful week already, now there name is out there. So again, F Hudson for doxing. It would have been easy to blind them out of the exchange (they did the hacker), but did not do so for the victim.

alias_neo• 2 years ago

I just had a quick read through a couple more posts on HR, and a lot of them end with something along the lines of "heh, should have bought protection from us", reeks like a racket.

clwg2• 2 years ago

This is the description of one of Hudson Rock's main products, "Bayonet".

"Imagine getting access to a lead-generation platform featuring hundreds of thousands of compromised companies around the world with active vulnerabilities that you can convert into customers."

I've seen and dealt with a couple of these types of companies. It's a pretty sleazy tactic, and it's low skill/effort from a technical point of view as well.

clwg2• 2 years ago

Doing some more digging, this is where the data is sourced

"Hudson Rock acquires and purchases compromised data directly from top-tier threat actors operating in closed circle hacking groups. What sets our data apart is its quality in providing high accessibility to hacker groups looking for potential targets, and the speed in which we make it available to clients compared to other threat intelligence companies. Our operational knowhow, and our boots-on-the-ground approach to cybercrime originates from the IDF's 8200 Cybercrime division, and its efforts to thwart nation-state adversaries and professional threat actors."

https://cavalier.hudsonrock.com/docs

This would imply that they are financially engaging with and supporting cyber criminals.

vasco• 2 years ago

That screenshot where the attacker agrees that buying their services would help makes me have a very strong suspicion this attack is either faked, sponsored by or in some way Hudson Rock is in cahoots with the attackers. That screenshot is a pure 100% advertisement manufactured by Hudson Rock one way or another.

As they usually say.. I'd check whoever is running Hudson Rock's hard drive.

fhoffa• 2 years ago

Hi, Felipe at Snowflake here.

Here is the latest from Snowflake on this issue: https://community.snowflake.com/s/question/0D5VI00000Emyl00A...

We'll keep updating that URL with any further news.

p0seidon10• 2 years ago

Will there be any direct comment regarding the article here?

catchnear43212• 2 years ago

the ad for protection services?

demosthanos3• 2 years ago

The linked post now contains unambiguous denials of the majority of claims in TFA, and TFA has been taken down.

Galanwe3• 2 years ago

The salesforce.com servers are temporarily unable to respond to your request. We apologize for the inconvenience. Thank you for your patience, and please try again in a few moments.

Kailhus• 2 years ago

Sill down. “…Visit http://trust.salesforce.com for current system status and availability.”

BlackNitrogen• 2 years ago

The OP Hudson Rock writes something that I understand is saying: This was more than a breach of one customer's credentials, they got some employee creds and they weren't protected by 2 factor so they got into other customer accounts using that engineer's creds.

The snowflake writeup reads to me as if a customer's account creds got compromised - and it implied to me that was the end of it, no central or other account access on thoes creds. Nothing about this use of some employee account info that didn't have 2 factor auth on it.

1. I'm sure snowflake wants all access creds of any kind for their internal employees to use 2fa.

2. It used to be at least as a customer you could create a name/password without 2fa to log in to your own info there if you wanted to, like say as a customer you create a db or table and want to access it.

desert_rue• 2 years ago

For 1. Those accounts would be set up by customers, so if they didn’t require 2FA, it didn’t happen.

hn_throwaway_993• 2 years ago

So, just to be clear because I found your link filled with a bit too much corporate speak:

1. The linked Hudson Rock post is explicitly claiming that the breaches at Ticketmaster and Santander Bank were caused by a Snowflake employee whose credentials were compromised.

2. This bullet point, "We did find evidence that similar to impacted customer accounts, the threat actor obtained personal credentials to and accessed a demo account owned by a former Snowflake employee. It did not contain sensitive data." (emphasis mine) says pretty clearly to me, then, that Snowflake believes the Hudson Rock account to be false.

So is that a correct understanding then?

BSDobelix• 2 years ago

Wow that's what i call a anti-ad for Hudson-Rock, should have bought services from a public relation firm...yes i agree, it would have helped for sure.

Are theirs HQ located in Hollywood?

zamalek• 2 years ago

Okta

Just how many pwns involve this keyword? Between Okta itself being pwned and it allowing stupid shit like ignoring expiry, I am strongly inclined to believe that rolling your own login system might be the strongest security posture these days. Last I heard, Okta doesn't use any form of FIDO internally: how completely and utterly worthless.

tmpz22• 2 years ago

Okta is a single point of failure but its still probably way better then what the average developer will throw together under deadline.

cedws• 2 years ago

should have bought protection from Hudson Rock

Damn that is so not classy. Who approved that?

Lammy2• 2 years ago

were able to sign into a Snowflake employee’s ServiceNow account using stolen credentials, thus bypassing OKTA

It's probably not technically relevant to the breach, but it's at least interesting that the CEO of Snowflake is the former CEO of ServiceNow: https://en.wikipedia.org/wiki/Frank_Slootman

nphard85• 2 years ago

The current CEO is Sridhar Ramaswamy

Lammy2• 2 years ago

Yes (since February), although according to the article the infection happened in October 2023 during Slootman's tenure.

_tk_• 2 years ago

I'm more than disappointed in Hudson Rock. They lead with confirmation of a breach, but detail no more than allegations and swagger. The decision to not redact the login is not just a moral failure, but it shows that they do not understand the subject matter that they purport to be experts in. Shameful.

31337Logic• 2 years ago

This. Thank you.

weinzierl• 2 years ago

"To understand how the hack was carried out, the threat actor explains that they were able to sign into a Snowflake employee’s ServiceNow account using stolen credentials, thus bypassing OKTA which is located on lift.snowflake.com."

I'm unfamiliar with ServiceNow and their website seems to be just marketing bable.

Can someone explain the role of ServiceNow in this scenario and why it important?

p_l4• 2 years ago

ServiceNow is a system used for many ticketing, CMDB, etc. tasks, including automation.

That said, the quoted sentence boils down "the cookie we stole authorized us to service now instance bypassing SSO prompt".

skybrian• 2 years ago

I don't know anything about Snowflake or enterprise sales, but I'm wondering how that sales process works. A sales representative sets up a demo account. Then what, they load it with real customer data, with the customer's cooperation?

It seems like if a potential customer screws this up then they are wrong, but also, sales might be wrong to encourage them to share their data insecurely, and a sales process that encourages insecure sharing of enterprise data by potential customers might be bad company policy.

Perhaps doing it right would be tricky when it requires educating the customer. And it might get in the way of sales.

Ylpertnodi• 2 years ago

A sales representative sets up a demo account. Then what, they load it with real customer data, with the customer's cooperation?

Better than Fujitsu who actually went into UK Subpostmasters accounts and 'adjusted' their accounts live. No links - there are too many, but I would direct you to the various testimonies of the CEO of the PO, paula vennells (no capital letters for her) over the last....15 years.

dimask• 2 years ago

Template for writing GDPR subject access requests [0] for whoever it may concern. It is a bit too harsh, but it may be useful for one to know what info may have been compromised.

https://www.linkedin.com/pulse/nightmare-letter-subject-acce...

teej5• 2 years ago

In response to this incident, I just put together a Python CLI to help you monitor and take action against suspicious sessions in your account.

https://github.com/Titan-Systems/titan-security-tools

pjot2• 2 years ago

Where are the blocked ip’s from? Also rapeflake? Hesitant to google that

teej5• 2 years ago

Snowflake published these rules earlier today - https://community.snowflake.com/s/article/Communication-ID-0...

notsurewhynow• 2 years ago

The practice of how Snowflake team uses customer data and builds/helps them and their access is not expired is still on Snowflake; such access should be temporary in nature, tied to an active ticket upon its closure to auto expire. The fact that Snowflake has managed to attain (and/or keep?) their FedRamp High certification despite this, is a separate story for some investigative journalists (not just for the label but also to look into what public sector data might be at risk or already compromised...)

bbbruno222• 2 years ago

Hudson Rock exposing the name of the employee whose credentials were stolen shows me that Hudson Rock is not a serious company. I hope Snowflake is supporting her.

nycdatasci• 2 years ago

Snowflake’s blog about this incident mentions “former employee”. If she was actually fired as a result of this, I would hope that the entire security organization also went with her.

KingOfCoders3• 2 years ago

I've read Genestealer and thought, what?

Simon_ORourke• 2 years ago

I'll bet there's a few of those reputedly 400 companies keeping an eye on the news of this and another eye on their stick price.

If data gets out, especially financial data being reported to investors, that spells curtains for a company generally.

greyadept• 2 years ago

The article now seems to have been deleted and redirects back to their homepage.

a-ungurianu• 2 years ago

They seem to have unpublished the article, but it’s still available on the Wayback machine: https://web.archive.org/web/20240531140540/https://www.hudso...

shkkmo• 2 years ago

Not a good look. If they got something wrong, a retraction is appropriate after making these kinds of claims and accusations. Simply removing the post without explanation is irresponsible and unprofessional.

sherburt3• 2 years ago

I'm sure the Snowflake legal department is ready to open a can of whoop ass on them if that's the case.

darylteo• 2 years ago

May have been a cease desist from Snowflake and they don't want to take the heat alone?

Jgrubb• 2 years ago

They’ve definitely gotten the repeatedly-named Snowflake employee in a lot of hot water.

tillulen• 2 years ago

https://archive.is/PnDzt

https://web.archive.org/web/20240531225301/https://www.hudso...

cloud-ranger• 2 years ago

This is my first ever HN post, and I actually got distracted/took timeout from studing for a Snowflake cert (what are the odds; first ever story I saw about Snowflake on HN).

So, I was reading the comments and then in my mind I said to myself, this sounds like something connected with a Pegasus type company. The very next line in the comment I was reading was: "our boots-on-the-ground approach to cybercrime originates from the IDF's 8200 Cybercrime division"!

As others have noted, doxing the SE seems unnecessary...unless, that was part of the threat/proposal to Snowflake. You can imagine companies that had worked with that SE being concerned/need reassuring they're not affected.

I wouldn't at all be surprised if someone had bet against Snowflake stock before this story broke, if the story was hyped up enough or it was bad enough to spook the market.

Snowflake "strongly recommends" using 2FA for the admin account role, but users are free to decide whether to use it or not. Snowflake's website states: "MFA is enabled on a per-user basis; however, at this time, users are not automatically enrolled in MFA. To use MFA, users must enrol themselves." - I assume a future update will change that so they can enable it by default. MFA related questions always appear in lower-level Snowflake certs.

I'd assume (as a consultant) It would be against company policy to use a username/password for client work. Sometimes that's one of the first bad practices we see working with new clients. Perhaps that's why the SE is an ex-employee.

voytec• 2 years ago

Currently the linked URL 302-redirects to '/'. @dang, consider replacing with:

https://web.archive.org/web/20240531140540/https://www.hudso...

demosthanos3• 2 years ago

@dang is a no-op, consider contacting at the email in the footer.

But on the other hand, that they yanked the blog post is interesting and shouldn't be glossed over by just linking to the archive. What changed?

Edit: It looks like Snowflake's official response denies essentially all of the claims in TFA. Maybe Hudson Rock got taken in by a dishonest source and pulled the article when they realized their mistake?

https://community.snowflake.com/s/question/0D5VI00000Emyl00A...

beardedwizard• 2 years ago

extremely likely scenario. Hudson tried to write a splashy hit piece and was met with reality.

dandandan• 2 years ago

Quietly pulling the piece without a retraction is pretty shady as well

skilled3• 2 years ago

I don’t think the source was dishonest. Also, Snowflake hasn’t really denied anything. That press release still reads the same, and still acknowledges that customers were impacted.

Also, both infosec groups and journalists were given sample data. On top of that, Live Nation explicitly said in their SEC filing that the breach happened through a third-party cloud provider, which is what Snowflake is.

I think that if Ticketmaster starts to send out breach emails to their customers next week, that will put any suspicions about this being fake to rest.

It’s difficult to imagine ShinyHunters getting duped as well, since they are the initial strongest link to this whole thing.

vx-underground says that Mandiant and Crowdstrike worked with Snowflake and tl;dr of it is that Snowflake wasn’t breached. That is also strange. They weren’t breached but some customers were affected? :jackie_chan_holding_his_head_pic:

thejosh• 2 years ago

I'm not surprised. A few months ago I was reverse engineering their login flow (I was writing a driver for Elixir & Rust), and was getting MANY stack traces and weird bugs happening with their authentication flow, especially around OAuth.

Hacker confirms access through infostealer infection [withdrawn]

Related Stories