Microsoft Recall still storing credit card, social security numbers

https://www.tomshardware.com/software/windows/microsoft-recall-screenshots-credit-cards-and-social-security-numbers-even-with-the-sensitive-information-filter-enabled

by geekinchief • 2 years ago

221 111 2 years ago

tomshardware.com

Terr_2• 2 years ago

When I entered a credit card number and a random username / password into a Windows Notepad window, Recall captured it, despite the fact that I had text such as “Capital One Visa” right next to the numbers.

That undesirable outcome doesn't surprise me at all. Even if someone coded up logic to look for surrounding clues that X is a secret, that other data ("X is a password") might only become available seconds or weeks later.

For the foreseeable future, these idiot-savant systems (especially with append-only autocomplete at their core) will continue to be smart enough to get into trouble but not smart enough to get out of it.

dangus2• 2 years ago

I don’t even know if this is a negative aspect of what Recall is intended to do.

The whole point of Recall seems to be to allow you to…recall previous work you were doing.

It reminds me a lot of whole-device backups. If you have some secrets in plain view then the backup itself is going to need to be secured.

From the Microsoft website:

To use Recall you need to opt in to saving snapshots, which are screenshots of your activity. Snapshots and the contextual information derived from them are saved and encrypted to your local hard drive. Recall does not share snapshots or associated data with Microsoft or third parties, nor is it shared between different Windows users on the same device. Windows will ask for your permission before saving snapshots. You are always in control, and you can delete snapshots, pause or turn them off at any time. Any future options for the user to share data will require fully informed explicit action by the user.

iAMkenough• 2 years ago

It's doing what Microsoft intends, but not what customers intend. Entering my credit card or social security details in a secure form should not result in it being copied to a different folder on my local storage or added to my backups without my approval.

Snapshots present a new way of circumventing data protection implemented by other apps and websites (if it was visible on your screen, that data is now copied somewhere else).

Yes you can delete snapshots AFTER you've determined something sensitive has been inappropriately captured, but you are not warned or prompted to do so and at that point why even use the feature.

jazzyjackson6• 2 years ago

Well it's just kind of silly that Microsoft is silo'd enough that they didn't just use data loss prevention that they've implemented elsewhere in the office stack. Microsoft outlook stops me from sending bank wire details over email.

  With a DLP policy, you can identify, monitor, and automatically protect sensitive items across:

  Microsoft 365 services such as Teams, Exchange, SharePoint, and OneDrive accounts
  Office applications such as Word, Excel, and PowerPoint
  Windows 10, Windows 11, and macOS (three latest released versions) endpoints
  non-Microsoft cloud apps
  on-premises file shares and on-premises SharePoint
  Fabric and Power BI workspaces
  Microsoft 365 Copilot (preview)

  DLP detects sensitive items by using deep content analysis, not by just a simple text scan. Content is analyzed:

  For primary data matches to keywords
  By the evaluation of regular expressions
  By internal function validation
  By secondary data matches that are in proximity to the primary data match
  DLP also uses machine learning algorithms and other methods to detect content that matches your DLP policies

https://learn.microsoft.com/en-us/purview/dlp-learn-about-dl...

ethernot• 2 years ago

If you start with a sieve and try and plug all the holes to make a bucket you're going to end up with a leaky bucket.

jfghi• 2 years ago

I read that as Steve.

fxtentacle2• 2 years ago

I like that first comment:

"Microsoft continues to have a terrible abusive relationship with its customers. It's what Microsoft wants, not what the customer wants."

Yup, that pretty much sums up why I left for Linux.

hypeatei• 2 years ago

Recall was the catalyst for me switching to Linux. I do not wish to tinker with Windows so that it doesn't feed my credit card information into an AI black box.

Telemetry by default is unacceptable as well, but not to that degree.

eGQjxkKF6fif2• 2 years ago

Good choice.

I got Ubuntu, Lubuntu, Manjaro, Endeavour, Mx Linux, Fedora, Red Hat, Debian, Gentoo galore. Desktop Linux has come a long way, and if you like true unix here let me tell you some more. I got GhostBSD (https://www.ghostbsd.org/) rocking MATE IDE, if you want lightweight Linux desktop I got LXDE, LXFC, Gnome, KDE, and a whole lot more.

If you want true freedom, ditch the spyware and head on over to desktop Linux where we'll welcome you with open arms.

The excuse of 'But this... and that...' let me stop you right there.

You don't need to configure anything. Download Kubuntu and have fun with customizing everything in the settings.

When it comes to Windows and Microsoft products the answer is 'No.'

Keep it that way. Freedom matters.

bilekas• 2 years ago

Sounds like you have some commitment issues with picking a distro?

Or are you just trying them all out to let us know? Because you've named a few that are all the same family.

People who are unfamiliar with Linux will look at this list and just ignore it as too complicated. When in reality it's much simpler.

stuckkeys• 2 years ago

I got ubuntu desktop, I love it.

jazzyjackson6• 2 years ago

Gnome Fedora 41 is super pleasant and instantaneous on Apple silicon via Asahi. I haven't run into a single issue since installing, really impressive work. AFAIK thunderbolt support is the only thing lagging behind.

trod12342• 2 years ago

I hate to reign on your parade but Canonical (Ubuntu) is doing the same thing as Microsoft just to a slightly lesser extent.

The same pretty much goes for any of its many derivatives. Redhat is no longer open. Arch and alpine based have hardware issues.

The choices you once had with Linux are far less today than they have been in the past. You almost need to know how to build a distro from scratch to get comparable features and privacy to yesteryear.

booleandilemma• 2 years ago

This sums up almost all software today though.

My Google Pixel continuously asks me if I want to enable photo backup.

Uber Eats continuously asks me to enable notifications.

I tell the software no, again and again, day after day, and it keeps prompting me.

Most software I use is like this, it seems to be inescapable.

tdeck3• 2 years ago

My Google Pixel continuously asks me if I want to enable photo backup.

Google Photos does this on non-Pixel android phones too. Many times when I open the gallery app I'm confronted with a dialog where the backup checkbox is enabled and there's a continue button. It's like Google is trying to trick me into enabling it. This is an incredibly user hostile design. One day I will do it by mistake.

I mentioned this to a friend once and he said his kid got ahold of his phone for a few minutes and did enable it, causing a massive upload that put his Google account over quota. Of course it's a huge hassle to go through and unsync the photos without deleting them from the device.

sitzkrieg2• 2 years ago

and by no you mean the special, "maybe later" that every software loves to use

Symbiote• 2 years ago

It sums up proprietary software.

FOSS software does not have these misfeatures.

dizhn• 2 years ago

Play Store shows me a notification to ask whether I want to enable important notifications. In 3 years of having this phone I probably closed that thing 100 or more times, yet all it would take for it to go away is for me to say yes once.

KaoruAoiShiho• 2 years ago

Ironically Recall is what made me go back to windows.

layoric• 2 years ago

I was someone who switched to Linux in the last 12 months due to what was for me an increase in user hostile behavior of Windows. Care to share why, for you, Recall was important enough to switch OS from macOS/Linux(?) to Windows?

tetha• 2 years ago

A long time ago, Windows was a simple, hassle-free system to run games on. I knew Linux, but I knew Linux would take time to maintain and tinker around with, since that was my job. Thus, my private systems stuck with Windows.

Then Wine and especially Steam and Proton started to get to work, and Windows started to turn anti-consumer to a degree. At that point, I was kinda on the fence. Actively switching seemed somewhat scary, but if I had to setup something new, it was kinda... meh either way.

Then my Windows-OS SSD died. So I had to reinstall my OS anyway. Then I made the choice to install Fedora and switch back to Windows when it grows annoying. That was... two or three or more years ago? And the thing is, games work. Apps work. Funny enough, old games work better with DosBox and Wine than on windows. I have like two old games I can't get to run, but that's mostly because I don't have the energy to figure out QEMU with Direct3D support, which VirtualBox dropped some time ago for security reasons.

And now I'm looking at stuff like this.

nucleardog• 2 years ago

Hey, me too.

I managed Linux and BSD on servers all over the place. I worked pretty much exclusively using a Linux VM ... under Windows. Keeps most of the gremlins at bay. Hardware stuff's all handled by Windows, so it generally just works. Trying to play a game? Just run it in Windows and don't worry about it.

Every year or so I'd grab a live image and boot it and see what happened. But always ended up at "okay, too much hassle still".

Then a few things happened:

1. Valve invested heavily in Proton. They released the Steam Deck so suddenly developers _wanted_ their games to work in Proton.

2. One day I booted up a linux live image and stuff like my webcam just worked out of the box.

3. Microsoft released Windows 11.

Upgraded my computer at one point and I just went ahead and threw Debian on it and haven't really run Windows since. (Though I still have it installed. I still haven't found a good alternative to Adobe Lightroom.)

I too am really glad I got off of that ride when I did.

pjmlp3• 2 years ago

Yet the only way to have a gaming ecosystem is to translate Windows APIs, until the day Valve learns the OS/2 lesson.

They got lucky once, as WinRT ERA didn't took off as Microsoft expected.

Next time maybe not.

raxxorraxor2• 2 years ago

One reason it didn't take off was probably lacking compatibility layers to Win32 apps though.

And a world were you provision apps only through the Windows Store is pure hell. Perhaps it was a strategic decision to not support "legacy" apps. If so, Windows RT was killed by just that.

raxxorraxor2• 2 years ago

In contrast to Recall, they pixelate credit card information in Teams calls. Wanted to use the credit card of a colleague and we had to meet face to face like some cavemen. His office was 10m away too.

macspoofing• 2 years ago

Microsoft Recall screenshots credit cards and Social Security numbers, even with the "sensitive information" filter enabled

Because PII detection is a probabilistic exercise. You will miss things.

chawco• 2 years ago

While I agree as a whole, there are parts that are easily captured even with some small false positive rate, like credit card numbers. I do think it's acceptable to do PII detection probabilistically for some classes of identifiers/quasi-identifiers, because you can't really do any better without crazy false positive rates, things like credit card numbers have enough structure that it's more work to do it entirely via an ML model with a higher chance of failure, versus just building a simple heuristic for it.

Add to that the fact that missing a credit card number is way higher stakes than missing something like a zip code, you can understand why something like this is just not acceptable in a product like this, with the resources Microsoft has at their disposal.

kiratp• 2 years ago

Specifically regarding credit card security, it’s not really my problem is it? It’s the bank’s problem.

The law makes it so.

HideousKojima3• 2 years ago

Doesn't make it not a pain in the ass to resolve if your card information is stolen.

drdaeman8• 2 years ago

Stolen how? I don't get the fuss.

Please correct me if I'm wrong, but IIRC Recall is opt-in and keeps data on-device, and doesn't share it with any other systems or parties. And if one's device is compromised, they're screwed either way (keyloggers, password managers' data, etc), so while Recall data can be an interesting target, it's not like it's some game changer. I could be wrong, but I believe first iteration was user-accessible SQLite3 database (which was an issue), but Microsoft had tightened the permissions and isolated those files, so AFAIK it now requires additional authentication to access. I don't currently use Windows, so I can't really check, but that's what I've read.

If something is opt-in, local-only and partitioned away (inaccessible to regular-user processes to avoid easier abuse by malware and exploits) that sounds like a decent privacy-respecting option to me. There are plenty of crappy anti-user moves Microsoft had pulled with Windows, but Recall doesn't seem like one to me.

Unless, of course, they're forcing this on people (like how they aggressively do with Edge and OneDrive), or pull this data somewhere despite saying they don't do it, etc etc.

pdpi• 2 years ago

Specifically regarding credit card security, it’s not really my problem is it? It’s the bank’s problem.

Replace "problem" with "legal liability" and you're probably right. It's still very much your problem, though.

tpxl2• 2 years ago

Nope. If you give out your cc info, and it gets abused, the liability is on you.

drdaeman8• 2 years ago

That's if you willingly give out your card information? If you pull out your card and someone unauthorized looks behind your shoulder (or, more accurately, looks at a camera recording and sees your card in there), it doesn't make them authorized somehow.

pluc• 2 years ago

Have you ever been on the phone with a bank? Any bank? Banks don't lose. If this goes mainstream, you can bet there will be limiting legislation lobbied by banks to reduce their liability in the first week.

teeray• 2 years ago

No, it’s “Identity Theft.” The con the bank uses to shift liability back to you.

lesuorac• 2 years ago

Well at first it's the bank's problem.

If you end up with a large history of causing problems for banks they will drop you as a customer and it becomes your problem.

brunoqc• 2 years ago

Can we disable recall?

_joel• 2 years ago

Don't use windows?

drdaeman8• 2 years ago

Sure, just... don't enable it? Last I've heard, it's opt-in.

pavel_lishin• 2 years ago

It sure is today!

alkonaut• 2 years ago

Specifically for credit card numbers those should be easy to detect as they are a specific format.

But generally for PII, passwords etc there is no way to know when something is or isn’t secret or sensitive so either you should accept that the recordings are protected enough or just not record.

Did this controversy arise from Microsoft first assuming that the ”recordings are safely stored” would be enough but then public reception was negative and now they are trying to ”fix” it?

solarkraft• 2 years ago

They could have prevented a lot of the backlash by not forcing it on people.

spogbiper• 2 years ago

who did they force it on?

JohnMakin• 2 years ago

The fact it is on your machine at all and cannot be removed means it is forced. The fact that it is for now opt-in is irrelevant; windows is well known for enabling features without user permission that were previously not.

int_19h2• 2 years ago

The initial iteration (that was scrapped) was opt-out rather than opt-in.

araes3• 2 years ago

How is this even possibly a valid software pattern to enable on normal users?

Regularly capturing screenshots of their entire desktop, that 90% of users likely do not comprehend, and obviously associate with malware behavior. Screenshots. They're not even capturing the forms, or the specific input data. Taking entire desktop pictures of typing on Notepad.

Using a software that's difficult to tell whether it's installed. And then it keeps the credentials, makes it difficult to tell whether they've been stored, what info has been stored, whether they've been deleted correctly, and makes it difficult for the actual computer user to even access the stored images.

  The screenshots appear to be files in a subfolder called AsymStore.
  I couldn’t open those either and I tried to open them as PNGs, BMPs or JPGs.
  Perhaps hackers will figure out how to open these files, but as far as I could tell, a typical user can’t open them outside of the Recall app.

This reads like a virus pattern.

Several notable examples of malware, creepware, Remote Access Trojans (RAT) that do almost this exact activity: Agent Tesla [1], Dark Comet [2], Bifrost [3], and just the general category of Remote Access Trojans [4]. Corporate malware.

[1] https://en.wikipedia.org/wiki/Agent_Tesla

[2] https://en.wikipedia.org/wiki/DarkComet

[3] https://en.wikipedia.org/wiki/Bifrost_(Trojan_horse)

[4] https://en.wikipedia.org/wiki/Remote_desktop_software#RAT

drdaeman8• 2 years ago

It would be, if it would be stealthy enabled, unbeknown to machine owner. That - and not the fact it records screen - is what differentiates malware from legit software.

It is opt-in, which makes it equivalent of user explicitly setting up a camera to record their work, for a well-intended ability to review those recordings if they need to recall something.

If we'll start saying that end-users are somehow incapable of comprehending what screen recording means, then we're basically giving up our agency and arguing we need a nanny. I sincerely hope we don't. Like, literally, it's a screen recording, anyone with a working brain (no matter whenever they're technically literate or not) should be able to tell what consequences - positive and negative it would have.

I found a screenshot - the opt-in prompt literally says "Allow Windows to save snapshots of your screen?" If that's not clear or comprehensible, I don't know what is. People who are caught by this must simply ignore and not read what it says on the screen.

araes3• 2 years ago

Maybe if it did something very specific you had to set up. Maybe.

  "You're using Microsoft Edge, would you like to record your (specific) usage of passwords, personal data, and form entries?" (that can be found encrypted (here) if you're interested)

Not some large scale screenshot operation. The issue is with taking "snapshots of your screen" with no real knowledge of what is even going to be worked on or recorded afterward. How horrible are the identity theft issues? (BS non-threats requiring logins: social blab / news website account with no money attached, Normal dangerous stuff: credit card use (cancel/contest), Somewhat rare: bank accounts (long bank fight), Rare and dangerous: scanned federal IDs (endless nightmares with the feds)) It even makes the task a greater difficulty because there's so little specificity about what's being recorded or what form the recorded information is going to take. And the results are notably spotty like the article mentions.

Also, 95% users. [1] "Across 33 rich countries, only 5% of the population has high computer-related abilities, and only a third of people can complete medium-complexity tasks" Even if we put this in the "only Poor and Terrible skill users would fail" that's still ~50% of the population.

How many click through stuff with "Yet another BS alert to get rid of. Go away, I want to actually use my computer." The WWW has long ago deadened most to annoying spam popup authorizations.

[1] The Distribution of Users’ Computer Skills: Worse Than You Think: https://www.nngroup.com/articles/computer-skill-levels/

ghelmer• 2 years ago

I'm not apologizing for MS, and I have no idea what PII protection Recall actually has. If Recall does have real PII logic, it should recognize that a legitimate VISA payment card numbers must start with '4' + be 16 digits in length, and AmEx cards must start with '34' or '37' + be 15 digits in length; also, the LUHN algorithm must be satisfied over the card digits.

With Recall, it seems false positives for PII-type protection rules would be more acceptable than false negatives. But with the negative press already around the technology. I'm not sure it will ever gain acceptance.

Rygian2• 2 years ago

There are card numbers that don't satisfy Luhn.

drdaeman8• 2 years ago

I'm really curious now - which cards don't conform?

I always thought it's basically an industry standard and no network issues cards that don't have a correct checksum.

exikyut• 2 years ago

TIL, although this isn't a field I need perfect knowledge of.

I wonder if people with such cards have inexplicable problems with certain payment systems?

Edit: just saw the sibling comment, was viewing an old copy of the page

Rygian2• 2 years ago

Folks at $dayJob keep a list of prefixes of credit card numbers that don't satisfy Luhn, currently there's 8 of them.

And while we're at it, length of credit card numbers is not fixed, can go from 12 to 19 numbers (both Luhn and non-Luhn).

johnklos• 2 years ago

If there isn't a law yet, there should be: if something can be done incorrectly, Microsoft will do it incorrectly multiple ways.

Granted, we're a somewhat technical bunch here, so I have to ask: do regular people not know that Microsoft is so bad at security and self awareness that they literally can't do something like protect users from their own products? Do people still think, "Oh, well - 80% of the world can't be wrong"?

I bet they're going to make it exceedingly difficult to disable or uninstall, like Edge, once it becomes a mandatory part of Windows, aren't they?

weikju• 2 years ago

Do people still think, "Oh, well - 80% of the world can't be wrong"?

Yes and “if they’re so big they must be doing something right”

And “if they did something wrong the govt would come down on them”

And “they already have all my data anyway so who cares?”

I’ve heard all of those and more.

hulitu3• 2 years ago

do regular people not know that Microsoft is so bad at security and self awareness that they literally can't do something like protect users from their own products?

Some people don't know, some are paid to not know. /s

nerdjon• 2 years ago

Honestly at this point did they actually bother to make something specifically built and trained to remove sensitive data or did they just modify a system prompt to "Don't save possibly sensitive data like credit card numbers" and hope that an LLM could magically handle this properly?

Why do I have a feeling its the later given all of the other issues around this entire thing.

sitzkrieg2• 2 years ago

i will never move off windows 10 LTSC :-)

josefritzishere• 2 years ago

Worst product ever. I think Microsoft's "telemetry" initiative has acclimated them to the point where they're just making naked malware features now.

doright• 2 years ago

I find it interesting that searchable history of computer activity is a problem with enough interest that independent developers have worked on solutions themselves, some of them open source. But a company like Microsoft ought to have much more resources such that they can fix these edge cases or realize they can't fix them all and design the service accordingly.

So I guess my question is: is a company as big as Microsoft that approaches this problem space doomed to fail from the start, because of the perception issues? Would this be any different if say Apple had developed a Recall alternative and they also found it impossible to censor credit card information in an arbitrary Notes window someone whipped up as an edge case, like in the article? Or could a stricter (outward) stance on privacy make it palatable again?

Suppose if everything were assured to be kept under enough layers of encryption and the data wasn't synced online at all, would storing a credit card number surreptitiously captured on my computer be seen as much better if it's Microsoft/Apple I have to trust to engineer their AI recall feature in a secure manner?

alt227• 2 years ago

I never heard anything bad about https://rewind.ai which was launched exclusively on Mac before Recall was, in fact it was praised.

int_19h2• 2 years ago

As a third party app, most people haven't even heard about Rewind.

Recall, by virtue of being a core OS feature - and the one enabled by default at that - got a lot more coverage.

luma• 2 years ago

One major difference would be in how it is distributed. Rewind is an app you go find, install, and deploy. Not a lot of concerns around consent there.

Building it into the OS, potentially default on, is the same sort of technology. It's not the same sort of consent.

jazzyjackson6• 2 years ago

Rewind made a big ballyhoo about being local, not reliant on cloud services, not syncing anywhere etc

Windows 11 is practically a cloud OS when paired with office365 and OneDrive, users may be forgiven for expecting their history will be exfiltrated from their machine, employers will have access to their screen recordings etc.

btown• 2 years ago

EDIT: I may have spoken too soon on the below; I checked myself, and the journalist's test card numbers in the OP do not, in fact, pass the Luhn algorithm! So perhaps some grace is deserved. But I'm preserving my comment below, if anything to be illustrative of how companies should approach this, and that if you want your product to be secure from criticism from journalists who don't know how to make test credit card numbers, you should possibly use even more robust approaches than what one random person on HN comes up with in real time.

===

But these are incredibly solvable problems! If a series of digits on screen passes the Luhn algorithm (https://en.wikipedia.org/wiki/Luhn_algorithm), or matches AAA-BB-CCCC, prevent that screen/area from being captured! And Microsoft literally owns the code for https://docs.github.com/en/code-security/secret-scanning/int... for determining other high-entropy secrets.

We're not nearly at the level of https://xkcd.com/1425/ - and even that canonical example has been entirely solved by now.

The problem isn't that these things are fundamentally impossible. And the problem isn't even that Microsoft decided speed to market was more important than safeguarding their users' data - I get speed to market!

The problem, allegedly, is that Microsoft said these things were fixed without actually fixing them [EDIT: see above], and didn't think that their users' data was important enough to assign a red team or even an empowered SDET to do even the simple tests this journalist did before making that announcement.

geekinchief• 2 years ago

FYI, seeing this comment, I went back and tested again with my actual credit card number and the situation was the same.

therein• 2 years ago

Even if you trust Microsoft not to keep it, not to access it and index it on their cloud, and even if they were kept encrypted and local, it is still searchable and index is available on runtime. So you have yet another process that has this unencrypted in memory. Great target for malware no matter what.

sensanaty• 2 years ago

There's a universe of difference between an individual choosing to install this kind of software of their own volition on their own machines (or making their own version as you pointed out), and the comically evil megacorporation that is microsoft forcing it on everyone who uses windows with no way of knowing if disabling it actually disables it, and with no way to trust a single word they say because, again, it's microsoft we're talking about. The same company that already phones home with egregious amounts of data for everything you do and see and shoves ads and other bloatware right into your taskbar which requires user scripts to temporarily disable before being reinstalled anyway on a system update.

How anyone can have trust that rewind won't be misused is baffling to me, I wouldn't trust a Janitor at M$ to not sell their own family out for a 1% YoY profit increase, yet alone the engineers working there and their psychotic C-suite.

sydbarrett74• 2 years ago

My plan is to run HardenedBSD for most things, MacOS for games, and Windows for anything that absolutely won't run otherwise. Nadella has shown his contempt for power users way too often.

jgalt212• 2 years ago

So long as MSFT stock remains airborne, there will no changes to its behavior.

soraminazuki• 2 years ago

We’ve updated Recall to detect sensitive information like credit card details, passwords, and personal identification numbers.

What's up with companies giving these kinds of non-answers? It rubs me the wrong way every single time. This is definitely not an answer to the author specifically telling MS that their defense measures aren't working.

Microsoft Recall still storing credit card, social security numbers

Related Stories