Employees are feeding sensitive data to ChatGPT, raising security fears

https://www.darkreading.com/risk/employees-feeding-sensitive-business-data-chatgpt-raising-security-fears

by taubek • 3 years ago

553 339 3 years ago

sigstoat• 3 years ago

when it first came out and my boss was behind himself about how cool it was, he was feeding it all of his emails with other businesses to have it clean them up. boggled my mind.

dannyobrien• 3 years ago

do those other businesses use gmail? does your company?

SketchySeaBeast2• 3 years ago

I think those are different models.

Gmail has a vested interested in keeping any knowledge it gains about you secret - it's competitive advantage is knowing more about you than anyone else does.

ChatGPT's strength is its ability to clearly communicate the knowledge it has (including training data it gains from people it interacts with) to give you good responses.

guestbest• 3 years ago

This is a user led data leak that ranks up there with Facebook and LinkedIn asking for email passwords to “look for your contacts to add”.

varunjain99• 3 years ago

Not only do you have to worry about employees directly sharing data, but many companies are also just wrappers around GPT. Or they may use your data in the future to roll out new AI services.

While this is not a new problem -- employees share sensitive data with Google all the time -- the data leakage will be more clear than ever. With ads-based tracking and Google search, the leakage was very indirect. With generative AI, it can literally regurgitate memorized documents.

The security risk goes beyond data exfiltration. Folks are already trying to teach the AI incorrect information by spamming it with something like 2+2 = 5.

Data exfiltration + incorrect data injection are super underrated risks to mass adoption of generative AI tech in the B2B world...

cuuupid2• 3 years ago

We block ChatGPT, as do most federal contractors. I think it’s a horrible exploit waiting to happen:

- there’s no way they’re manually scrubbing out sensitive data so its bound to spill out from the training data when prompting the model

- OpenAI is openly storing all this data they’re collecting to the extent that they’ve had several leaks now where people can see others’ conversations and data. We are one step away if it hasn’t already happened from an exploit of their systems (that likely weren’t built with security as the top priority as opposed to scale and performance) that could leak a monumental amount of data from users.

In the most innocent case they could leak the personal info of naive users. But largely if Linkedin is any indication, the business world is filled with dopes who genuinely believe the AI is free thinking and better than their employees. For every org that restricts ChatGPT use, there are fifty others that don’t, most of which have at least one of said dopes who are ready to upload confidential data at a moments notice.

Wouldn’t even put it past military personnel putting S/TS information into it at this point. OpenAI should include more brazen warnings against providing this type of data if they want to keep up this facade of “we can’t release it because ethics” because cybersecurity is a much more real liability than a supervised LM turning into terminator.

Ensorceled• 3 years ago

Let's also not discount that for every "dope" there is at least one "bad actor" who is willing to take the risk to get an edge in their workplace or appease their managers demands. The warnings will only deter the first group.

comment_ran• 3 years ago

If your competitor use ChatGPT to compete with you and they're 10x productive than yours, are you still willing to insist? If the productive is 100x, will you?

amelius• 3 years ago

Uh, there's no sign of that yet.

brobdingnagians• 3 years ago

It might be just as likely that ChapGPT will cause a mistake like Knight Capital because no one bothered to thoroughly verify the AI's looks-good-but-deeply-flawed answer, and the two aren't mutually exclusive possibilities.

tomatotomato37• 3 years ago

This isn't an argument of ChatGPT vs nothing. This is an argument of "external" ChatGPT vs some other AI sitting on your own secured hardware, maybe even a branch of ChatGPT.

bcrosby953• 3 years ago

Security and privacy should be table stakes. Speaking for my country, we needs privacy laws with teeth to punish bad actors from shitting people's private information where ever they want in the name of a dollar.

roflyear2• 3 years ago

Man the fanboyism is out of control here.

michaelteter2• 3 years ago

Possibly I don't know how this all works, but I think if the host of a ChatGPT interface were willing to provide their own API key (and pay), they could then provide a "service" to others (and collect all input).

In that case, you wouldn't know to block them until it was too late.

Ultimately either you must watch/block all outgoing traffic, or you must train your people so thoroughly that they become suspicious of everything. Sadly, being paranoid is probably the most economical attitude these days if IP and company secrets have any value.

dragonwriter2• 3 years ago

Possibly I don’t know how this all works, but I think if the host of a ChatGPT interface were willing to provide their own API key (and pay), they could then provide a “service” to others (and collect all input).

Well, GP was referring to blocking ChatGPT as a federal contractor. I suspect that as a federal contractor, they are also vetting other people that they share data with, not just blocking ChatGPT as a one-off thing. I mean, generic federal data isn’t as tightly regulated as, say, HIPAA PHI (having spent quite a lot of time working for a place that handles both), but there are externally-imposed rules and consequences, unlike simple internal-proprietary data.

yathrowaway015• 3 years ago

So you block internet access for all employees? Cos anything you think is being pasted into ChatGPT is being pasted everywhere, whether its Google, Slack, Chrome Plugins, public Wifi.

toyg• 3 years ago

Or all the ChatGPT clones that have sprung up and will continue to spring up every other day.

It's a stupid and patronizing position, but corporate IT are sadly incentivised to be stupid and patronizing.

kube-system• 3 years ago

Yes, these things are sometimes blocked in higher security workplaces… up to and including the public internet. Honestly airgapped systems are not all that uncommon anywhere that human life is at risk.

probablynish• 3 years ago

This really depends on the cost/benefit tradeoff for the entity in question. If using ChatGPT makes you X% more productive (shipping faster / lowers labor costs / etc), but comes with Y% risk of data leakage, is that worth it in expectation or not? I would argue that there definitely exist companies for which it's worth the tradeoff.

By the way, OpenAI says they wont use data submitted through its API for model training - https://techcrunch.com/2023/03/01/addressing-criticism-opena...

heyyyouu• 3 years ago

But that's the API, not the Chat input or Playground.

Companies can use Azure OpenAI Services to get around this -- there's data privacy, encryption, SLAs even. The problem is it's very hard to get access to (right now).

eldaisfish• 3 years ago

the #1 problem with corporations saying things is that many things they say are not regulated or are taken on good faith. What happens with OpenAI are acquired and the rules change? These comments are often entirely worthless.

osterbit2• 3 years ago

To anyone who may be pasting code along the lines of 'convert this sql table schema into a [pydantic model|JSON Schema]' where you're pasting in the text, just ask it instead to write you a [python|go|bash|...] function that reads in a text file and 'converts an sql table schema to output x' or whatever. Related/not-related--great pandas docs replacement is another great+safe use-case.

Point is, for a meaningful subset of high-value use-cases you don't need to move your important private stuff across any trust boundaries, and it still can be pretty helpful...so just calling that out in case that's useful to anyone...

eptcyka• 3 years ago

Risk of leakage? It is not a risk, it is a matter of time.

crabmusket• 3 years ago

If using ChatGPT makes you X% more productive (shipping faster / lowers labor costs / etc), but comes with Y% risk of data leakage

X and Y are not alike, and should not be compared. X is a benefit to you(r employer), whereas Y is a risk to the customer who has entrusted you with their data.

HeavyStorm• 3 years ago

You've certainly not worked with _real_ sensitive data. The kind that can bankrupt your business.

I do and if it could be leaked through ChatGPT I would have it blocked.

Risk isn't a single dimension, it's a combination of exposure (chance of happening) and impact (how much will you lose)

nostromo• 3 years ago

Does blocking ever work? People are smart and usually just work around them.

mnd999• 3 years ago

The smart ones don’t paste in all their private data.

And yes, if the bypassing the block is combined with disciplinary action, it does work. It’s not worth getting fired over. This is likely what heavily regulated industries like financial services and defense are doing.

tedunangst• 3 years ago

Blocks are effective reminders of policies.

PeterisP• 3 years ago

It works in the sense that it does add an extra "reminder" and requires specific intent. I mean, in this scenario all the people already have been informed that they're absolutely not allowed to do things like that, but if someone has forgotten that, or simply is careless and just wants to "try something out" then if it's unblocked they might actually do it, but if they need to work around a restriction, that forces them to acknowledge that there is a restriction and they shouldn't try to work around it even if they can.

m000• 3 years ago

Does US intelligence have access to OpenAI data? Private organizations is one thing. But with all the dopes in government positions around the world, OpenAI logs would probably be a treasure trove for intelligence gathering.

grepfru_it2• 3 years ago

They are just one national security letter away from all US-held data.

px43• 3 years ago

Microsoft is well known for piping data to US intelligence as a service. It's almost certainly why they bought Skype, then removed all the end to end encryption.

speedgoose• 3 years ago

USA has the patriot act and the cloud act to request the data from any USA company. Like AWS, Microsoft 365, Google…

SrslyJosh• 3 years ago

Wouldn’t even put it past military personnel putting S/TS information into it at this point.

Hey, they need someone to proofread their War Thunder forum posts to make sure they're using correct spelling and grammar when leaking classified info. ;-)

(Ref if you don't get the joke: https://taskandpurpose.com/news/war-thunder-forum-military-t...)

RF_Savage• 3 years ago

Not only that, but the European theater nuclear forces leaking security arragements and even door PIN-codes for nuclear weapons bunkers via online flash card sites might be a better example.

As the leaks ware more inadvertent.

nomilk• 3 years ago

there’s no way they’re manually scrubbing out sensitive data

I was under the impression OpenAI weren't using questions as training data for future models. I recall Sam Altman saying they delete questions after 1 month, but I can't locate the source for that.

whitemary• 3 years ago

Even if they do today, they could change their mind tomorrow, or even tonight.

Schroedingersat• 3 years ago

Right. They also haven't been caught lying about everything else from cuttoff dates to live functionality.

wkat42424• 3 years ago

Even without training the model, it will still end up in logs. For example you can now see your previous questions in the UI and it'll probably be stored in other places of their backend too.

This is still a serious data loss risk.

capybara_2020• 3 years ago

I am curious, do you block MS Edge? It has a grammar check for all input boxes that sends data to MS servers to check. Similar to what Grammarly does.

MS also "helpfully" asks you if you want to use that enhanced grammar check in MS Word(as far as I have seen, might be there in other office products too). I cannot imagine sending all my documents to MS. But I am not sure most users will realize what is happening.

All these companies offer helpful services but are hovering up data and no one knows the consequences yet. It feels like ChatGPT is just one symptom of a bigger problem.

lazyasciiart• 3 years ago

You can disable the feature entirely in group policies, I imagine organizations with a decent IT org will do so before deploying the update.

cuuupid2• 3 years ago

We don't but the grammar check is disabled. In general anything cloud-based services are vetted before being allowed.

I think MS Edge is getting even worse about this, with the big fucking Bing icon in the corner and making it impossibly hard to get rid of it.

nukeman• 3 years ago

Some military folks put nuclear weapons storage training materials onto Quizlet, so I don’t doubt for a second people would try to put ChatGPT onto a classified computer system.

check_muk• 3 years ago

By using Azure, you can access ChatGPT and GPT, which come with enterprise-grade security and established data agreements, setting them apart from OpenAI. I'm not entirely sure of the technical details, but you can explore this option.

ozim2• 3 years ago

Problem is you will have to heavily advertise it in your org because people will not understand why, where etc.

They will go to OpenAI directly and do stuff because "they want it now" and they don't understand why not.

Microsoft is already building it into Office 365 with the same enterprise-grade agreements so it might get easier that way.

tempestn• 3 years ago

they’ve had several leaks now where people can see others’ conversations and data

Do you have a source for this? I know some people have claimed to see others' data, but I haven't seen any evidence that that's what's actually being seen, vs LLM hallucinations. OpenAI claims, and I can't imagine they're lying, that the training data is fixed and ends in 2021, so I don't see how it would be possible for user prompts to be leaking into output, absent a massive and very unlikely bug (compared to the much more likely AI hallucination explanation).

nfm• 3 years ago

Yep: https://openai.com/blog/march-20-chatgpt-outage

Some kind of concurrency bug in a library they were using to retrieve cached data from Redis led to this leak.

We took ChatGPT offline earlier this week due to a bug in an open-source library which allowed some users to see titles from another active user’s chat history. It’s also possible that the first message of a newly-created conversation was visible in someone else’s chat history if both users were active around the same time.

PUSH_AX• 3 years ago

I think they’re referencing this: https://openai.com/blog/march-20-chatgpt-outage

hannob2• 3 years ago

We block ChatGPT, as do most federal contractors. I think it’s a horrible exploit waiting to happen:

Do you also block pastebin? Anything else that has a web form? How is ChatGPT special compared to any other service on the Internet where people can paste data in a form?

I mean... I see the problem, but I think one needs to realize that it's a far more generic problem that has basically nothing to do with ChatGPT and AI. If people paste confidential data into random webpages that's of course bad. But if you block ChatGPT because you fear that, it means you expect that people might do that. And then your problem is not ChatGPT, but lack of awareness what is confidential data and what to do with it.

detaro• 3 years ago

Because "awareness only" has such a great track record when it comes to security-adjacent issues, and totally satisfies auditors/customers/regulators/...?

gpderetta2• 3 years ago

Do you also block pastebin? Anything else that has a web form?

pastebin and indeed most things that has some sort of public webform is blocked in all the companies I have worked with.

It is probably a losing battle though, as it is very hard to block everything without default deny.

Paradoxically, maybe GPT could be used to veto websites on first access :)

MarkusWandel• 3 years ago

Wouldn't it be trivial to add a "read-only" mode to the LLM's operation, where it uses stored knowledge to answer queries but doesn't ingest new knowledge from those queries?

humanistbot• 3 years ago

From https://help.openai.com/en/articles/7039943-data-usage-for-c...:

You can request to opt out of having your content used to improve our services at any time by filling out this form (https://docs.google.com/forms/d/1t2y-arKhcjlKc1I5ohl9Gb16t6S...). This opt out will apply on a going-forward basis only.

It goes to a google form, which is I guess better then them building their own survey platform from scratch that may have more vulnerabilities.

meghan_rain3• 3 years ago

I'd be worried this is also the "how to get banned from OpenAI in the near future" form. and if OpenAI retains a monopoly like Google does for search, you are basically screwed.

eternalban• 3 years ago

them building their own survey platform from scratch

Funny. This nobel prize winner raises an interesting question:

If your AI is so great at coding, why is your software so buggy?: https://paulromer.net/openai-bug/

johntiger12• 3 years ago

Why isn't it read-only by default? it's not even connected to the internet

ben_w4• 3 years ago

ChatGPT, and I think all the GPT LLMs, is only accessible over the internet as far as I can tell.

And the thumbs up/down are there on the chat interface because it's partly trained by reinforcement from human feedback.

w_for_wumbo• 3 years ago

This is the issue with a tool so powerful, you can't just tell people not to use it, or to use it responsibly. Because there's too much incentive for them to use it. If it saves hours of a persons' workday, and they're not seeing any of the harm caused from data leakage, there's no incentive for them to not use it.

Which is why a private option is so critical. To not fight against human nature, means providing an ability to use the tool in a safe way.

JohnFen9• 3 years ago

There's a dev here who is using ChatGPT extensively in his work. The rest of the team is just waiting for him to get caught and fired. Sharing company data with unapproved external entities is very definitely a firing offense.

yathrowaway015• 3 years ago

Glad I work for a company where the CEO pays for everyones ChatGPT Plus for the devs. If you think your code is special then you're wrong.

arcticfox2• 3 years ago

If they're more productive by doing it, I think it's an equal chance said dev gets promoted.

mym19902• 3 years ago

If you really care about your company's security, you should report it, otherwise you are just complicit.

ghaff2• 3 years ago

1.) It's not at all clear it's nearly as powerful as you think. Certainly in my domain--writing about various topics--it's not.

2.) Of course, you can tell people not to use it. Unlike people at SV companies apparently, people in government and government contractors accept restrictions like not having phones in secure labs all the time. Start firing or even prosecuting people and people will discover very quickly they don't really need some tool.

And, yes, private versions of this sort of thing helps a lot.

raincole3• 3 years ago

you can't just tell people not to use it

Uh, why can't you tell people not to use it...? If security is that important for your company, of course you can tell your employees which tools to use.

A fun fact: in many areas of TSMC, smart phones are banned. No one says "you can't just tell people not to use smart phones."

q845712• 3 years ago

https://www.bleepingcomputer.com/news/technology/fitness-tra...

wil421• 3 years ago

That’s very common in a lot of secure places. There are tons of government contractors who have to put smartphones away due to the cameras on them. Not to mention the more secure government places.

KirillPanov3• 3 years ago

in many areas of TSMC, smart phones are banned

This does not surprise me at all. What I want to know is how they enforce it.

Unless they have something better than "fear of somebody seeing you using the smartphone", it isn't getting enforced. If they do have something better I want to know what.

phineyes• 3 years ago

Let’s not forget that we’re also feeding in all our code into OpenAI Codex.

nodesocket• 3 years ago

Which I assume also means sensitive code that may be .gitignore but being pushed up to OpenAI. I.E. secrets, passwords, api keys.

est• 3 years ago

Many people like me like to paste stuff into an editor to strip the formats.

And Github Copilot will gather that tiny amount of data to who knows where.

agloe_dreams3• 3 years ago

Meanwhile over at Github Copilot...

Hahahahahahaha

dubbelboer• 3 years ago

If you're using Github already then Copilot isn't seeing anything new.

agloe_dreams3• 3 years ago

Correct, but that level of security is expected from GitHub proper, they have all sorts of independent security reviews for their partners. Does all of that exist for Copilot?

varunkmohan• 3 years ago

Exactly why our product Codeium (Copilot alternative) supports self-hosting.

bob10295• 3 years ago

We published an internal policy for AI tools last week. The basic theme is: "We see the value too, but please don't copypasta our intellectual property until we get a chance to stand up something internal."

We've granted some exceptions to the team responsible for determining how to stand up something internal. Lots of shooting in the dark going on here, so I figured we would need some divulgence of our IP against public tools to gain traction.

meghan_rain3• 3 years ago

Inform us when you figured out a way to host something with the quality of ChatGPT internally :-)

bob10295• 3 years ago

Even if we had a 100% private ChatGPT instance, it wouldn't fully cover our internal use case.

There is way more context to our business than can fit in 4/8/32k tokens. Even if we could fit the 32k token budget, it would be very expensive to run like this 24/7. Fine-tuning a base model is the only practical/affordable path for us.

visarga2• 3 years ago

You can use chatGPT inside Azure, like any other service. It's not the same one used by OpenAI, and there are different guarantees.

ChatGPT is now available in Azure OpenAI Service

https://azure.microsoft.com/en-us/blog/chatgpt-is-now-availa...

knodi123• 3 years ago

I can host llama's 7b model internally. It hallucinates more often than not and has a tendency to ramble, but dammit it's local and secure!

philsquared_3• 3 years ago

Just use the API? It deletes your data after 30 days...

BonoboIO• 3 years ago

Not using Chatgpt is easy, but things like GitHub or VSCode with Copilot (which is a special version of GPT3) and in the future Copilot X (gpt4) this will get hard.

One developer opening a folder in VSCode with Copilot enabled aaaaand it’s gone. You never know what part of the folder left your building.

mewpmewp28• 3 years ago

What if you host your code in GitHub? That concern is weird to me, because you already give Microsoft pretty much everything.

You use Windows, VSCode, etc, all of this has access to your code.

wkat42424• 3 years ago

Our policy for now is the exact same.

The problem is the verification of this of course.

brightball• 3 years ago

This was my first concern when it came to IDE plugins.

yawnxyz• 3 years ago

I think there's more fear of OpenAI leaking data than say, Airtable or Notion or Github or AWS/S3 or Cloudflare or Vercel or some other company that has gobs of a company's data. Microsoft also has gobs of data: anything on Office and Outlook is your company data — but the fear that they'll leak (intentional or accidental) is somehow more contained.

If we want to be intellectually honest with ourselves, we can either be fearful and have a plan to contain data from ALL of these companies, OR, we address the risk of data leaks through bugs as an equal threat. OpenAI uses Azure behind the scenes, so it'll be as solid (or not solid) as most other cloud-based tools IMO.

As for your data training their data: OpenAI is mostly a Microsoft company now. Most companies use Microsoft for documentation, code, communications, etc. If Microsoft wanted to train on your data, they have all the corporate data in the world. They would (or already could!) train on it.

If there's a fear that OpenAI will train their model on your data submitted through their silly textbox toy, but NOT through training on the troves of private corporate data, then that fear is unwarranted too.

This is where OpenAI should just get a "corporate" tier, charge more for it, and is basically make it HIPAA/SOC2/whatever compliant, and basically do that to assuage the fears of corporate customers.

rrdharan• 3 years ago

1. Azure has the worst security of the major cloud providers; multiple insanely terrible RCE and open readable DB exposures.

2. Azure infrastructure still likely has far better security/privacy by virtue of all their compliance, (HIPAA, FedRAMP, ISO certifications etc.) than whatever startup-move-fast-ignore-compliance crap OpenAI layers on top of it in their application layer.

philwelch• 3 years ago

For most of those tools you can get your own self-hosted version if you’re worried about your data.

krisoft• 3 years ago

I think there's more fear of OpenAI leaking data than say, Airtable or Notion or Github or AWS/S3 or Cloudflare or Vercel or some other company that has gobs of a company's data.

There is zero fear. OpenAI openly writes that they are going to use ChatGPT chats for training. On the popup modal they show you when you load the page. That is not a fear, that is a promise that they will do leak it whatever you tell them.

If i tell you “please give me a dollar, but i warn you you will never see it again” would you describe your feeling over the transaction as “fearful that the loan won’t be repaid”?

crop_rotation• 3 years ago

In my experience most corporate employees just take the path of least resistance. It is not uncommon for people to paste non public data into websites just to do json formatting, and paste base64 strings to random websites just to decode them. So just telling people not to do something won't accomplish much. Most corporate employees also somehow think they know better than the policy.

Any company that doesn't want to feed data into ChatGPT should need to proactively block both ChatGPT and any website serving as a wrapper over it.

japhyr• 3 years ago

and any website serving as a wrapper over it

I agree this would be a good move, but it's going to be harder and harder to do that definitively.

sophiabits2• 3 years ago

Perhaps we need an “LLM Block” browser extension? :)

itronitron• 3 years ago

A while back I got to hear about how the IT team running my then-employer's internal time reporting tool was sending all the usage data through Google Analytics and how neat that was for them to look at :\ .

I shudder to think what they are doing now.

mewpmewp28• 3 years ago

I am not sure I see the issue here?

dougb5• 3 years ago

Given that they use all the labor of the Internet without attribution, we should assume that they will use every additional drop of data we give to them for their own ends.

ChatGTP2• 3 years ago

This is what I hate about it.

DoingIsLearning• 3 years ago

Funnily enough I wrote a cautionary comment on this just 2 days ago :

https://news.ycombinator.com/item?id=35299695

syntaxing2• 3 years ago

I’m curious if anyone’s employer has set up their own LLM. My employer has a couple of A100 sitting around which could easily host a couple instance of 65B LLaMA or Alpaca. Convincing upper management to allow me is the hard part.

tiborsaas• 3 years ago

Team up with someone from sales, marketing :)

AnthonyMouse2• 3 years ago

I just run it on my desktop? 64GB of DDR4 is <$150.

syntaxing2• 3 years ago

I'm assuming a quantized version?

swyx2• 3 years ago

what on earth do you need to convince them apart from gestures at all this

Mizoguchi• 3 years ago

"In one case, an executive cut and pasted the firm's 2023 strategy document into ChatGPT and asked it to create a PowerPoint deck."

There's really not much you can do here. This is complete lack of very basic common sense. Having someone like this in your business, particularly at the executive level, is a liability regardless of ChatGPT.

yellow_postit• 3 years ago

This cycle happens regularly and it seems often times the service provider wises up and charges for extra controls.

Yammer pre-Microsoft and nowadays Blind — lots of “insider” information seemingly posted.

As usage goes up the target size, and opportunity cost, both go up.

NoZebra120vClip• 3 years ago

This is nothing new at all. How many people have Grammarly plugins installed? They are advertising aggressively, so I'd think it is the new hotness. Don't tell me Grammarly is not hoovering up all of the Slack, Word, Docs, and Gmail data that everyone sends it, and holding on for some future purpose. We'll see.

belter• 3 years ago

2017... "Grammarly Vulnerability Allows Attackers To See Sensitive Data of Their Customers" - https://www.invicti.com/blog/web-security/grammarly-vulnerab...

sophiabits2• 3 years ago

Grammarly is an OpSec nightmare that’s somehow managed to slip under most people’s radar. I know folks who won’t use the Okta browser extension because of the extensive permissions it asks for, but will happily use Grammarly on everything.

Last time I looked (maybe things have improved…?) Grammarly would automatically attach itself to any text box you interacted with, and immediately send all of your content to their servers for processing. How this software gets past IT departments is a mystery to me.

johntiger12• 3 years ago

It's alright, I just told it that I don't consent to my data being used. Checkmate openAI!

satvikpendem• 3 years ago

I noticed this too, this is why I'm working on a startup that lets you train your data internally through a self-hosted OSS LLM product. It's going to start off with just reading emails for now until we can integrate with other products too, like spreadsheets, Airtable, databases, etc. Basically an OSS version of OpenAI ChatGPT plugins, I suppose. If you're interested, message my at my email in my profile.

baxtr• 3 years ago

We went pretty quickly from:

No way I’m giving Google any of my data! I will use 5 different browsers in incognito mode and never log in.

To ->

Sure I will login with my name and email and feed you as much of my most personal thoughts and data as I can dear ChatGPT!

grammers• 3 years ago

Not quite. These are the same people that use only Chrome while being logged in to their Google account. Convenience wins.

smt88• 3 years ago

These are the same people that use only Chrome while being logged in to their Google account

This situation is much dumber than that. ChatGPT is very clear that you shouldn't give it private data and that anything you type into it can/will be used for training.

Google is nowhere near that level of transparency.

contravariant• 3 years ago

Any examples where those two were the same person?

Because both types of people have always existed. Heck, lack of vigilance among the ancient Greeks is what put the Trojan in Trojan horse.

smodo• 3 years ago

It was a lack of vigilance among the Trojans, actually. The Greeks did the burning an pillaging. So uh OpenAI is the Greeks, ChatGPT is the horse and Microsoft is king Menelaos or something. Achilles is dead, but he did make TempleOS.

croes3• 3 years ago

I bet you find some on HN.

Sometimes couriosity beats caution.

philsquared_3• 3 years ago

Google takes your data and sells it. Literally making your data available to the highest bidder. Is OpenAI doing that? If Google existed in its current form during the early internet it would be classified in the same category as Bonzai Buddy. Spyware. That is what Google is. So I can very reasonably understand why people would trust OpenAI with data they wouldn't trust Google with. OpenAI hasn't spit in the face of its users yet.

noncoml• 3 years ago

Google takes your data and sells it. Literally making your data available to the highest bidder.

Even if they are not doing it now(?), what makes you think that they will not do so in the future? It's not like your data has an expiration date.

jankeymeulen• 3 years ago

Not really. Even the most evil Google one can imagine would realise "your data" is the most valuable thing they possess, selling it would be bad for business. They're selling ads to the highest bidder who's looking for someone with a profile based on your data, but not your data itself.

JohnFen9• 3 years ago

I can very reasonably understand why people would trust OpenAI with data they wouldn't trust Google with. OpenAI hasn't spit in the face of its users yet.

In other words, having been burnt once by touching a flame, the conclusion these people draw is that the problem was with that particular flame and they're fine with reaching for a different one?

SketchySeaBeast2• 3 years ago

Google takes your data and sells it. Literally making your data available to the highest bidder.

But it doesn't, does it? It sells the fact that it knows everything about everyone and can get any ad to the perfect people for it. It's not going on the open market and telling people I regularly buy 12 lbs of marshmallow fluff and then use it in videos I keep on my google drive.

dragonwriter2• 3 years ago

Google takes your data and sells it. Literally making your data available to the highest bidder.

No, Google takes money to present ads to people of different demographics, and uses your data to do that. It doesn’t sell your data, which is, in fact their competitive edge in ads – selling your data would be selling the cow when they’d prefer to sell the milk.

bcrosby953• 3 years ago

My problem with Google is they'll ban me from gmail for something I do on youtube.

robbywashere_• 3 years ago

The real innovation of chatgpt will be how you go about getting permission to use it from your corpo middle management overlords.

cloudking2• 3 years ago

ChatGPT Business Edition seems pretty obvious and I'd surprised if OpenAI isn't already working on it. Separate models for each customer, data silos and protection. The infra is already there on Azure.

repler• 3 years ago

It actually is on Azure, exactly as you described.

https://learn.microsoft.com/en-us/azure/cognitive-services/o...

cloudking2• 3 years ago

Yep, they just need to provide a business specific frontend chat UI.

tombert• 3 years ago

This is scary, but it doesn't surprise me even in the slightest. ChatGPT is useful for so many things that it's extremely tempting to convince yourself that you should trust it.

For example, I was having some issues with my LTO-6 drive recently, and I had to finagle through a bunch of arcane server logs to diagnose it. I had the idea of simply copypasting the logs into ChatGPT and having it look at them, and it quickly summarized the logs and told me what things to look for. It didn't directly solve the problem, but it made the logs 100x more digestible and I was able to figure out my problem. It made a problem that probably would have taken 2-3 hours of Googling take about 20 minutes of finagling.

I'm not doing anything terribly interesting or proprietary on my home server, so I didn't really have any reservations sharing dmesg logs with it, but obviously that might not be the case in a company. Server logs can often have a ton of data that could be useful for a competitor (whether it should be there or not), and someone not paying attention to what they're pasting into ChatGPT could easily expose that data.

nemo44x• 3 years ago

Looking forward to AI on chip not too far down the road. As long as we use APIs for the model itself we can't really use it for much.

buzzscale• 3 years ago

This is one of the reasons Databricks created Dolly, a slim LLM that unlocks the magic of ChatGPT. A homegrown LLM that can tap into/query the datasets of all the data in an organizations Data Lakehouse will be hugely powerful.

I am working with customers that are looking to train a homegrown LLM that they host and have blocked access to ChatGPT.

https://www.datanami.com/2023/03/24/databricks-bucks-the-her...

https://news.ycombinator.com/item?id=35288063

ngngngng• 3 years ago

This reads like you had an LLM write an ad for you

wantsanagent• 3 years ago

We saw these same fears with the release of Gmail. Why would you trust your email to Google?!! Aren't they going to train their spam filters on all your data? Aren't they going to sell it, or use it to sell you ads?

Corporations constantly put their most sensitive data in 3rd party tools. The executive in the article was probably copying his company strategy from Google docs.

Yes, there are good reasons for concern, but the power of the tool is simply too great to ignore.

Banning these tools will go the same way as prohibition did in the US, people will simply ignore it until it becomes too absurd to maintain and too profitable to not participate in.

Companies which are able to operate without these fears will move faster, grow more quickly, and ultimately challenge companies restricted to operate without.

Now I think the article should be a wake-up call for OpenAI. Messaging around what is and what is not used for training could be improved. Corporate accounts for Chat with clearer privacy policies would be great and warnings that, yes, LLMs do memorize data and you should treat anything you put into a free product on the web as fair game for someone's training algorithm.

jackson1442• 3 years ago

I think this is different in that ChatGPT is expressly using your data as training in a probabilistic model. This means:

* Their contractors can (and do!) see your chat data to tune the model

* If the model is trained on your confidential data, it may start returning this data to other users (as we've seen with Github Copilot regurgitating licensed software)

* The site even _tells you_ not to put confidential data in for these reasons.

Until OpenAI makes a version that you can stick on a server in your own datacenter, I wouldn't trust it with anything confidential.

hgsgm2• 3 years ago

Google had all the same problems, until it found a balance of functionality, security, and privacy.

OpenAI just hasn't started to try adding privacy and security yet.

renewiltord3• 3 years ago

Not that I don't expect them to do this, but how is it expressly said to be so?

https://help.openai.com/en/articles/5722486-how-your-data-is...

OpenAI does not use data submitted by customers via our API to train OpenAI models or improve OpenAI’s service offering. In order to support the continuous improvement of our models, you can fill out this form to opt-in to share your data with us. Sharing your data with us not only helps our models become more accurate and better at solving your specific problem, it also helps improve their general capabilities and safety.

waboremo3• 3 years ago

Sticking it in your own datacenter doesn't really prevent any of these problems (except maybe #2), only now your leaks are internal and because of all the false sense of security, you might wind up leaking far more confidential and specific information (ie. an executive leaking to the rest of the team in advance that they are planning layoffs for noted reasons, whereas that executive might have used more vague terms when speaking to public chatGPT).

jacquesm2• 3 years ago

I think this is different in that ChatGPT is expressly using your data as training in a probabilistic model.

Google tries hard to sell you on their auto-answers for emails ('smart reply'), wonder how those got trained...

wodenokoto• 3 years ago

Well, you can stick it on Azure.

paxys• 3 years ago

Trusting Gmail with corporate communication was was a terrible idea (and explicitly illegal in a lot of industries), and companies didn't start to adopt it until Google released an enterprise version with table-stakes security features like no training on the data, no ad targeting, auditing, compliance holds and more.

There's a huge difference between trusting a third party service with strict security and data privacy agreements in place vs one that can (legally) do whatever they want with your corporate data.

fnordpiglet2• 3 years ago

Was?

ChatGTP2• 3 years ago

This is vital for professional adoption. We cannot live in a world where basically all commercial information, all secrets are being submitted to one company.

marcosdumay• 3 years ago

Companies which are able to operate without these fears will move faster

Or the fears are real and companies that operate without them will be exploited, or extinguished for annoying their customers.

raincole3• 3 years ago

If your company's code is all repositories on Github (or bitbucket, or any similar service), worrying about ChatGPT is quite silly.

And on the other hand, if your company doens't use Github etc due to security concern, it's a very good sign telling you need to ban ChatGPT too.

MattPalmer10863• 3 years ago

No, it's not silly to worry about it. Many companies store data in third party systems to which they retain control over access. Once you put data into chatGPT what control do you have over it?

Thorentis• 3 years ago

too profitable to not participate in

Sorry, but I really struggle to see how a non AI company will actually become more profitable simply by getting their employees to use ChatGPT. In fact, the more companies that use it, the more demand there will be for "human only" services.

EGreg2• 3 years ago

I recommend you take all your proprietary code and copypaste it to ChatGPT. You can help improve our collective generator.

If you’re an artist, just send all your work to DALL-E. Why have money or fame?

belorn• 3 years ago

If a company/government risk model allows for giving Google all the most sensitive information, then that says something about trust. It has not gone unnoticed how those risk model differs when something like tiktok arrived, or with earlier Huawei 5G modems.

The enterprise version of gmail was just an additional step to instill trust. In practice it is still a decision based on trust rather than physics. An "enterprise version of privacy guaranties" for Huawei 5G modems or tiktok apps would not make governments suddenly happy with the risk model where sensitive data would have a minor risk of ending up in China.

wkat42424• 3 years ago

Google wasn't yet evil when most people adopted Gmail.

And corporations have strict agreements with their providers. They are even required to in many cases due to GDPR and the likes. Users connecting to ChatGPT on their own accounts bypass this.

kevin_thibedeau• 3 years ago

We saw these same fears with the release of Gmail. Why would you trust your email to Google?!!

The original Gmail TOS explicitly stated that they scanned the content. They only stopped for the rollout of Gsuite.

flutas• 3 years ago

IIRC (been a while, so maybe I'm wrong) Google was also the reason Amazon swapped email formats for purchases. They realized they were giving a ton of data to Google through the receipts about products purchased, so now they just give you the vague order emails.

ozim2• 3 years ago

Company I work for uses GMail - but we have business relation with them as we pay for Business licenses that have business data handling in the agreement.

If employee sets random GMail account that is not covered by agreement that is personal account. Sending company data to personal email account might be grounds for firing person.

Setting up some account at random with OpenAI and putting company details like customer names or else there is data breach.

Companies will let people use the tools - but it is not like one can start setting up random accounts without approval from management. Of course there are different types of companies with less or more red-tape.

jcims• 3 years ago

The wild thing is that you can just annotate the content with the associated organization information then ask GPT to tell you what it sees in the logs.

nbzso• 3 years ago

OpenAI. The heist of the century. I am waiting for A.I. generated blockbuster in the near future.

yieldcrv• 3 years ago

I was just watching Altman's interview from the Lex Friedman podcast a few days ago

It really does feel like YC was a plot to fund the harvesting of all with an OpenAI climax. Not a serious conclusion I have, its just funny to watch it unfold, as if nobody even cares about the optics.

Rzor• 3 years ago

First thing that went through my mind as I read the headline was Zuckerberg comments on people posting their info on Facebook.

jedberg• 3 years ago

No one cares about security because there is no consequence for getting it wrong. Look at all the major breaches ever. And look specifically at the stock price of those companies. They took small short term hits at best.

Worst case the CISO gets fired and then they all play musical chairs and end up in new roles.

Heck, even Lastpass, ostensibly a security company, doesn't seem particularly affected by their breach.

My point is, especially with ChatGPT, where it can reasonably 10x your productivity, most people will be willing to take the risk.

faeriechangling2• 3 years ago

I don't even agree to begin with the idea that a tool that vastly increases your productivity is a security risk in the grand scheme of things, since you know, you can just allocate the time you were spending before on writing boilerplate towards securing systems.

Endpoint security software is a security risk too.

MattPalmer10863• 3 years ago

Yikes! Aside from the fact that nobody will take the "spare" time saved and spend it on internal security, once you have a lot of your sensitive data outside in a third party system you have lost control.

interstice• 3 years ago

Could there be eg a browser extension for scrubbing sensitive data from input on paste?

I'm hoping OpenAI will implement something like this on their end soon like data monitoring apps do (sentry etc), but if they don't client-side is an option.

bradgessler• 3 years ago

Apple will make a killing if they can deploy private LLMs to their M-series chips.

reneberlin• 3 years ago

just because your code is art of course, which should be weighted in gold and copyrighted for the next 2000 years living in a cold wallet. Did you get that memo, that humans implemented all the faulty security that lasted the past decades by accident?

balls187• 3 years ago

And what percent of employees send sensitive information via unencrypted email?

deltree7• 3 years ago

Classic fearmongering article targeted to HN crowd.

i) In this world, there are very few people whose private conversation is worth anything to anybody (celebrities, journalists -- so around 10,000 people)

ii) A tiny tiny %age of information is truly secret (mostly private keys).

iii) Business strategies are mostly a result of execution, not any 'trade secrets'. Meta will succeed because it has executed it's metaverse strategy, not because they kept the metaverse strategy secret.

People who take risks and not care about irrelevant details (just like how they took risk with internet shopping, cloud, SaaS) will win. Losers like the ones who thought AWS will steal their data will be left behind

danielmarkbruce• 3 years ago

Seems like a temporary problem. Surely OpenAI will have a version which runs in a customers public cloud VPC, orchestrated by OpenAI.

twblalock• 3 years ago

If you think this is bad, imagine what JSONLint has seen over the years.

WhereIsTheTruth• 3 years ago

That's why privacy is important in the real world

Imagine if everyone knew your inner secrets just by looking at you at the bar..

Why is it different online? i have no idea.. well i kinda know but.. oh well.. we deserve it i guess

phn• 3 years ago

Employees have been feeding sensitive data to online translators in the same way.

Not to say it isn't a problem, but it's not an AI/chatGPT specific one.

VLM• 3 years ago

I believe there were FUD pieces like this when internet search engines were rolled out, and again when social media became popular. I suppose its universal for new technologies.

I had an interview awhile ago at a place where during the phone screen "they can't talk about their tech stack in detail" so I looked on linkedin and figured out their entire tech stack before on the onsite interview. Come on guys, according to linkedin, you have an entire department of people doing AWS with Terraform and Ansible, you don't have to pretend you can't say it in public.

EGreg2• 3 years ago

Why don’t people just copypaste all their proprietary code to ChatGPT lol. And maybe send all their artworks to DALL-E. Then good luck trying to sue them for remixing your stuff

javajosh• 3 years ago

Yep. I've been playing with it only in a Stack Overflow way for this reason. I would certainly not mention my personal proclivities or ask it to generate a new password for me. That might be appropriate for a locally hosted model, depending on how secure you can keep it.

derekhsu• 3 years ago

Good point. And why don't you just unplug your computer and do all your stuff on your paper and pen? I believe it's the most secure way to protect your data. Remember to store these paper in your basement and bring your key.

kjgkjhfkjf• 3 years ago

Are we similarly concerned about people posting company secrets on the Korean website/app "Blind" on a daily basis?

bobse• 3 years ago

All information should be free, chuds.

temporallobe• 3 years ago

This should be treated like any other data breach and employees should be reprimanded accordingly, especially for breaches of classified data (not sure if that ever happened, but I’m sure some dumbass has tried it).

jerkstate• 3 years ago

my understanding of GPT is that the only vector for your data to get "into the model" is if it's used in fine tuning/RLHF. My guess is if you do the thumbs up or thumbs down, the session probably will be, but otherwise probably not. Still wouldn't put in private employer data primarily because of the other exposure risks - it's obviously not stored securely on the OpenAI side. But besides typical IT risk, the big unknown is whether or not the model will spit out what you put into it in somebody else's session. and my understanding is, that's only possible if your conversation is used for RLHF.

I guess another way to say that is, OpenAI (or another service provider with a better security track record) could broker this service in the cloud, with guarantees around not using the session data for RLHF, not storing session data, stronger auth (OpenAI has had a couple of incidents that show that they have pretty lax security in their backend), etc. and could make a killing selling or re-selling ChatGPT to businesses.

waldenyan20• 3 years ago

It seems like an enterprise product is going to have a market here.

jacquesm2• 3 years ago

For fun I once just made a blank from with a submission button and a giant text field.

It was quite amazing what people would submit unprompted, so I'm not at all surprised that people would feed sensitive data into ChatGPT. The next cycle will be that ChatGPT gets - surprise - trained on that data, and may start using fragments of it - which may well still be sensitive enough to cause trouble - as its output.

Don't paste confidential information into a textbox, in fact don't trust anybody or any company with your confidential information unless there is a strong contractual relationship backed up by penalties if it gets broken. And even then: the ultimate responsibility is yours, you may be able to recover some $ for damages but your reputation may well be toast.

ryanSrich• 3 years ago

What’s not clear from OpenAI is what they actually store.

Clearly they store information from ChatGPT, but if I hit their chat/completions API with the same exact question do they store that as well?

danjc• 3 years ago

You know what's hilarious about this - they quote specific examples of information that was prevented from being sent to ChatGPT. Someone at Cyberhaven is reviewing the requests it blocked.

patal2• 3 years ago

How do you feed sensitive data to ChatGPT? I've got no account, therefore no idea. I thought you just ask it questions, or task it to write some code. How does the sensitive data get there?

maigret• 3 years ago

- Translate this upcoming announcement in Spanish

- Can you summarize that product roadmap?

- Can you add comment to this piece of (proprietary) code?

- Please extract the birth dates from this employee list

I could go on and on and on

patal2• 3 years ago

I see. Thanks.

larsonnn• 3 years ago

I bet they post sensitive data to all platform which may help them.

Also I guess chat with Microsoft teams or slack is ok.

nareshorange• 3 years ago

Test

Employees are feeding sensitive data to ChatGPT, raising security fears

Related Stories