Claim: Private GitHub repos included in AI dataset

https://post.lurk.org/@emenel/112111014479288871

by latexr • 2 years ago

193 161 2 years ago

post.lurk.org

latexr10• 2 years ago

Full text from the post:

If you had code on GitHub at any point it looks like it might be included in a large dataset called “The Stack” — If you want your code removed from this massive “ai” training data go here:
https://huggingface.co/spaces/bigcode/in-the-stack
I found two of my old Github repos in there. Both were deleted last year and both were private. This is a serious breach of trust by Github and @huggingface.
Remove all your code from Github.
CONSENT IS NOT OPT-OUT.

samtheprogram• 2 years ago

Thought it was bait, but I can confirm I can find private repos in the search results. What the heck?

tailspin20193• 2 years ago

Were those repos always private?

simonw10• 2 years ago

Which private repos? Any that you're willing to share (I get that sharing names of private repos goes against the whole idea of them being private!)

rfoo3• 2 years ago

Since your repo's name is public now anyway, could you please help us and post it here? I'm really curious about what happened. Since public GitHub activities were archived [1], if you post it here we can check if it were ever public or it's truly private at all time.

[1] https://www.gharchive.org/

tailspin20193• 2 years ago

"Both were deleted last year and both were private"

I'd really like to know if they were ever public at any point because that might explain it.

None of my repos that have always been private are included (apparently).

That's not to say I'm not concerned by this...

kevincrane• 2 years ago

god the language in that opt-out link is patronizing as hell. do AI people just assume everyone is happy to be a cog in their monetization scheme?

CatWChainsaw• 2 years ago

They assume that they don't need to care and their assumption has so far proven correct.

codazoda• 2 years ago

This is interesting...

I have 66 repositories picked up and put in The Stack. I spot-checked the first 10. All 10 are Public on GitHub. 8 of the 10 do not have a license of any type, meaning they are covered by copyright, at least in the US, unless GitHub has some terms extending the license of public projects to 3rd parties.

One of mine is Private and was an extension I sold for a short time. I can't say if I ever made it public or not.

https://github.com/codazoda/like_roller

pluc• 2 years ago

Could "private" mean "kept from the outside world"?

tailspin20193• 2 years ago

From one of the comments on that post:

I work for github. This post was shared in our slack, on a non work related channel. We don't think it's us.
They say they get data from SoftwareHeritage, a website that archives repos from github. If your repos were ever open, they might have been archived there even after you deleted from github.
That's my best guess.

latexr10• 2 years ago

The OP responded to that:

https://hachyderm.io/@emenel@post.lurk.org/11212861313743638...

thanks for the additional info. I’m pretty sure the repos of mine were private. If it were only me i could be misremembering something, but i have heard from a number of people that they also found repos in this dataset that were private. So i’m not sure what to think or how to explain it.

simonw10• 2 years ago

This is talking about The Stack. The poster says:

"I found two of my old Github repos in there. Both were deleted last year and both were private."

The Stack was constructed a while ago, so "deleted last year" wouldn't have an impact if it was constructed before then.

"Both were private" is the thing that needs to be unpacked here. Were these genuinely private repositories that had never been made public on GitHub?

https://huggingface.co/datasets/bigcode/the-stack-v2 talks about where the Stack comes from: "This dataset is derived from the Software Heritage archive, the largest public archive of software source code and accompanying development history"

You can search that here: https://archive.softwareheritage.org/browse/search/?q=simonw... - it would be interesting to know if the OP's "private" repos are included in that collection.

errantmind• 2 years ago

For info, I checked and it looks like none of my AGPL licensed repos are included in The Stack. Neither are my private repos.

tom_• 2 years ago

My GPL ones seems to be excluded.

I've got a couple that are intended to be GPL, but you wouldn't know unless you go to the GitHub issues to find the issue I raised about the licence file not being in the repo. They are included.

latexr10• 2 years ago

Were these genuinely private repositories that had never been made public on GitHub?

The poster responded to that:

https://hachyderm.io/@emenel@post.lurk.org/11212861313743638...

I’m pretty sure the repos of mine were private. If it were only me i could be misremembering something, but i have heard from a number of people that they also found repos in this dataset that were private.

simonw10• 2 years ago

This is a really big claim. I think we need specifics on this - I'm inclined to think this is people not understanding the GitHub public/private repo model (or misremembering the history of their repos) over GitHub deliberately leaking private code to third parties.

chefandy2• 2 years ago

I'd be interested in hearing from the cited number of people. If it's like 5 randos, then that's quite possibly a misremembering, or even conceivably the victims of some unrelated code theft. If it's a few dozen people, well that would have very different implications.

RandallBrown• 2 years ago

None of my private repos are included in the software heritage project.

cavisne• 2 years ago

Considering private repos used to require a paid subscription misremembering seems likely.

It used to be public by default, and enough people got confused by this that AWS & Github used to specifically scan repo's for accidentally public AWS credentials.

simonw10• 2 years ago

Here's a reply from a GitHub staff member: https://mastodon.social/@correcthorse/112128192392083842

I work for github. This post was shared in our slack, on a non work related channel. We don't think it's us.
https://huggingface.co/datasets/bigcode/the-stack-v2
They say they get data from SoftwareHeritage, a website that archives repos from github. If your repos were ever open, they might have been archived there even after you deleted from github.
That's my best guess.

beeboobaa• 2 years ago

Depending on the license, how is this legal?

calvinmorrison7• 2 years ago

The software heritage sent me an email they were going to steal copies of my software. I think they assumed because it was published as a git repo they had unlimited rights to steal, reproduce, sell, etc my copyrighted work.

To quote the email they sent me:

"The mission of Software Heritage is to collect, preserve and share all the publicly available source code: https://www.softwareheritage.org"

So they're telling me their intent is to reproduce (share) my work, just because it was publicly available.

Upon my reply they did offer to cancel the request, but also told me they are facilitating the storage of my code for users private theft

" "Add forge now" requests are submitted by Software Heritage users who think that a forge is worth being archived.

After a careful examination of your arguments, we acknowledge that your forges may not be archived, so we won't process their ingestion, and close this add forge now request. However, we cannot prevent a publicly available from being archived by anyone using our "Save code now" feature

remotefonts4• 2 years ago

Steal your repos? So you don't have them anymore because they took them? Maybe ask nicely to have them back and with a bit of luck they'll give them back to you.

buffington2• 2 years ago

Do you happen to still have that email? I'd like to search my inbox for anything similar, and would love some snippets to search for.

Edit: I asked this before the parent commenter included the contents of the email. Thanks parent commenter!

AshamedCaptain5• 2 years ago

Have you figured out if there is a way to prevent them from doing that? The email I got, which I assume is similar, was suspiciously lacking a "just GTFO and leave my website alone" option.

ryan-c5• 2 years ago

However, we cannot prevent a publicly available from being archived by anyone using our "Save code now" feature

This is not true. They just had to remove about 500 public repos to comply with my copyright.

buffington2• 2 years ago

I have about fifty private github repos and hundreds more public github repos (half of them forks of other public repos). I've verified that none of my private repos are in the dataset, and all of my public repos are.

I was expecting to see at least one repo that shouldn't be there depending on when the dataset was put together. In 2015 I changed a repo from public to private, which I think might suggest that the dataset was built after 2015 since my now private repo isn't in the dataset?

godelski• 2 years ago

I did the same and see none of my private repos. Which is quite a lot. I do see plenty of deleted repos but those were public.

hn72774• 2 years ago

Github publicly streams the global change log including all public repos. I have a public repo and noticed there were multiple clones daily after every commit I made. That's when I discovered the API: https://docs.github.com/en/rest/activity/events?apiVersion=2...

If the repo was public even for a single commit, that was likely cloned and replicated elsewhere.

sdflhasjd• 2 years ago

Hmm, I do see some of my repos there, but only public ones. It would seem a bit too foolish of GitHub to put private repositories in a public dataset.

Is there any other corroboration or proof?

jaggederest• 2 years ago

It would be fun to upload a private repo with AWS keys in it as a canary. If the account ever gets used, the repo is no longer private?

ergonaught• 2 years ago

As with others, I can say they only seem to have my repos that were ever public. Nothing of mine that was always private is showing.

jddj• 2 years ago

Same here, public only

derriz• 2 years ago

Yep - same. My public repos show but not my private one.

roland35• 2 years ago

It is always interesting to see how copilot autocompletes things like "Todo: " or "my ssh key" and see what data leaks through!

throwitaway222• 2 years ago

I mean, there is also a lot of "public" github repos that contain absolutely copywritten work too.

nromiun• 2 years ago

I also see a 7 years old private repo from my account. But I may have made it public initially, I don't really remember it. Anyone knows how to check if a repo was ever made public?

rfoo3• 2 years ago

Check if it was ever appeared in gharchive [1].

[1] https://www.gharchive.org/

netule• 2 years ago

The Wayback Machine [0] if you're lucky (?) enough to have had it indexed.

[0] https://web.archive.org/

abdullahkhalids• 2 years ago

If the repo was public for a while (say 2 years), and then became private, then the code in the Stack for your repo must be 5 years old.

If on the other hand, it's tracked to the latest commit, that is a different scenario.

nyc_data_geek• 2 years ago

I am Jack's complete lack of surprise.

In all seriousness, without strong data privacy regulations (ie: GDPR), we will continue to see this sort of stuff, as the potential monetary rewards for using this sort of data far outweigh the potential liability. Cost of doing business type stuff, rather than an existential risk for abusing public trust.

My opinion is that data should be treated like a fissile element - very dangerous to hold and store, but extremely powerful when properly employed. However, it's only dangerous if the liability of storing it is significant and real, as of today, it's not (in the US).

psuedo_uuh• 2 years ago

It blows my mind that we’re all fine with “the home of open source software” is closed source

bogwog• 2 years ago

The way I rationalize my use of Github to myself is by framing it as me stealing free compute and storage from the evil Microsoft.

HaZeust• 2 years ago

Trust me, they're making it back.

mdaniel4• 2 years ago

especially with the number of actual FOSS alternatives available right now, but that network effect, whew, it's strong :-(

mnau3• 2 years ago

I am considering moving away because of network effect. The more popular repo is, the more work is there (issues, PRs...). It's not scalable.

Using another platform/self-host would introduce a friction.

bmitc3• 2 years ago

What FOSS solutions give me an editor in the web, Codespaces, free CI/CD compute, free website hosting, etc.?

jmuguy• 2 years ago

Need more proof than "pretty sure they were private" and "heard from a number of people".

pityJuke• 2 years ago

This organization claims that they'll allow you to opt-out, but they haven't seemingly done so for the oldest request on their repository [0]. Pathetic.

[0] https://github.com/bigcode-project/opt-out-v2/issues/1

misterpigs• 2 years ago

This is a nothingburger.

I have "private" repos listed in the dataset but they were all at one time public. Searching the SoftwareHeritage site I can find those once-public repos with ancient commits.

My private repos that were always private are not listed in the dataset.

Retr0id3• 2 years ago

47 of my repos are in the stack, and none of them are private (and I have plenty of private repos)

multimoon• 2 years ago

Same here. I think some people are confusing private now vs private since creation.

Retr0id3• 2 years ago

I do think, however, that "repo no longer public" should be treated as an opt-out signal.

Mtinie2• 2 years ago

Which is valid, but what's the timeframe for rechecking and validating if there's been a change? I'll posit it's practically impossible to catch every single change within a reasonable timeframe to ensure that "a repo no longer public, today, within a data set collected earlier" would be excluded.

I may be misunderstanding your suggestion. If so, I'm curious to learn what I missed.

inopinatus• 2 years ago

None of mine are in the stack and I am now personally offended that all those crappy little one-file Swift and sh utilities will be excluded from the singularity

johncoatesdev4• 2 years ago

From the reports it looks like they have to have been private and deleted

caesil• 2 years ago

All of the comments here where the commenter found their repos say either "it only has the public ones" or "it has some private ones from x years ago and i don't quite remember if they were ever public".

So it seems probable this is a case of repo owners misremembering.

jtietema• 2 years ago

I agree. Only my public repos are in the data set.

angst_ridden3• 2 years ago

I'm curious how they get around licenses. For example, I have repos that show up in The Stack. Some have licenses that require inclusion of the copyright in any source re-use or redistribution.

IANAL, but it seems like inclusion in the data set and subsequent distribution without the copyright notice would be a violation.

xyst2• 2 years ago

Probably some obscure GH legal clause stating “we own your data. Ownership is implied and we may do anything with it. Private vs public is a concept of accessibility over the internet. Not necessarily means it’s not accessible via intranet or other non-public means”

It’s the similar legal clauses used for decades on social and video hosting platforms.

angst_ridden3• 2 years ago

I don't think that gets around the code licenses. They may use it as if it does, but I'm not convinced that would hold up if soemone were wealthy enough to make a court case.

ADeerAppeared• 2 years ago

GH interestingly doesn't grant themselves nor others that many rights.

---

  You grant us and our legal successors the right to store, archive, parse, and display Your Content, and make incidental copies, as necessary to provide the Service, including improving the Service over time. This license includes the right to do things like copy it to our database and make backups; show it to you and other users; parse it into a search index or otherwise analyze it on our servers; share it with other users; and perform it, in case Your Content is something like music or video.

  This license does not grant GitHub the right to sell Your Content. It also does not grant GitHub the right to otherwise distribute or use Your Content outside of our provision of the Service, except that as part of the right to archive Your Content, GitHub may permit our partners to store and archive Your Content in public repositories in connection with the GitHub Arctic Code Vault and GitHub Archive Program.

---

  Any User-Generated Content you post publicly, including issues, comments, and contributions to other Users' repositories, may be viewed by others. By setting your repositories to be viewed publicly, you agree to allow others to view and "fork" your repositories (this means that others may make their own copies of Content from your repositories in repositories they control).

  If you set your pages and repositories to be viewed publicly, you grant each User of GitHub a nonexclusive, worldwide license to use, display, and perform Your Content through the GitHub Service and to reproduce Your Content solely on GitHub as permitted through GitHub's functionality (for example, through forking).

---

They don't even include training for copilot (though a dodgy lawyer will likely try to include that as "part of providing the service). 3rd parties only get a license to fork your repo, seemingly not even a license to do anything with that repo. (And hot take: Github should just let people disable the fork button already.)

AshamedCaptain5• 2 years ago

In The Stack FAQ, they claim that they are doing minimal analysis of the LICENSE file and SPDX tags.

I'd bet that this is enough to detect cases like GPL code, but I also bet that if this analysis fails instead of falling back to "unknown license, assume proprietary, don't copy" they fall back to "free lunch!". Because reasons.

angst_ridden3• 2 years ago

I suspect you're right.

Even though even permissive licenses like MIT and BSD require attribution and preservation of copyright notices. Maybe their AI just can't "reliably detect" licenses.

ryan-c5• 2 years ago

xyst2• 2 years ago

The crypto craze stole our energy to validate digital currency transactions on the blockchain.

Now the AI craze stole our code and collective knowledge to ultimately train their models, and hopefully replace SWEs and other knowledge based fields (ie, medicine).

At least with blockchain, some people got rich. But with the AI craze the only people getting rich are the rich themselves.

latexr10• 2 years ago

At least with blockchain, some people got rich.

But plenty more lost money or went bankrupt. The one’s who got rich did so at the expense of other non-rich people.

remotefonts4• 2 years ago

But plenty more lost money or went bankrupt. The one’s who got rich did so at the expense of other non-rich people.

[citation needed]

simonw10• 2 years ago

Since a lot of this depends on whether someone had ever had a private repo public in the past, I was hoping it could be resolved using the GitHub security audit log.

You can access that for your repos here: https://github.com/settings/security-log

Then search for repo:simonw/datasette or similar.

But... it looks like the audit log only goes back 6 months, so sadly it's not useful for reviewing this particular situation which involves repos that could have been 5 or more years old.

The ClickHouse copy of the GitHub Archive is useful for reviewing things and goes back a lot further. Try it here:

https://play.clickhouse.com/play?user=play

You can run this query to see relevant events for a specific username:

    with public_events as (
      select
        created_at as timestamp,
        'Private repo made public' as action,
        repo_name
      from github_events 
      where actor_login = 'simonw'
      and event_type in ('PublicEvent')
    ),
    most_recent_public_push as (
      select
        max(created_at) as timestamp,
        'Most recent public push' as action,
        repo_name
      from github_events
      where event_type = 'PushEvent'
      and actor_login = 'simonw'
      group by repo_name
    ),
    combined as (
      select * from public_events
      union all select * from most_recent_public_push
    )
    select * from combined order by timestamp

The PublicEvent one is "When a private repository is made public" according to https://docs.github.com/en/rest/using-the-rest-api/github-ev...

I just built a tool for running this query without having to type in the SQL: https://observablehq.com/@simonw/github-public-repo-history

Explained in this TIL: https://til.simonwillison.net/clickhouse/github-public-histo...

drpossum• 2 years ago

I'm guessing these were public once (mine were), but I'll be damned if I let github host anything of mine again.

latexr10• 2 years ago

https://archive.is/nWzk4

mtam• 2 years ago

Just to add yet another data point. Only my Public repos included. All my private repos are not included.

falsandtru• 2 years ago

Is there any other good service for enterprise development?

zzo38computer2• 2 years ago

I do not use private GitHub repositories. (If I want something private, I will store it on my own computer and/or on DVDs, etc.)

If they are using private data in AI dataset (or other uses) then that is a serious issue; they are copying data which is meant to be private. (Public files are public and should be made copies that others can use too, though.)

t1c• 2 years ago

Even though they were only my public repos in there, I really don't like this being opt out instead of opt in, especially since some of those repositories were not under fully permissive licenses

AshamedCaptain5• 2 years ago

So let me understand this.

Step 1) "Software Heritage" crawls the web (not only Github; they definitely crawl ANY gitlab instances they find online among practically everything else). They "store" ANY type of source code, irregardless of the license of that code. They admit as much in their own FAQ ( https://www.softwareheritage.org/faq/#24_What_is_the_policy_... ) :

Software Heritage archives everything that is publicly available, without preliminary tests or checks [for LICENSE file or others]. You are responsible for checking whether the source code you find in the archive can be reused, and under which terms.

Step 2) "Hugging Face" uses the Software Heritage dataset to build some AI training dataset ("The Stack"), again, completely ignoring licensing. You apparently have to manually opt-out if you don't want your COPYRIGHTED source code to be included there. But as far as I can see, opting out is only considered AT ALL for Github repositories via https://huggingface.co/datasets/bigcode/the-stack-v2 . If you have your source code published outside Github, then your code appears to be used, period.

The Stack v2 is a collection of source code from repositories with various licenses. Any use of all or part of the code gathered in The Stack v2 must abide by the terms of the original licenses [...]

Step 3) Eventually and inevitably someone releases some model trained on this dataset, ignoring licensing again.

Step 4) Someone uses code generated from such chatbot, unknowingly violating everything software license known to man.

Step 5) ???

Step 6) Profit!

Where do these "Software Heritage" guys mention which user-agent their crawler is using so that I can permanently ban them from my websites?

At least archive.org does that and they have a much nicer way to request exclusion from their archive.

a_wild_dandan2• 2 years ago

You lost me at step four. If reading source is fair use and generating content isn't derivative work, this is kosher. As litigation mounts, we'll learn more.

To me, a learning path that's acceptable for humans is acceptable for AIs. All of the rationales for treating AI/human learning distinctly feel flimsy and arbitrary. "The difference is learning scale." So what? "LLMs just parrot derivatives of their training data." No, they don't. That's demonstrably false, and theoretically absurd, for all the common cited reasons. "AIs obviously aren't people. So their learning restrictions must be different." Why? "People are financially benefiting from the AI's knowledge!" Yep, that's generally how employment works. Etc, etc.

My take could be totally wrong. I get that. It'll be interesting to see where the courts fall on the issue. But personally, the critics' arguments feel profoundly crummy and counterproductive; applying their logic to pre-LLM systems usually results in horrendous alternative realities. But that's just, like, my opinion, man.

AshamedCaptain5• 2 years ago

Having an agenda to push much?

Anyway, this is strictly one level up above that debate, as here they are scraping source code which is NOT free, and whose licenses could explicitly forbid copy for use in training datasets, or by the military, or even by people whose eye color I don't like.

For example, example code for proprietary tools which usually allows you to only copy it strictly for purposes of extending the proprietary tool.

gavinhoward2• 2 years ago

I had this problem with Software Heritage for non-Open Source stuff in my personal Gitea.

I eventually had to specifically block their IP addresses from accessing my Gitea and threaten legal action before they took the offending repos down.

I still have them blocked.

matt32102• 2 years ago

I and most of my coworkers have all found at least one private repo which we believe was always private to be in the data set version 2.

matt32102• 2 years ago

Opt-out consent is the equivalent of assuming a partner consents because they haven't said anything.

Consent is not Opt-out == Yes means Yes

max_• 2 years ago

Time for an end to end encrypted Git Service.

LightFog• 2 years ago

Time to start large scale poisoning of repos.

tamimio• 2 years ago

Encrypt your private repos..

Claim: Private GitHub repos included in AI dataset

Related Stories