Tell HN: GitHub will delete your private repo if you lose access to the original

https://news.ycombinator.com/item?id=34600985

by baobabKoodaa • 3 years ago

530 270 3 years ago

news.ycombinator.com

I was surprised to see this in my email today:

Your private repository baobabKoodaa/laaketutka-scripts (forked from futurice/how-to-get-healthy) has been deleted because you are no longer a collaborator on futurice/how-to-get-healthy.

That was an MIT-licensed open source project I worked on years ago. We published the source code for everyone to use, so I certainly did not expect to lose access to it just because someone at my previous company has been doing spring cleaning at GitHub! I had a 100% legal fork of the project, and now it's gone... why?

Turns out I don't even have a local copy of it anymore, so this actually caused me data loss. I'm fine with losing access to this particular codebase, I'm not using HN as customer support to regain access. I just wanted everyone to be aware that GitHub does this.

Alex3917• 3 years ago

Thanks for the heads up, this is pretty nuts.

mattl• 3 years ago

Your fork wasn't public?

baobabKoodaa8• 3 years ago

Correct.

sbierwagen• 3 years ago

Is futurice/how-to-get-healthy a public repo? It's not visible on https://github.com/orgs/futurice/repositories

If you fork a private repo and they remove you from the collaborators list, it's reasonable that your fork would be removed.

baobabKoodaa8• 3 years ago

Well, it's not reasonable to me, but we'll just have to respectfully disagree on this.

IncRnd6• 3 years ago

It probably seems more reasonable to the people who own the repo.

glerk• 3 years ago

The most “expected behavior” here would be to remove the fork relationship between the two repos and leave the copied repo as a plain private repo. I don’t think it is reasonable to just delete a private repo by default.

csteinbe• 3 years ago

This is documented at: https://docs.github.com/en/pull-requests/collaborating-with-...

fragmede• 3 years ago

“But the plans were on display…”

“On display? I eventually had to go down to the cellar to find them.”

“That’s the display department.”

“With a flashlight.”

“Ah, well, the lights had probably gone.”

“So had the stairs.”

“But look, you found the notice, didn’t you?”

“Yes,” said Arthur, “yes I did. It was on display in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying ‘Beware of the Leopard.”

― Douglas Adams, The Hitchhiker's Guide to the Galaxy

agolio• 3 years ago

To clarify the appropriateness of this analogy:

This is unexpected behaviour from Github here which may (and has, by the anecdote of OP) cause permanent data loss. Documentation is not good enough, as users should not have been expected to have read the entire documentation.

darekkay2• 3 years ago

That's why I usually don't use the official "fork" feature, but clone and push the repository manually instead. I would like to keep the fork network connection on Github, but I don't want to see my fork deleted because of an error, malice or simply lack of knowledge.

rezonant5• 3 years ago

It will only be deleted if the repo you fork from is a private repository. The documentation [1] covers the other scenarios, in all of which you keep your copy of the code (including when the public repository is made private later).

[1] https://docs.github.com/en/pull-requests/collaborating-with-...

devmunchies• 3 years ago

Same. Another reason is I don't like how Github inserts "Forked from ..." in the project name. If your "fork" becomes extremely divergent after a couple years (maybe you had a different vision for the project), you are still stuck with the "Fork from..." sub-header, which basically tells users that they should look at the original. I'm otherwise fine putting attribution in a README.

tux3• 3 years ago

I think this is under the assumption of "employee has a private fork of the company repo, then leaves, employee should not keep the fork"

So when "removed as a collaborator" which apparently includes the original being deleted, you lose access to the main repo and all forks, even yours. As if leaving a company.

chrisfosterelli7• 3 years ago

Yeah I understand how this is unfortunate but for 99% of cases this is what you want to happen when you remove someone's access to a private codebase.

It seems like this is an unusual use case that OP had an MIT-licensed open source project whose source only exists in a private repo.

cma• 3 years ago

Is this scenario possible?

- you fork a public repo

- it's visibility is changed to private but you have access through the org

- they delete it

- you lose your fork, which was only ever of the public repo

nly• 3 years ago

What's the point? You can always clone locally and push to a new repo without forking on GitHub. Repos will be unlinked and history will be intact.

chrisfosterelli7• 3 years ago

It's for the same reason that you don't leave ex-employees with access to other intellectual property. Just because they could make their own copies during their tenure doesn't mean you shouldn't try to reduce your exposure.

charcircuit2• 3 years ago

Github knows the license of projects. It knows if you forked an open source project.

jzb2• 3 years ago

It knows the license applied to the repository. For a private repo it may not be "released" under that license but planned for release.

If AcmeCorp is planning to release - but hasn't - a project under MIT or whatever, they may have the license declared in the repo but that's not a guarantee it's ever going to be released.

If it's a private repo, and your access has been under your status as an employee, then I don't know that counts as distributed to you under that license. If AcmeCorp later decides to change licenses or not release the software as open source, then it makes sense for GitHub not to let someone continue access.

There are a LOT of holes in the system, but I'm not sure GitHub is in the wrong for deleting access to a private repo if you lose access to an organization or whatever.

jiggawatts• 3 years ago

Microsoft does this a lot, where they assume that rules that apply to their own organisations apply to all organisations in precisely the same way.

If a Microsoft IC leaves, they should lose access to all Git repos, including forks.

If Joe Random open-source contributor is removed from an open source repo's access list, their fork shouldn't be wiped.

But Microsoft has One Rule To Rule Them All, so they won't make exceptions for unimportant people like their customers.

I see this a lot. A good example is Azure Active Directory, which is basically "Microsoft 365 Authentication" that they rebranded and sold to developers for their own use, i.e.: Azure AD Enterprise Apps, App Registrations, and B2C.

There are many aspects of the AAD design that make zero sense until you pause for a second and realise that it is not designed for you. It's designed for Microsoft 365!

For example, auditing. My customers are typically government agencies or banks, and they have strict auditing requirements, especially related to data access. All user authentication MUST be logged, including client IP address, and everything else. Most access is by their own staff, or by other orgs that have signed various contracts or agreements, so there is no expectation of privacy.

This is basically impossible with many configurations of AAD. It just refuses to collect meaningful audit logs. Why? Because GDPR applies to Microsoft 365 and they don't care about the data hosted on services such as SharePoint Online. That's not Microsoft's data, that's their customers' data, so its up to the customers to enable logging "on their end", in their individual AAD tenants.

There is no way to centrally collect logs as a service provider using AAD in a multi-tenant scenario.

When I asked Microsoft about this, they waffled on about GDPR and privacy regulations -- which apply to them, but not us.

Another example is Microsoft Teams, which hides the name of the organisation people are coming from. In large multi-org meetings this is infuriating, because you have no idea where anyone is from. Microsoft does this because they use outsourcers like MindTree for support, and they don't want their customers to see this in Teams meetings for Azure support tickets. No-one is allowed to see where people are from so that Microsoft can bullshit their customers.

donmcronald3• 3 years ago

There are many aspects of the AAD design that make zero sense until you pause for a second and realise that it is not designed for you. It's designed for Microsoft 365!

Business Basic accounts being limited to 7 days of login logs is a huge middle finger to the entire small business sector. Of course they think everyone should just buy Enterprise subscriptions. It's nothing more than a corporate version of "don't be poor".

djur11• 3 years ago

This behavior was not introduced to GitHub by Microsoft.

evouga• 3 years ago

This doesn’t make any sense to me.

My private forks are *mine* and I most certainly do not want GitHub guessing at whether and when to permanently delete them without my consent.

Companies of course have the right to manage access to their proprietary source code, for example by only giving access to corporate accounts under their control and reclaiming those accounts when an employee leaves.

naikrovek4• 3 years ago

My private forks are mine

Not if you create them using the "Fork" button in the UI.

Since this behavior has yet again surprised many people, here is the documentation: https://docs.github.com/en/pull-requests/collaborating-with-...

mike_d• 3 years ago

My private forks are mine

Your employment agreement disagrees. Blame the confusion on the blurry line GitHub draws between forking work repos into personal accounts.

Marsymars2• 3 years ago

Companies of course have the right to manage access to their proprietary source code, for example by only giving access to corporate accounts under their control and reclaiming those accounts when an employee leaves.

This is how it should be done, but is too much overhead for many "IT as a cost centre" companies.

vetrom• 3 years ago

Any (quite relevant actually) discussion of SCM policy and management aside, you've got to remember: There is no 'cloud', it's just somebody else's computer.

madduci• 3 years ago

So the main solution would be not forking, but cloning and straight create a separate project? Will it work?

comprev• 3 years ago

My client recently moved from GitLab (self hosted - many teams had their own isolated server) to GitHub.com and managing access for the thousands of developers has been a small headache. We were encouraged to use our personal GitHub accounts instead of making new ones.

They are promoting "internal open source", yet due to a wild variety of permissions, colleagues can't fork to their own space or push a branch for a PR. Chasing the repo owners or at least someone with authority to grant permission is rarely worth the hassle.

irjustin• 3 years ago

It really sounds like the forking and access should be "license aware" which sounds like an absolute nightmare to manage but that would help in this specific scenario?

I can see it simply creating more worms than helpful.

funcDropShadow2• 3 years ago

It really sounds like users of Github shouldn't trust Github to keep their forked repositories for them.

voytec• 3 years ago

OP states that it was "an MIT-licensed open source project". It appears that Microsoft has lack of licensing understanding not only when it comes to copilot.

If they are punishing users just under an assumption that user is doing what they are doing (using someone's codebase with non-permissive licence, possibly unlawfully), it's just silly.

b1nj0y• 3 years ago

agree

Dork1234• 3 years ago

Does it go into GitHub Desktop and remove it from your computer as well?

amelius2• 3 years ago

Why a mail saying that something was deleted instead of a warning that something will be deleted??

shagie9• 3 years ago

If you worked at a company and had a fork of a private repo that company maintained of some software... do you want employees who have left to have their access to the repos removed? or an email that says that it will be removed?

amelius2• 3 years ago

I'd prefer it if the repo was returned to the company, rather than removed.

kitkat_new15• 3 years ago

one of many scenarios in which it is probably(!) desirable to revoke access.

What about other scenarios?

xigoi• 3 years ago

What prevents them from having the code locally?

dpkirchner• 3 years ago

had a fork of a private repo

This is the real anti-feature. You should only be able to fork a private repo using an account that is directly managed by the organization that owns the repo. That way when you revoke access to the user they automatically lose access to the fork.

It's super weird that it's common to use the same account for work and non-work stuff on GitHub (myself included).

zxcvbn40382• 3 years ago

I have a number of git repos that the original developers deleted - because I sync’d them to a usb stick with gitea. I think that is how you have to do it - never entrust a service, especially a free one, with your only copy of anything you value.

If the YouTube algorithm nukes your account and all your videos, you should be ready to upload them to a new account. Same with anything else digital.

My current is standard is one copy in AWS S3 which is super reliable but too pricy for daily use, and one copy in Cloudflare R2 or Backblaze B2 which might or might not be reliable (time will tell) but is hella cheap for daily use.

no_butterscotch• 3 years ago

If the YouTube algorithm nukes your account and all your videos, you should be ready to upload them to a new account. Same with anything else digital.

Do you know if this is a common occurrence?

Also, I'm only a YouTube viewer and am not familiar with all the creator tools, problems, communities, etc. But would a creator really re-upload all their back-catalog if deleted? Just to try to get back to views and things?

ChrisMarshallNY• 3 years ago

I remember someone posting an agonized screed, some time ago, about YT deleting their channel, and all the videos.

Apparently, they had not kept the source/rendered originals of the videos, so it actually clobbered their business.

I am a scarred, limping old coot, and have learned [the hard way] that backups are goooood.

tpxl• 3 years ago

A few weeks ago, youtube changed their swearword policy. A creator I follow basically had to delete half their channel or risk termination.

MonkeyMalarky• 3 years ago

There were a whole bunch of artist and genre specific mixes I used to listen to on yt that are gone now. The uploaders accounts have all been nuked too. The sad thing is I can listen to it all on Spotify but it's not the same.. the creator's did not insignificant work to mix the songs together.

chipsa• 3 years ago

Maybe not on YouTube, but the gunTubers are having issues with YouTube changing their interpretation of the rules and instantly issuing 3 strikes against them for rule violations. And so, it'd be good to have a back catalog to upload to a different service to keep that older material available.

sidewndr46• 3 years ago

I'm reasonably certain that if YouTube deletes your account uploading them to a new account is expressly forbidden.

Gordonjcp2• 3 years ago

How exactly can they stop you?

j452• 3 years ago

I’ve read having a secondary test account to post videos under to pass YouTube scans before posting to the real account helps minimize issues.

wjdp• 3 years ago

For this I have a NAS with a pretty basic script that runs nightly to clone any new repos I have and update those already backed up. They get organised into a directory structure mirroring that of Github: `./github.com/user/repo`

If of any use to anyone else: https://gist.github.com/wjdp/a20cb15f76b651124b3b27cde06d121...

capableweb7• 3 years ago

because I sync’d them to a usb stick with gitea

Just a tip: no need to use gitea if you want to replicate a git repository to somewhere else on disk/other disk.

Just do something like this:

    mkdir /media/run/usb-drive/my-backup-repo
    (cd /media/run/usb-drive/my-backup-repo && git init)
    git remote add backup /media/run/usb-drive/my-backup-repo
    git push backup master

And now you have a new repository at /media/run/usb-drive/my-backup-repo with a master branch :) It's just a normal git repository, that you also can push to over just the filesystem

benatkin2• 3 years ago

Have you tried running gitea? It's very light on resources, has good documentation, and also defualts to a main branch. It's also very easy to control where all the data is stored, and works well w/ sqlite.

josephg2• 3 years ago

Even better with

    cd /media/run/usb-drive/my-backup-repo && git init --bare

Bare repositories don't have a working directory. You can still git clone / git pull from them to get the contents. You can also git push to them without clobbering any "local changes" (there aren't any).

More detail here:

https://www.atlassian.com/git/tutorials/setting-up-a-reposit...

dabber• 3 years ago

You can even take it a step further and have a push to origin update your backup as well.

https://stackoverflow.com/a/14290145

spartanatreyu• 3 years ago

I did not realise you could set the remote's URL to a local filesystem path.

That opens some interesting possibilities.

Cheers

sangnoir3• 3 years ago

Gitea has a cron task that pulls-in changes on an ongoing basis.

If a snapshot suffices, a once off "git push" or "git clone" works (but that's not too far off from downloading a tar ball, is it?). If you want to have a up-to-date local copies of multiple repos, a SQLite-backed Gitea instance is the simplest solution.

An added bonus to using Gitea is flexibility in mirroring LFS objects, which can be sent to S3 or minio

account426• 3 years ago

The same works also over SSH using user@host:path for the remote - you don't need a deamon running on your server to push to it.

stuaxo• 3 years ago

Wonder if any data hoarders have made scripts to clone every site you have started on GitHub?

rezonant5• 3 years ago

I have to assume that the original repository was private when you forked it. If it was public, and then made private, then this should not happen.

If the original code was "open source", then why exactly was it in a private repository? Putting "MIT licensed open source" into a private repository is not publishing that source code for the world to use.

It sounds like nothing weird happened here other than this company thinking a private repository was "publishing it as open source".

chrisfosterelli7• 3 years ago

It would have had to be privately forked. If you change your repository from public to private, GitHub detaches the forks and leaves them public, so a repo owner can't just delete someone's public fork.

baobabKoodaa8• 3 years ago

If the original code was "open source", then why exactly was it in a private repository?

It's been too many years to remember the exact reasons, but this was not the only repo in the project. This was the "working directory" that had all kinds of random stuff that a data science project might accrue over time. Later in the project we published 3 repos which were more "cleaned up" to be potentially useful to outsiders (I use scare quotes around "cleaned up" because the codebases are still a mess, sorry).

Anyway, 3 of the 4 repos appear to still be public:

https://github.com/futurice/health-visualizations https://github.com/futurice/health-visualizations-front https://github.com/futurice/laaketutka-prereqs

alerighi4• 3 years ago

Open source doesn't necessary mean that the code has to be share publicly on the internet, let alone on a GitHub repo! This is a common misconception. You can of course also decide to sell an open source software, that is have other pay you to obtain the software source code with an open source license (of course this person then can share the code, or even sell it to others legally, so it's not commonly done).

I can put a MIT or even a GPL license in a private repository that I have at my company. The meaning is that I don't release the source code, tough if one of my employees wants to take it and use it he can, and he can also decide to share it with other people, or put it on a public repo.

Why I don't want to put the repo public? Maybe I'm just lazy, I don't see too much value in the code, I don't want to write documentation, tests, whatever, I don't consider it of enough value, whatever, still I don't have problems with people that have access to the code that they use it, and share it if they want.

rezonant5• 3 years ago

I have no misconception of what open source is, if you want to put your foot near this foot gun intentionally by paying GitHub to host code you've open sourced (while telling no one outside your org that you've open sourced it) that's fine.

A common misconception is that publishing a public repo comes with obligations to add tests, documentation and whatever.

Encrypt-Keeper• 3 years ago

Self hosting something like Gitea and mirroring to that is a good solution to this.

bspammer• 3 years ago

Pretty bizarre behaviour given how easy it is to get around this. Instead of forking, just

git remote set-url origin <new-remote-repo>

The new remote can even still be on GitHub. So what’s the point? Anyone relying on this to retroactively remove access to source code has a false sense of security.

IncRnd6• 3 years ago

Your private repository baobabKoodaa/laaketutka-scripts (forked from futurice/how-to-get-healthy) has been deleted because you are no longer a collaborator on futurice/how-to-get-healthy.
and now it's gone... why?

Because it was a private not a public repo.

kitkat_new15• 3 years ago

so what?

Private doesn't imply (common sense) that the original repository has power over any fork.

dboreham• 3 years ago

It does, otherwise github just wouldn't allow forking private repositories. If they did allow that, and retained no control over the forked copy, now you can ride a coach and horses through the access control to a private repo by simply forking it when you have access. My guess is that forking a private repository is a feature github intended to be used where employees or contractors of an enterprise want to fork their employer's repository as part of their development activities for that employer. Github sees those forks as transitively controlled under the organization's access policies.

IncRnd6• 3 years ago

That is the exact meaning, which does seem to be common sense. You can fork from a public, private or internal repo. Public is public and the fork won't get deleted when the repo removes you. Private deletes your fork, since it isn't your repo. Internal requires gated access.

bastardoperator2• 3 years ago

It's not your code or data, it's your previous employers IP and they determine who has access and who doesn't and what licenses do or don't apply. Forks have worked like this for awhile and I consider it a feature, especially in cases like this.

baobabKoodaa8• 3 years ago

That's not how MIT license works

camsjams2• 3 years ago

So you're suggesting GitHub should determine the total "average" license of a private repo and determine if your fork is indeed valid or not, before revoking access.

bastardoperator2• 3 years ago

No, but that's how private repos work. It's really hard to open source code from a repo no one has access to.

shagie9• 3 years ago

The MIT license applies to the code that was licensed under the MIT license.

The MIT license does not necessarily extend to other contributions to the project that haven't also been specifically MIT licensed too. Those contributions may be under any (or no) license.

gregwebs• 3 years ago

This unfortunately makes sense because it is a private repo. Even if a repo is labeled as being MIT and has an MIT license in it, it still may contain other code of a different license.

Github could do better by warning the repo owner when they delete a private repo. Github could ask the repo owner if they want to convert it to public first (a "set it free" option) or otherwise give the option to avoid deleting the forks of others.

pavon4• 3 years ago

Yes, and under most FLOSS licenses (including BSD/MIT/LGPL/GPL/AGPL), companies are perfectly free to maintain modified versions of the software internally with no obligation to publish the source externally, until/unless they distribute the modified software externally (or allow external use the software remotely in the case of AGPL). All of the modifications are copyright the company and it is their choice whether to release them publicly. Employees having access to the modifications doesn't mean that the modifications have been licensed to them under the original license, and thus doesn't give them permission to distribute the modified software to others under that license.

Essentially what I just said is the same as what you did. The private modifications cannot be assumed to be under the same license as the original software. Gitlab has no way of knowing all these details, and have promised to keep private repos private, so their current policy is the correct one.

kitkat_new15• 3 years ago

"This unfortunately makes sense because it is a private repo."

I disagree. I expect a (i.e. my) fork to be independent of the original repository, no matter if it is private or not.

It's enough if a fork of a private repository is private then too.

bcrosby95• 3 years ago

This is why I don't use github's fork feature. There's more than just this restriction they impose upon you.

Instead I prefer to use a "git" fork. I just clone it and upload it to my own repo. Assuming the license permits of course.

naikrovek4• 3 years ago

Don't use the "Fork" button in the GitHub UI, then. It is intended for collaboration and establishes and maintains the parent-child relationship of "your fork" and if the parent repo is deleted, so are all forks. If the parent repo is private and goes public, so do all forks. If the parent repo is public and switches to private, so do all forks. This behavior is laid out in docs.github.com and is not secret.

This has been the case on github.com for over a decade, and I am slightly shocked that people don't know this. I guess the root of that is that I am surprised that this has not bitten more people than it has.

People assuming things are a certain way and never checking to verify that are by far the greatest source of "I shot myself in the foot" statements that will ever be known.

Mystery-Machine• 3 years ago

GitHub is not supposed to make such decisions for the user here. It is user's responsibility to make sure they delete their private forks if they shouldn't have access to the repo/fork anymore.

What's next? Should we all install spyware on our computers and let GitHub automatically delete local copies of forks as well?

GitHub and the company/person, who deleted the original private repo, should inform the owner of the fork that the main repo was deleted. If need be, company/person can request fork owner to delete their private fork and local clone as well.

pavon4• 3 years ago

I think this incident reinforces that private repos on github.com are a weird hybrid of the public github and on-premise github which creates various practical problems and misunderstandings when those two security models collide.

First off is the fact that forking a repo is often a necessary step in contributing to project if you don't have push permission, so these forks will be created during the normal development processes, not necessarily because the employee was intentionally trying to save off their own copy. So it is perfectly normal for the employer to consider those forks to be something it should own and manage, just like it would on an on-premise installation.

On the otherhand, github still encourages people to use a single account for both personal use and work[1]. Naturally the employees reasonably consider all the forks that are in their personal account to be something that they should own and manage. So you end up with situtations like this.

The lesson - mixing work and personal accounts/computers/devices is a horrible idea regardless of what Github says. Employers shouldn't allow it, and employees should avoid it even if allowed. Then both will have a clear idea of who owns and controls what.

[1] https://docs.github.com/en/get-started/learning-about-github...

jyrkesh• 3 years ago

MIT licensed code doesn't mandate distribution. Companies and organizations are perfectly within their rights to own a private fork of an MIT-licensed codebase in perpetuity.

With that in mind, if you fork an organizationally-managed repository, there's a good chance the owner doesn't want you to continue to have access to that codebase if you're no longer a part of the organization. And the local copy? Well there's a good chance you were only allowed to clone the repo on an IT-managed device with specific 2FA policies and some kind of agent/config to prevent/reduce data exfiltration from that device.

Is it a perfect system? Hell no. Data leaks, that's part of life. And I'm with you that it certainly could be more user-configurable.

But it's also extremely well-documented behavior[1], and seems like a key design choice that GitHub made a long time ago to protect the owners of private repos. Ultimately, if you don't care about who has access to your code, you signal that by making the repo public. Or by telling your private collaborators to make sure they hold on to a local copy.

[1] https://docs.github.com/en/pull-requests/collaborating-with-...

bunbun69• 3 years ago

Seems like you don't like GitHub. Have you considered not using it?

josephcsible• 3 years ago

Never use the fork feature on private repos. Instead, clone the repo locally, create a fresh GitHub repo, and push your local clone manually to that. Doing so will protect you from this attack.

theossuary• 3 years ago

It'd be nice to be able to manually specify an upstream for a repo, that's the main benefit of forking in the UI.

hashtag-til2• 3 years ago

Does that limit submitting PRs to the original repo somehow?

kitkat_new15• 3 years ago

you need to make a fork for that, however you could add that one as a second upstream repo

kevincox3• 3 years ago

Apparently yes. You can't submit PRs to a repo outside of the "fork network". (IIUC these all share a single Git repo under the hood)

bdcravens• 3 years ago

I'm sorry to hear that whoever trained you on using Github didn't explain this to you.

nobody99993• 3 years ago

I completely understand your frustration. It's a dick move to delete your content without (at least) giving you a chance to archive that work.

At the same time this situation points up an important issue: if you don't own/control the infrastructure where your data lives, you don't own that data. Full stop.

If you host your data "in the cloud" (i.e., on someone else's servers) then you don't own that data, or at least not any copies stored there).

I'm not advocating for any specific action/solution in this comment (see my comment history for more about centralization vs. decentralization and "the cloud"), but the above is an important consideration, especially WRT long-term storage of your data.

kaushikc• 3 years ago

It's simple! Maintain the control and the capability to retrieve your important data at all times. The internet is a wild place and everything you don't save could potentially be gone forever.

nobody99993• 3 years ago

It's simple! Maintain the control and the capability to retrieve your important data at all times.

An excellent point, but I'd go further and say that one should maintain multiple copies of important data, with at least one of those on hardware/infrastructure you control and have physical access to.

ben0x539• 3 years ago

At the same time this situation points up an important issue: if you don't own/control the infrastructure where your data lives, you don't own that data. Full stop.

We've known about this for many years!! And yet we for many varied reasons choose to make use of services anyway!! That doesn't mean we shouldn't get to complain about those services and giving people shit for that is really weird!!

nobody99993• 3 years ago

We've known about this for many years!! And yet we for many varied reasons choose to make use of services anyway!! That doesn't mean we shouldn't get to complain about those services and giving people shit for that is really weird!!

I don't disagree at all.

I just find it a little surprising that on a site where "not your keys, not your coins" is accepted wisdom, that "not your storage, not your data" isn't as well accepted.

That said, I am biased and have an agenda:

1. The centralization of network resources is a recipe for disaster;

2. There are many factors which have pushed us toward more centralization, and most of those factors (asymmetric bandwidth on consumer internet links, abusive terms of service, e.g., port blocking/traffic throttling, crappy consumer networking gear, etc., etc., etc) rarely get addressed;

3. The issues in (2) create perverse incentives for commercial entities to further abuse their "customers" (for "free" services that should read "product");

4. Those perverse incentives have morphed outside of paid and "free" SaaS and subscription tech services, encouraging manufacturers of all manner of products (cars, appliances, computers, communication devices and a raft of other products to employ these abusive, rent-seeking tactics as well;

5. Resolving the issues detailed in (2) (as well as those not detailed) could enable both libre and commercial self-hosting products to become a viable, profitable industry, both for products and support services. Thus enabling us (broadly, humans who use the global internet) to actually own and control our data, PII and privacy;

6. Solutions are plentiful, but the perverse incentives cut across the entire OSI stack and beyond, making the reversal of such incentives complex and difficult, especially because the hoi polloi either don't know or have been convinced that they shouldn't care about ownership (of physical products like phones, cars and appliances) of their data and PII. I don't have a comprehensive set of solutions, but creating competition (municipal last-mile broadband, interoperability requirements, etc.) and providing consumers with the tools they need to decide for themselves (symmetric bandwidth on internet links, "dumb" internet pipes, non-abusive TOS, etc.) how they should host/manage/control their data and possessions will be important steps forward in reversing such incentives.

I rant about this every so often (this being my latest offering), and while it's not specific to Github or how their TOS treats various data storage offerings (repos), it's absolutely an example of how these perverse incentives harm and abuse consumers. In my view, that's wrong.

Edit: Clarified my prose.

layer8• 3 years ago

This is an endless rabbit hole though, unless and until, I guess, this gets regulated into a law that makes cloud service providers accountable for the data stored under the user accounts they provide, or something along those lines. Until then, you can (rightfully!) complain about one thing, but then the next feature of the next service you use may again have similar issues.

hgsgm3• 3 years ago

OP specifically asked GitHub to keep the code private!

It's silly to be upset about that, after refusing to make any backups or distribute any copies of this "publicly licensed" software.

smeagull• 3 years ago

This destroys the very basic idea of opensource. That the first developer on a project can hand off to someone else to take over the project, and have the project continue past their involvement.

Now if the first developer deletes their account or repo everything dies.

watermelon0• 3 years ago

This does not affect public repositories, so open source code should be fine.

glitchc• 3 years ago

Do you have a local clone? If yes, just populate a new remote with the local clone. If not, always a good idea to keep an updated clone of every remote repo. That's how git is supposed to work. The latest pull is the backup.

j452• 3 years ago

Looks like a gitlab.com backup of everything is mandatory.

thewellington• 3 years ago

I wonder if this works the other way around. - create new repo in your personal GitHub acct. - fork it to company repo. - leave company years later - revoke access to company codebase. :)

pvaldes• 3 years ago

Totally expected from the minute one after Microsoft buy the vault, earning full control over the software and the people's access to the platform. I will not be surprised at all if some of this closed projects would appear included into closed software under new authors after a year.

capableweb7• 3 years ago

Is this behavior new? It has been like that for as long as I can remember, certainly before Microsoft bought GitHub.

autophagian• 3 years ago

It is not. I also remember this behavior around forking private repos from before the acquisition. But then again, this doesn't lend itself to a conveniently pithy comment about Microsoft ruining everything.

bdowling• 3 years ago

GitHub probably does this to protect itself from potential copyright claims by the copyright owner. In most cases, copyright owners will want Github to stop distributing copies of their code to people whose access they revoked. They won't care whether the copies are in a fork or not. If Github didn't revoke access to the code, including in forked repos, then the copyright owners could send DMCA takedown notices and potentially sue Github. Processing those takes time and money, so they do the easier and cheaper thing.

Sephr• 3 years ago

GitHub also blocks their built-in fork UI functionality when the original repo's owner uses the site's "block" feature against you, which doesn't align with my expectations. Ironically, anyone affected by this poorly implemented blocking behavior (and thereby copying the full repo instead of forking) would be protected from this bug by the OP.

tgsovlerkhgsel• 3 years ago

According to the documentation I remember reading, blocking someone would delete their forks of your repos (including public ones) back then. I believe this is no longer the case.

djur11• 3 years ago

This is because forking is a social feature. If the origin repo is unwilling to accept pull requests from you, there's no reason to let you fork.

s17n• 3 years ago

Same thing happened to me, it sucks. All the development had been in my fork (not just my own work, also my collaborators;) and now it's just gone.

Github should change this policy.

hashtag-til2• 3 years ago

Every day is a new joy with GitHub, these days.

HomeDeLaPot• 3 years ago

Try to treat YouTube, GitHub, and any other service the way you would a failing hard drive. If it's not backed up, it doesn't exist.

whstl• 3 years ago

I have an interesting but unrelated story about files going away in Github.

Remember when Windows Research source code that was used for teaching in university, and was leaked in the late-aughts, way before Microsoft purchased Github?

I had this code in one private repository called "ms" for more than a decade. It didn't have Git history or anything, it was just some random files, plus the leaked sources.

I totally forgot about it until a last year ago, when I checked, and the code is entirely gone.

I'm now more careful about what I put in private repos. In fact I don't have anything private there anymore...

naikrovek4• 3 years ago

This is how GitHub has always worked, and it's been documented the entire time.

Forks are not just copies of other repos which go on to get their own history (or lack thereof) from that point on. They are linked to the parent repo, and share its visibility. If the parent repo goes private, so does the fork, for example.

There's a parent-child relationship that GitHub creates when you fork a repo using the "Fork" button in the UI. That is how this relationship is created.

This has always been the case with the "Fork" button on github.com. This should not have surprised you.

kitkat_new15• 3 years ago

"If the parent repo goes private, so does the fork, for example."

not true

"GitHub will detach public forks of the public repository and put them into a new network. Public forks are not made private."

Vecr• 3 years ago

I'm not sure why people are defending Github on this issue, what if the original repo was a template or something, and your thousands of lines of code is gone because the original template repo removed you as a contributor. If I copy something, I expect where I copied it from to have exactly zero bearing on what happens to my copy. If they have a problem they can serve legal documents, giving everyone time to figure something out with zero data loss.

theturtletalks• 3 years ago

Guess I’ll be cloning repos locally and then pushing instead of forking.

kevincox3• 3 years ago

Apparently this removes you ability to open Pull Requests.

andybak• 3 years ago

I agree completely.

But at the same time - who doesn't have local copies of anything they care about? What are they thinking!?

sshine• 3 years ago

You’d be surprised at how many use GirHub as their remote code backup platform. Having private file system backups is a question of culture, and a lot don’t have it.

rezonant5• 3 years ago

This only applies to private repositories. Do not put templates up as private repositories, and do not use forks when consuming templates. That is not what forks are for.

account426• 3 years ago

It's irrelevant what they are for. Reality means that things like this get misused but deleting data without warning is still not ok.

sangnoir3• 3 years ago

If you have a Raspberry Pi wasting away in a drawer[0], I strongly recommend installing Gitea or Forgejo and mirroring all the repos you like (i.e. the ones you contribute(d) to and/or starred, not just on Github too!). You set it up once and it will sync in upstream changes as often as you like (default is daily)

0. Or a homelab, or a cheap 256MB VM, or a NAS that can run docker containers, or an old Chromebook: anything that can execute a Go binary, x64, arm6 or even mips

kitkat_new15• 3 years ago

I'll look for a tool that mirrors all my repositories (i.e. only forks). If I can't find one, I'll write it myself.

jmondi2• 3 years ago

I wrote a little tool to mirror my repositories to my gitea instance. It has been months to potentially a year or two since I ran this, but it does what you’re asking. You can mirror repositories of users and repos users have starred. It definitely needs some love.

https://github.com/jasonraimondi/deno-mirror-to-gitea

pabs3• 3 years ago

https://github-backup.branchable.com/

drudoo• 3 years ago

This is build in to Gitea. I mirror multiple github repos to my private gitea instance.

remram5• 3 years ago

I wonder if there is an efficient way to do direct incremental git-to-S3 backups, or if you have to do this, run a Git mirror and do regular filesystem-level backups of it.

anshumankmr• 3 years ago

a cron job ought to do this. Try something like Cloud Scheduler that can automate this for you.( I am not sure what the equivalent is in AWS)

yencabulator• 3 years ago

https://git-scm.com/docs/git-bundle creates a single file that contains everything in A not in B, so e.g. the delta between the state of the repository yesterday and the state today (for all the refs in it). You just need to produce the "rev list" to save.

tuxracer• 3 years ago

Recently setup Forgejo and it's been working out great as a self-hosted GitHub alternative. Once we have cross-domain pull requests and comments (wip) these alternatives can hopefully really take off.

oever• 3 years ago

You can send the code to the Software Heritage:

https://archive.softwareheritage.org/save/

Perhaps there's a userscript to send everything that you fork there.

Ideally, federated forges will become more widely used: then a fork would mean copying the code to a forge of your choosing and there is less risk of losing code.

ajb• 3 years ago

It's with noting that the person doing the Spring Cleaning would have been warned that they were about to delete your forks. They could have contacted you to mention this

I don't think there's actually an option not to delete them, which would have made sense in this case.

jacobsenscott• 3 years ago

I bet github is used as much or more for proprietary corporate development as it is used for open source development. This is the right choice for paying customers.

OJFord• 3 years ago

That was an MIT-licensed open source project

I wonder if perhaps they made it private before deleting it. It would make sense perhaps for GitHub to do this if the original was private and you lost access to it (though.. you should still somehow retain downstream work?) - and I can easily imagine that coming with a 'checking private status at time of delete not time of fork' bug.

dom96• 3 years ago

I'm getting a sense like something is not right at GitHub, maybe it's just a coincidence but just in the last week I saw GitHub breaking their CI (causing many runs to fail due to a billing bug), then the checksum tarball issue and now this.

djur11• 3 years ago

This is how private forks have worked at GitHub forever.

yieldcrv• 3 years ago

Turns out I don't even have a local copy of it anymore

Looks like you'll never do that again

karmaMeansCool• 3 years ago

This is why I never fork a repo from github. I checkout my own branch (like git is supposed to be) and create a repo on github and most importantly, another server.

Jorengarenar• 3 years ago

We ought to treat GitHub's "fork" more like "branch on my account".

If one wants to create an actual fork, then clone the repository locally, change the upstream and push to repo on your account.

barco• 3 years ago

Given the nature of super forks in GitHub, if you are no longer a contributor, but keep your fork, you would be able to see the commits that happen in the original repo and all its forks.

There used to be a way to bypass the deletion, which was to clone into a organization. That way, GitHub would not delete your fork. I am not sure if it is still the case.

I reported all this to GitHub a few years ago, but they said it was a non-issue .

marc_abonce• 3 years ago

Sometimes this can also happen with public forks too. I learned this the hard way when youtube-dl was temporarily taken down.

phendrenad2• 3 years ago

Seems like it's working as intended. The code is MIT-licensed, but github doesn't know that. It only knows that you were allowed access to a private repo, and that access was revoked. This is supposed to save you the trouble of going through all of your forks and ensuring that you aren't retaining any code from an organization you left. Personally I make new github accounts for every new org I join, but that's just me.

jakeogh• 3 years ago

Anyone know why github search seems to be broken? While logged in, if I query https://github.com/yt-dlp/yt-dlp/search?q=exec_cmd no results are returned for any category. But "exec_cmd" exists in a few places in that repo, for example: https://github.com/yt-dlp/yt-dlp/blob/master/yt_dlp/options....

On another note... It's a honest question to ask, does github state anywhere a policy on shadowbanning (treating a repo differently in any way)? It's becoming a law of tech nature that any org big enough starts to do it. Some variation of a warrantcanary (preferably in the form of a contract with the users "we have not done X Y Z") would be cool.

btfo• 3 years ago

You deserve it. This is what you get for using proprietary software and not hosting your own gitea/gitlab. The repository was never yours.

Boltgolt• 3 years ago

You don't think "you deserve it" is a little harsh?

dotancohen• 3 years ago

Anything not on your machine - or requiring phoning home - is not yours.

We saw this with the Heroku being discussed yesterday, we've seen it with Github before. Want to keep it? Store it locally.

joshxyz• 3 years ago

Had this happen to us before. Tradingview's JavaScript SDK is in an invite-only private repository in GitHub. We forked it, made our specific changes, then lost it once the Tradingview team removed our access after weeks of them trying to sell us stuff and not closing any sales from us.

The private fork we had, with our numerous changes, all gone.

hgsgm3• 3 years ago

We published the source code for everyone to use,

Published it where?

If it's published, how did you lose access to the software, even if the fork is gone?

Dr_ReD• 3 years ago

This is the typical boneheaded, executive-mandated, policy. Once they get enough backlash they'll probably backtrack.

Tell HN: GitHub will delete your private repo if you lose access to the original

Related Stories