Highlights from Git 2.34

https://github.blog/2021-11-15-highlights-from-git-2-34/

by todsacerdoti • 5 years ago

160 95 5 years ago

github.blog

IshKebab2• 5 years ago

ort (short for “ostensibly recursive‘s twin”)

Continuing Git's famously clear naming I see!

DON-MAC• 5 years ago

Can't mention git command naming without linking to the parody Git Man Page Generator[0].

[0] https://git-man-page-generator.lokaltog.net/

newsbinator• 5 years ago

git-chase-submodule chases remote submodules using downstream shells, while remoting fitting patches to impale the given submodules.
When COMMIT_ID_DEFINE_LOCAL_CACHE is not patched, the most recent counted grafts chased by submodules in a subtree, but that are not in other-bundle, may be chased by an active head ref. Using sub-recursed or pre-massaged flags only cautiously grips the tag, except when otherwise noted.

teddyh4• 5 years ago

They did it again. GitHub “stealing” the Git release announcement, contributing to the Git/GitHub confusion.

Comments from the last time: https://news.ycombinator.com/item?id=28207168

For those who don’t know: GitHub is owned by Microsoft, a historical (and current) adversary of all things not Microsoft, particularly Linux¹. Git, the tool, is not made by GitHub; GitHub is a web service which is using Git. Git, the tool, was originally made by the person who also originally made (and currently manages) Linux, and the purpose of Git was to manage the development of Linux itself.

1. https://en.wikipedia.org/wiki/Halloween_documents

judge2020• 5 years ago

IMO GitHub posting these is fine since they're headlined "Highlights from Git 2.3x", but the last discussion incorrectly just said "Git 2.33", which is likely where the confusion came from. Hopefully this post will keep the distinction front-and-center.

teddyh4• 5 years ago

If your name is GitHub and you announce a new release of Git, which your product GitHub uses extentively, and you do not explicitly and clearly mention that you do not have any role in making Git itself, then you are lying by omission; deliberately misinforming people who will naturally assume, given no information to the contrary, that GitHub are the makers of Git.

I think that most people are already under the misapprehension that Git and GitHub are the same thing. This style of deliberate confusion from GitHub must be seen to be by design. They are Microsoft; they do not get the benefit of the doubt.

tomnipotent2• 5 years ago

Oh look, the obligatory "Micro$oft is evil" rhetoric based on decades old grievances and very little new evidence. So useful.

It's perfectly reasonable for a company, that uses the tool in its name, to have a blog post going over new features. I'm never amazed the distances people will go to distort a boring blog post into some issue or another based on splitting hairs.

smegsicle• 5 years ago

very little new evidence

Off topic, but there was that dotnetfederation overreach shenanigans

teddyh4• 5 years ago

[…] based on decades old grievances and very little new evidence.

https://news.ycombinator.com/item?id=29251210

teddyh4• 5 years ago

[…] based on decades old grievances and very little new evidence.

https://news.ycombinator.com/item?id=29288052

hannob3• 5 years ago

I don't like Microsoft, but you can't blame microsoft for the fact that apparently the git project never made a proper webpage. Obviously that means people will try to find information about git elsewhere.

vlovich1232• 5 years ago

Isn’t [1] the official git home page? I don’t follow your comment.

[1] https://git-scm.com/

Aicy• 5 years ago

Has git published their own release announcement like this for people to read?

remram• 5 years ago

It's in the repo: https://raw.githubusercontent.com/git/git/master/Documentati...

[edit: in fact it's linked from GitHub's blog post, at the bottom, and it's included in the announcement email: https://lore.kernel.org/git/xmqq8rxpgwki.fsf@gitster.g/]

vlovich1232• 5 years ago

Actually my impression is that Microsoft is probably the largest corporate sponsor currently for git devs. They’re primarily focused on large mono repo support and ecosystem improvements that fit into their GitHub strategy. Overall, I’d characterize their Git involvement as positive.

er4hn2• 5 years ago

It's really neat that ssh keys can be used for signing commits, but I'm not a big fan of this.

When auditing a source repository you want to see how commits are signed over time. In other words you have a key X which is valid at time Y. As Y changes, key X will change as well. To audit a repo, you need to know the history of what key X was valid at time Y.

With GPG you can set up a hierarchy of commit signing keys which have defined lifetimes and are all subordinate to a master key which you do not use for signing (and hopefully keep somewhere secure). One irritant is that the major git-as-a-service products, i.e. github + gitlab, do not respect the use of GPG keys as something varying over time or subordinate to a master key. Instead they let you set a current active key and mark on their own if it was verified at the time. If you want to validate the repository's security yourself, you don't have the proper information to do so.

The use of ssh keys as signing keys furthers this incorrect usage of a single key as something that does not vary over time. You can use ssh certificates as a way to have a hierarchy, but that still doesn't acknowledge that there is historical data of note.

Now, a logical question is: If each git commit is the summation of everything before, why does historical data matter? Why can't I just trust the git{hub,lab} when it says it was "Verified" at the time? A few reasons:

- Github allows someone logged in to make edits via their web editor which they helpfully sign with a universal github key that is always verified. So if your AuthN is bad, that is a problem.

- You are trusting that your git-as-a-service provider will never have their repositories tampered with and your org will just download the newest version of the repo.

- It relies on git-as-a-service providers to track this. If you aren't using a managed service provider you need to roll your own auditing tools.

hannob3• 5 years ago

When auditing a source repository you want to see how commits are signed over time.

I understand your reasoning and the idea behind GPG and everything, but here's my challenge to all of this: Has this ever happened?

Like, the scenario we're talking about is that at some point some bad code, maybe a backdoor, got introduced into a code repo, and it later is discovered and someone launches an investigation. And they find out "this was committed by XYZ but XYZ claims he didn't do it".

I don't remember any such incident ever. Given that I feel a slightly weaker form with SSH keys may still be justified. (And it is still easily conceivable to have some out-of-band-metadata, e.g. a project that feels this is important could just have a list of developer SSH key fingerprints within the repo and thus have the backlog of who's keys were relevant at what point in time.)

er4hn2• 5 years ago

Has this ever happened?

Unfortunately this has: https://www.wired.com/2015/12/researchers-solve-the-juniper-...

While the full results were not released, it was pretty clear that a rogue commit happened to the VCS. This is the highest profile example that has occurred in my corner of the tech world.

The broader category of this attack is "insider threat" and it is a real thing. Even googling for "malicious code commit" turns up this other example for me as one of the first results: https://www.theregister.com/2021/03/29/php_repository_infect...

eropple• 5 years ago

This is a totally valid criticism IMO, and I think platforms should allow for disallowing signatures based on origin, but there's something that, for the majority of users, really holds true:

With GPG you can set up a hierarchy of commit signing keys which have defined lifetimes and are all subordinate to a master key which you do not use for signing (and hopefully keep somewhere secure).

Within epsilon of nobody is going to do this, but they all have an SSH key. Heck, I don't do this, and I even actually know how to make GPG do it. It's a huge hassle for what, at this point, is really only useful to me for...signing Git commits? I don't need to sign anything else with it, and GPG mail is nonexistent.

(Also, fwiw, if you aren't using a managed service provider, I hope you have a directory service where you can have those keys. If you aren't using a managed service provider and you don't have a directory service, I'm gonna go out on a limb and say that I'm guessing security isn't that important to you.)

agwa• 5 years ago

It's possible to set "valid-before" and "valid-after" timestamps on SSH keys in the allowed signers file:

https://man.openbsd.org/ssh-keygen#ALLOWED_SIGNERS

When validating the SSH signature, Git can set the verify-time option to the commit timestamp and the signature will be rejected if the key is not valid at that time. Git doesn't yet do this, but will in a future version.

This is considerably simpler than the rigamarole of GPG subkeys.

Androider2• 5 years ago

All of this signing is only useful if someone is actually checking or enforcing those signatures, otherwise it's just LARPing.

Being able to sign commits with your SSH keys makes signing actually useful, because it enables a new workflow that developers will use:

- You give every dev on your team a Yubikey

- They generate an ed25519-sk key that only resides on the Yubikey, no software required as it works out of the box with both openssh and GitHub

- They upload the public ID of the key to GitHub, same as before

- You enforce commit signature verification for your GitHub org. You're done, no need to install any software, everything Just Works.

You now have:

- No private keys on developers machines, rendering all types of supply chain attacks like NPM stealing your .ssh files ineffective

- Enforced 2FA for everyone without any hassle

- Every commit signed by developers, enforced and with no developer overhead. Checks a lot of boxes for those SOCs and ISOs.

entropie• 5 years ago

They generate an ed25519-sk key that only resides on the Yubikey, no software required as it works out of the box with both openssh and GitHub

How does this work? Can you like me a ressource?

dec0dedab0de• 5 years ago

- No private keys on developers machines, rendering all types of supply chain attacks like NPM stealing your .ssh files ineffective

Would it be able to get the key from the usb device? Or does the hashing happen on the device itself with the private key never exposed? If that's how it works, that would be awesome, but would a malicious program be able to sign something as you, instead of stealing the keys?

I'm going to look into this more and maybe recommend it at work.

ublaze• 5 years ago

I interviewed one of the creators of the Git sparse index a while back: https://www.softwareatscale.dev/p/software-at-scale-32-derri... which had some interesting tidbits.

sigzero• 5 years ago

It doesn't look like you can actually get that release yet.

bastardoperator• 5 years ago

It's there now: https://mirrors.edge.kernel.org/pub/software/scm/git/

  git-2.34.0.tar.gz   15-Nov-2021 16:48 10M
  git-2.34.0.tar.sign 15-Nov-2021 16:48 566
  git-2.34.0.tar.xz   15-Nov-2021 16:48 6M

childintime• 5 years ago

Related: it's been a year since Pijul (https://pijul.org) began working towards 1.0 (https://news.ycombinator.com/item?id=25032956). Seems to be coming along nicely, they're in alpha now and it's showing.

kadoban4• 5 years ago

I keep checking in on that. In my case what I need to start seriously using it is two-way git interop. I need to be able to work in pijul locally and "publish" somehow to git. So far that still doesn't seem to exist.

martinvonz2• 5 years ago

If the Pijul feature you care most about is "first-class conflicts", you may want to check out my VCS: https://github.com/martinvonz/jj. It has git interop.

pmeunier2• 5 years ago

First, you can easily import Git repos into Pijul.

One issue with two-way interop is that while importing a commit into a patch is easy, doing the opposite is more complicated. Indeed, since Pijul is patch-based, there is no way to map patches onto commits in a unique way, since a single patch is quite independent from the context (other patches around it), which is what makes Pijul easy to use.

Also, if Alice and Bob work on a common Pijul server in parallel with their Git server, they'll get artificial conflicts in Pijul, even though they have the same contents.

avar8• 5 years ago

I've got some patches in this release. If anyone's got question that someone who contributes to git might be able to answer I'd love to help.

Are there any specific things that git does that you wish were done better? I probably won't be able to help with very generic things like "it's complex", but if there's specific drawbacks in some particular commands I might be able to fix them.

Or anything else, I'll monitor this thread while there's interest.

henrikeh• 5 years ago

Well, since you ask. `git add -i` has different "escape"/"exit" semantics for different actions.

If I want to quit/escape adding a patch in the interactive mode I type `q` and then enter.

If I want to quit/escape the interactive add subcommand I type `q` and then enter.

But if I want to quit the `update`, `untracked` etc. I have to enter nothing and then press enter. It would be such a wonderful thing if it also accepted `q` so my muscle "memory" is correct.

I looked into changing this once, but it is written in shell and Perl, which I don't know enough about to figure out.

avar8• 5 years ago

Aside from the UX question you've got, which I haven't had time to look at (sorry). We've shipped a pure C version of "add -i" that's been on if you've enabled feature.experimental=true since v2.29.0.

It looks like the ball may have been dropped on opting users in to it by default, but in any case there's numerous bugs in "git add -p" and "git add -i" that got fixed along the way, and fixing existing ones should be a lot easier now.

Note: I've also worked on the perl compiler/interpreter in a past life, so it's not that I'm afraid of hacking on Perl, but doing so in the context of the Git project is and was always particularly painful. If something's written in C you could use all the APIs git had internally, but anything in .sh or .perl was usually stuck parsing text, or reinventing the wheel.

harrisi2• 5 years ago

I actually just ran into this yesterday and completely agree! I ended up just C-c-ing out and used -p.

The other thing that was frustrating with that was that the help text for what commands do is printed before everything, so if your window is small and there's a lot of output, it's not obvious that the ? you typed did anything other than re-print the hunk. It seems odd to not print the help information at the end so you will always see it (unless your window is less than one row tall, but.. you're probably here for dragons anyway in that situation).

tux32• 5 years ago

I'm not always very happy with the conflicts git rebase gives me.

For example when I try to bring an old feature branch forward, or when I have two private branches with a common history that I've touched on one side and want rebase to apply my history change onto the other branch, I'm often made to resolve a lot more conflict that seems necessary.

There are scripts like git-imerge that give some much nicer conflicts. imerge may not be perfect, but I think it shows there's room for a more clever algorithm.

I'd happily have a git rebase --full-resolver --wuauctl option that consumes a mysteriously large amounts of CPU time, if it saves me some manual conflict resolution =)

IlGrigiore• 5 years ago

They have shipped a new default merging strategy, 'ort', that is supposedly better than the previous default strategy at resolving this kind of issue. You should probably try and see if it makes any difference.

pletnes2• 5 years ago

Sometimes I almost lose my mind over these situations where git on windows reports a permissions change (git status claims a diff) but there is no way to commit nor revert. This happens maybe once per year and I’ve no idea what a proper fix is. Typically blocks rebasing etc but revert and commit can’t fix it. (Apologies for the poor and ranty bug report.)

ohgodplsno• 5 years ago

git config core.filemode false ?

JoshStrobl• 5 years ago

I ended up putting this in my .gitconfig (global)

    [alias]
        permission-reset = "!git diff -p | grep -E \"^(diff|old mode|new mode)\" | sed -e \"s/^old/NEW/;s/^new/old/;s/^NEW/new/\" | git apply"

So when I run git permission-reset, it sorts it out.

Lammy• 5 years ago

Are there any specific things that git does that you wish were done better?

I wish `git-log`'s `--follow` wasn't optional so GitHub et al would show the complete history for files I `git mv`. Mercurial got this one right :)

avar8• 5 years ago

Yeah, me to. I think that was always the plan, but nobody's had the time to go back, re-visit it, and make it so.

I'd say that most of the work of driving features forward in git is hunting down those various edge cases, taking the time to understand them, and explaining them to others. So if anyone's interested in flipping that particular switch it's very open to first-time contributions.

beermonster3• 5 years ago

Do you know what github does differently when performing a merge? I’ve noticed sometimes github says there’s a merge conflict yet when i do the merge locally using the default recursive strategy it often just merges without any conflicts. I know there are cases it can’t handle when there are conflicts, but it seems odd that it cannot merge when there are no merge conflicts.

harrisi2• 5 years ago

This might help https://docs.github.com/en/repositories/configuring-branches...

avar8• 5 years ago

GitHub (which I'm not affiliated with) does some (all?) of their server-side merging with libgit2, not git itself, although I know they're considering moving to git with the "ort" work. So perhaps it's one of the cases where libgit2's behavior differs?

ethomson2• 5 years ago

GitHub turns off rename detection and turns off recursive base-building when creating merge commits in pull requests. Both of these selections would cause differences in when a merge produces conflicts. This is for backward compatibility with historical mechanisms for merging pull requests, but it seems like something that we might want to revisit.

namrog84• 5 years ago

I really enjoy git way more than others but working in unreal engine and with binary assets. I've always felt like git filr locks were an area that still needs more attention.

I know file locks are almost an antipattern for git. But there are definitely good use cases and arguments for better git support and related tooling support around them, especially for binary heavier areas.

The other thing I saw other day was sparse index or something. At work we have a monorepo and I wish that was more control or granularity over checking out a subset of a repo somehow. Our clones even with shallow fetch and lfs can be 5+ minutes for an area that should only take 3 seconds. But no good subset checkout.

gravypod• 5 years ago

I have two wishes for git:

1. A consistent cross-language API for reading and writing from a bare/remote git repo in ways that would make sense to an average programmer. Something like `git.stat("/file.txt", commit=...)`, `git.open("/file.txt", branch=...)` `git.list_commits(since=...)`, `git.create_commit(parrent_commits=[], branch="", ...)`

2. A good story, better than LFS, for storing large binaries over time.

For #2 I want to reproducibly build a debian image from vendored packages or recompile gcc from source so I need to check in a pre-built compiler. etc

btschaegg4• 5 years ago

Something that has been nagging me for my workflow is how Git handles --autosquash.

I really like the idea, and wrote my personal little `git autosquash` command that tries to figure out the oldest commit it has to run `git rebase -ir --autosquash` on in order to merge all current "fixup! …" commits into their counterparts.

However, doing this in a simple shell script is rather dodgy if you run into more generic commit messages. Personally, I'd love it if git allowed me to create "fixup! <commit hash>" commits instead. I mean -- it's a fixup. It's not supposed to be readable anyway.

avar8• 5 years ago

You can "git commit --fixup <commit>", but that creates a commit whose subject is the subject line of the commit to be fixed up, so that if you run one rebase and the hashes change you can still run --autosquash and the like (and of course, manually move them around).

Do you mean that you've got commit subject that are ambiguous within the series you're rebasing, so --autosquash won't correctly attribute them?

saurik• 5 years ago

OK: I have a large number of large git submodules that I never edit, but they make every other command that just incidentally creates a diff output slow. I have found command line options for things like git status/diff (--ignore-submodules=dirty) and git commit (--no-status) to avoid this penalty, but I use git rebase -i constantly and wany to use git stash a lot, but the cost of these commands is somewhat brutal when they sometimes cause incidental status reports. What I really want is a setting that like "avoid incidental status" that disables status reports from any command except git status (as I do want to be able to see this status), as I simply don't ever need the noise (such as in the git commit default message) and it really is a pretty annoying delay when rapidly throwing around commands to edit history to have one which suddenly decides to take forever. If nothing else, though, a --no-status for git stash/rebase (I use all these commands through aliases anyway) would be very helpful. (I would be happy to be more specific if this sounds interesting but isn't enough to know what I mean, but I figure it might either be something you wouldn't get behind anyway and I also really want to take a nap and so want to leave a message earlier ;P. I also happen to be saurik@saurik.com and @saurik on Twitter.)

jakub_g• 5 years ago

Major: A new default merge strategy

In Git 2.34, ort is now the default merge strategy, so you should notice faster merges with fewer bugs just by upgrading
a rewrite allowed Git to implement a merge strategy that doesn’t operate on the index

thriftwy• 5 years ago

Git is awesome but it would really be great if they sunsetted merge instead of putting lipstick on it.

Merge of two good commits will silently produce bad commit.

Merges are not repeatable and can't be reasoned about.

Merges get in a way of git bisect.

Merge makes simple "git log" not contain all the log.

Good luck cherry-picking a merge commit.

Merge makes a commit which has important changes not apply to any specific revision of git log.

It's an enormous foot gun and a single contributor may inadvertenly taint your repository for life.

Merge is a make-believe working set operation, like if you could pretend to add apple to mango and get a pomelo.

Merge is VCS analog of lead acetate. Something you have no reason to ever put in your mouth.

joconde2• 5 years ago

How else could we take two diverging branches and mix them back into one?

pietherr• 5 years ago

"... the speedup is over 9000x" :)

kazinator4• 5 years ago

It would be fantastic if someone maintained a bugfix only fork of git 1.7.x.

will4274• 5 years ago

? What's special about this version?

kazinator4• 5 years ago

Works fine; I have no issues with old installations that still use it.

Highlights from Git 2.34

Related Stories