Lore – Open source version control system designed for scalability

https://lore.org/

by regnerba • 1 month ago

1276 675 1 month ago

lore.org

headwayoldest• 1 month ago

How long before Epic starts giving away other software and suing git to support lore?

glouwbug2• 1 month ago

I can’t remember the last time anyone actually played the game they got for free on epic’s store

tombert2• 1 month ago

I play Brotato fairly often. I am not sure I have played any of the other free games I’ve gotten though.

Tomte• 1 month ago

Kerbal Space Program, Civilization 6, Hogwarts Legacy. All games nobody ever played…

WillPostForFood• 1 month ago

Citizen Sleeper, the game that is going free tomorrow on the Epic store is really good. Space Cyberpunk themed RPG/Survival Sim.

It is interesting that people are so cynical about Epic giving out free games. I get that people love Steam, but competition in the storefront market is not bad.

falcor842• 1 month ago

I've been getting them for years now and have enjoyed many of them (though not a large percentage), and would love to use the Epic store more, if only they would implement some version of reviews/scores. Without that, I feel that I can't trust them as a marketplace.

RobotToaster• 1 month ago

I played hogwarts legacy for about an hour before realising it was boring, does that count?

rbits• 1 month ago

I definitely have. But they're all the games that they gave out in the early days, the more recent games haven't really appealed to me. And it didn't convince me to spend any money, I still haven't spent a single cent on anything Epic Games.

UltimateEdge• 1 month ago

Ahah, the second and third links on the page are to GitHub

echoangle• 1 month ago

Well it seems to be intended for repos with large blobs like video games so it kind of makes sense that their own pure source code is still managed with git.

MBCook• 1 month ago

It makes total sense. It’s just kind of ironic.

AndriyKunitsyn• 1 month ago

Epic's Unreal Engine development is made in Perforce, and then it was mirrored to git/Github. Or that's how it was a couple years ago. That's kind of expected, VCS is one thing, a forge is another thing.

bachittle• 1 month ago

this looks cool for game development, because using Git for projects in Unity and Unreal Engine definitely has it's issues. I'm personally not a fan of Git LFS, especially since GitHub charges you to use it (which makes sense, binaries and assets are big, code is small, relatively speaking).

gbraad4• 1 month ago

What makes lore better or worth considering... when svn and git never failed me...

hootz• 1 month ago

Apparently, how it handles binaries when developing games.

gbraad4• 1 month ago

At least a concise argument that makes sense. Not sure why the downforce happened.

Are there any benchmarks/comparisons

bpicolo• 1 month ago

It’s optimized for projects—including games and entertainment—that combine code with large binary assets, and caters for the needs of developers and artists alike.

gbraad4• 1 month ago

Stating the page initial remarks does not help. Why?

I got my answer when someone mentioned perforce.

wongarsu• 1 month ago

Git does great with text files, but game development contains a lot of binary assets (textures, videos, 3d models) and correspondingly huge repos. git-lfs tries to patch around that, but that makes a complex tool that creatives struggle to understand even more complex. Perforce is a pretty popular solution, and was used by Epic in the past

frollogaston12• 1 month ago

Have to think they're doing this out of a real need, given they were already using Perforce and must've considered Git too. It's also not like 2010s when version control was a hyped thing.

koolala• 1 month ago

Is there no git trick to turn off version control on non-text files but still store them? How does Lore handle them better?

pdpi• 1 month ago

When you have a game that weighs in at 100GB, only a tiny fraction of that is built from code. The rest of it is binary assets that most VCSs struggle with. What makes this worth considering is the fact that they designed it to handle binary assets.

maccard14• 1 month ago

And if your resulting game is 100GB, the source repo was 4TB.

gbraad4• 1 month ago

Not sure why the downvotes, as this is a genuine question. In game development svn is often considered due to issues that git has with large files. Just stating what currently is done, and curious to understand what lore improves. Svn is still going strong among many shops.

The comparison with perforce explains more

moralestapia• 1 month ago

What a waste of a phenomenal domain name.

Razengan• 1 month ago

My first thought exactly :(

rirze• 1 month ago

I can't imagine it was cheap to acquire it, so any company with VC/big Corp money would've taken it any ways.

throwawy4321• 1 month ago

What an epic domain name :)

bel8• 1 month ago

repo: https://github.com/EpicGames/lore

Looks very git-ish. But probably better equipped for large binary files.

    echo "Hello, Lore" > hello.txt

    lore stage hello.txt

    lore status --scan

    lore commit "Initial revision"

    lore push

Snafuh• 1 month ago

Git-ish CLI is great. The GUI is more important though. Non-programmers don't want to dabble with CLI. One reason why Perforce is the defacto standard IMO. The GUI covers 99% of daily used operations and is easy to use.

speps• 1 month ago

They’ve been dabbling in this space within Unreal Engine for a few years. Perforce is the de facto standard in AAA studios from my experience, curious to see what’s going to happen to them.

glouwbug2• 1 month ago

I’ve always wanted a git with five commands, and maybe with AST based diffing

simcop2387• 1 month ago

The five command part isn't really possible but you can use custom diffs for merges, git diff, etc. pretty easily. There are projects like diffsitter ( https://github.com/afnanenayet/diffsitter ) for doing more intelligent diffs like this for supported languages.

EDIT: and then an example for the merge stuff I couldn't find while typing before: https://mergiraf.org/ and HN discussion a few years ago: https://news.ycombinator.com/item?id=42093756

frollogaston12• 1 month ago

clone, pull, push, branch, merge, add, commit are the ones I use, but that's 7

gritzko3• 1 month ago

Five is enough. Beagle uses five HTTP verbs: GET, POST, PUT, DELETE, PATCH. And it is syntax-aware.

https://replicated.wiki/blog/uris.html

pushcx• 1 month ago

I had wanted the same thing for a long time and jj + difftastic has satisfied me.

feelamee4• 1 month ago

How does jj handle problems described in the lore website? I'm curious. I thought jj didn't have any goals about large binary files

ryukoposting• 1 month ago

Hosted onn GitHub. Heh.

Tuna-Fish• 1 month ago

Lore itself is not an example of a program that meaningfully benefits from any of the key features of Lore.

Lore is meant for situations where your repository is going to contain gigabytes of binary files, such as art assets for games. Git is technically great at everything but that, and even the external solutions for that situation still kind of suck.

Wowfunhappy3• 1 month ago

Is Lore worse at managing text files, though? If not, it might make sense to adopt it fully in an organization so you can just use one tool.

LoganDark4• 1 month ago

Interesting to note that this does not seem like a DVCS in the traditional sense because it depends on coordinating with a central server where all repositories will be hosted. I can't tell if servers can pull/push from eachother.

applfanboysbgon3• 1 month ago

It is not a DVCS.

3.2 Explicit non-goals¶
Peer-to-peer decentralization. Lore is centralized by design. Two clients communicate through the remote, not directly.

LoganDark4• 1 month ago

Didn't catch that, thanks. I just wanted to point out that 'version control system' does not mean it is like Git, Mercurial, Sapling, etc.

dofm• 1 month ago

So it's more like git in practice than git in principle, I guess.

I am not sure it is much of an issue to implement a centralised system to solve domain-specific problems that are caused by or are in the context of inevitable centralisation.

I can see a bunch of media companies liking this, maybe even small design shops. Quite a lot of CMS and change control work for media is not much more than asset management; version control is often a bit of an afterthought.

Web interfaces to this might end up interesting for non-profits and charities with significant media outreach.

LoganDark4• 1 month ago

Clearly this works for Epic Games internally, so it's not an issue. A top-level comment also makes the more appropriate comparison to Perforce.

jacobgold• 1 month ago

I'd trust this project more if it was named Data.

analog83742• 1 month ago

All Data is Lore. I mean lore is a superset of data. I mean data is lore with a special attribute.

I'm not just picking nits here. And this is not cynicism.

so there you go.

falcor842• 1 month ago

I'm not the parent, but I suppose it was a joke reference to Star Trek where Data (an android character) discovers he has a brother named Lore [0].

[0] https://en.wikipedia.org/wiki/List_of_minor_Star_Trek:_The_N...

testdelacc1• 1 month ago

The likely source for Lore - https://memory-alpha.fandom.com/wiki/Lore

HansHamster• 1 month ago

Yeah, I'm still concerned about crystalline entities suddenly showing up. Have they ever fixed it? I don't see anything in the issue tracker, probably because no one was left last time to report it...

zdw• 1 month ago

Dunno, this seems fully functional?

superkuh2• 1 month ago

"Lore" is appropriate. Epic games is a very unethical company that steals from people. Myself in paticular. I bought Rocket League the game for linux from Psyonix. Epic bought Psyonix and immediately removed the game clients for linux and mac os. I can no longer play. They stole from me and many others. It'd be one thing if they just shut down the game entirely, but stealing it from only some people while keeping it going for others is worse than just killing games.

stronglikedan• 1 month ago

They stole from me and many others.

It's more likely that they took advantage of some things in the license that you weren't aware of, since stealing would be illegal.

VibrantClarity• 1 month ago

I haven't played it since the buyout either, but I'm told it works perfectly with Proton.

CommanderData• 1 month ago

Oh brother.

Don't be too hard on Lore.

manifoldgeo2• 1 month ago

If it were Data instead of Lore, it would reject any commit messages that used contractions such as "I'm" and "can't" ;-)

niek_pas• 1 month ago

Just today as I pushed some changes to Github, I was thinking how user-unfriendly Git's UI is:

    Enumerating objects: 5, done.
    Counting objects: 100% (5/5), done.
    Delta compression using up to 10 threads
    Compressing objects: 100% (3/3), done.
    Writing objects: 100% (3/3), 290 bytes | 290.00 KiB/s, done.
    Total 3 (delta 2), reused 0 (delta 0), pack-reused 0
    remote: Resolving deltas: 100% (2/2), completed with 2 local objects.

I know all of these things communicate something to the die-hard Git user, but for most people (even most people using Git, I bet) this is just complete gobbledegook. What the hell is "delta compression"? Why do I care how many threads it's using? What is an 'object' and what does it mean when it's 'local'? What does 'pack-reused' mean?

From the documentation, it looks like Lore does a bit better in this regard:

    Pushing 1 fragment(s)
    Pushed 1 fragment(s), 124.00 bytes
    Pushing a3f8c2d1... to branch main
    Pushed revision 1 -> a3f8c2d1... to branch main

archerx• 1 month ago

I use GitHub desktop app that pushes to my local Gitlab. It’s a nice and simple GUI, it might be what you’re looking for.

cedws• 1 month ago

Git as a data structure is clever, but Git as a CLI is atrocious.

raverbashing• 1 month ago

Yes, the famous debate between plumbing and porcelain

Still the porcelain is more like cold stainless steel

cmrdporcupine3• 1 month ago

Obligatory: https://git-man-page-generator.lokaltog.net/

VikingCoder3• 1 month ago

I feel like, everyone near Git has decided, "Well, all abstractions leak - so we might as well stand in the rain like Andy Dufresne when he escaped from Shawshank Prison!"

171862744013• 1 month ago

But that's all Git originally aspired to be. It was supposed to be the base VCS, that you eventually can build a UI layer on top.

e40• 1 month ago

I think we can all agree that information should be behind a -v CLA. It's probably just something no one has thought of doing. I've learned over the decades to just ignore it.

maccard14• 1 month ago

It's probably just something no one has thought of doing

There are 1000 things that's true of about git. At a certain point that becomes a problem in and of itself.

foresto• 1 month ago

It's probably just something no one has thought of doing.

One might reasonably think that about a number of git's rough edges, and one might be surprised at the reality.

Some years ago, the annoyance of git's inconsistent terminology drove me to look into consolidating "cache", "index", and "staging area" in git's help text and documentation. What I found was that others had (of course) thought of it before, but when they tried to do it, it was rejected by git's gatekeepers.

171862744013• 1 month ago

I do want a progress indicator for something that might take a while without asking. If you clone things locally, Git only reports that it cloned something.

kristjansson• 1 month ago

Those are just the sounds that animal makes. Live with the animal long enough, you learn how the sounds correspond to its internal states, even if you don’t really know what they mean.

I’d be a bit worried if git didn’t heave that particular contented sigh when I ask it to push

agumonkey• 1 month ago

I actually like this underlying logs. Could have a concise / project level summary though.

spelunker• 1 month ago

The lights are blinking, so everything must be working!

russdill• 1 month ago

Basically, sometimes there commands take a long time. It's nice to have feedback that something is happening even if it's just effectively a blinking light.

yomismoaqui• 1 month ago

This is what happens when a kernel developer creates tools that need some kind of UX (I say this both as a shitty UX developer and Linus fan)

yoyohello133• 1 month ago

He makes a good tool? Honestly I don’t get the git hate on HN. I’ve been using it for years with no issue. I just read the first 3 parts of the git book and never looked back. I even setup a git server at home with the basic tools.

altmanaltman• 1 month ago

Linus really has very little to do with git's development. He has stated that himself multiple times, and it's the factual truth. "This is what happens when a kernel developer creates tools..." is funny but not factual.

js27• 1 month ago

Objects are your files. Underlying git is a content-addressable filesystem.

The objects are referenced by trees. A tree is just a directory.

The trees are then referenced by commits and/or tags into a DAG with named pointers into various parts of it (which are your branch and tag references):

https://git-scm.com/book/en/v2/Git-Internals-Git-Objects

Because it would be terribly in-efficient to have a bunch of loose objects, git periodically groups them together into packs. To save space, the objects are compressed against one another (delta compression) within the packs.

https://git-scm.com/docs/git-pack-objects

https://github.com/git/git/blob/master/Documentation/technic...

When pushing or pulling, the git transfer protocol basically enumerates what objects each side has so that it only needs to transfer the difference. On top of that, it delta compresses the objects on each side that aren't already grouped into packs against each other to save space.

https://github.com/git/git/blob/master/Documentation/technic...

Because git is an open-source project written by nerds, it shows you all of this information. Feel free to ignore it!

But if you really want to know, it's all documented both in the git book and git documentation directory, both linked above.

(Caveat: I'm working from memory and surely got some detail at least slightly wrong.)

genocidicbunny6• 1 month ago

Now explain this to an artist with very little programming experience beyond what they picked up from their coworkers.

I don't mean to be too glib, but some programmers have this decrepit idea that anyone working with computers should understand programming to be able to fully utilize them.

I worked in gamedev, and many of my colleagues were brilliant, but your comment would read as complete nonsense to many of them. That is the problem git has in the gamedev space. You're trying to manage teams of programmers, designers, sound engineers, gameplay specialists, producers...even c-levels. The parent comment to yours is right, to many many people who work on things that Lore could be useful for would find git to be gobblygook.

anigbrowl• 1 month ago

Respectfully, I think you're missing the point. The GP is not asking for an explanation (and could look that up from the Git docs) but commenting on how user-hostile it is. There's lots of software like this, which is so unfriendly that once you've learned the minimum of how to do something and verify it worked, you don't want to touch it any more unless you really need to.

CLI output should be in plain language and omit or minimize unnecessary detail absent a -verbose flag - for example, I'm just not interested in how many threads something took unless I'm working on it. As a user, I want to be focused on the task I'm doing, not on the perfromance of the tool.

mvdtnz• 1 month ago

Do you know what a rhetorical situation is? Do you understand that the parent was not actually seeking answers to these questions, but was using the questions themselves to make a point about the Git UI?

dansmith19195• 1 month ago

The point is that if it's useless information for 99.9% of users, what's it doing in the UI?

sundarurfriend2• 1 month ago

Because git is an open-source project written by nerds, it shows you all of this information. Feel free to ignore it!

This is the type of attitude that kept most Unix tools quite user-unfriendly for several decades. What information to show the user, and when, are important design decisions to make. Just dumping it all on the user and making them wade through it is not doing the user a favour. Thankfully newer tools seem to be better about this, which has brought the Unix shell forward by leaps, even if there's still ways to go. (You can still make a conscious design decision that the info needs to be all there, and git is one of tools where that's at least somewhat justifiable, but a lot of the time the attitude is more like in the quoted text, that dumping out more info is always better.)

mherkender• 1 month ago

I'd rather see some gobbledegook than extended pauses or idealized (read: fake) information. Those are specific tasks it is doing when you run that command, there's a simplicity to it.

Not saying Lore's approach is bad, but sometimes "worse is better".

dosshell• 1 month ago

Every place I worked at has a git introduction where all new employees learn about how git works internally. Takes 1h, and all junior devs stops memories random commands and actually start to understand. I highly recommend to you to poking around in the .git directory.

The git support for new employees drops basically to zero.

gritzko3• 1 month ago

Every Beagle command:

    gritzko@spot ~/beagle $ be get
     19:07  get ?#0ac49e6a
     16:58  post ?0ac49e6a#POST-018 put:/post: banner on stdout
     19:07  new beagle/test/be-post-put-banner.sh
     19:07  upd dog/INDEX.md
     ...more stuff...
     19:07  del test/post/01-bare-msg/01.put.err.txt
     19:07  del test/post/01-bare-msg/02.post.err.txt
     19:07  get abc?4222dfab

danudey12• 1 month ago

You don't care about any of this information, but that's fine; unless something is going wrong, you can ignore any information that isn't interesting to you.

Having this output is useful for when it does break and you need to copy-paste your terminal output to someone who does understand it to explain it to you or explain how to fix it, but you're correct that 90% of this is effectively debug output that is almost never useful or relevant.

In most cases I would say they should remove any output that isn't necessary, but given that some git operations can be extremely long-running it's beneficial to have some kind of output so the user knows what's going on.

Case in point, this is the output I get when I try to clone the Linux kernel:

    Cloning into 'linux'...
    remote: Enumerating objects: 11623749, done.
    remote: Counting objects: 100% (396/396), done.
    remote: Compressing objects: 100% (189/189), done.
    Receiving objects:   1% (181683/11623749), 90.11 MiB | 19.17 MiB/s

Generally not useful information most of the time, but if I didn't have it I would be staring at a blank terminal for an hour wondering what was happening.

Also, I assume you're not but in case anyone is interested in the answers to these questions:

What the hell is "delta compression"?

The 'delta' is the difference between one thing and another - usually one version of a file and another. Git does some fancy thinking to figure out which files are which other files but with changes, so that it can store just the changes from one version to the next.

For example, a 100 KB file where we only changed 500 bytes ten times would be 1000 KB, but because Git can store the deltas from one to the next it can be 100 KB (the original) plus ten 500-byte changes, for a total of about 105 KB.

Why do I care how many threads it's using?

Because it directly affects how fast the process works; using 16 threads is 16x faster than using 1 thread (on average). Git automatically detects how many CPU threads are available and uses as many as it can, but if it's being very slow you might look and see 'oh, right, this VM only has two CPUs'.

What is an 'object' and what does it mean when it's 'local'?

Uh, this one is deliberately vague I guess. An object is a thing that Git keeps track of. Usually this will refer to a blob, which is 'a bunch of bytes that make up a file', or a 'tree', which is a list of files and other trees - basically a directory structure, or a commit's information, but anything that Git keeps track of is an object.

Local just means that you already have a copy on your system. in the 'remote:' line you see output from the other end (where you're pushing to), so that's the server saying that it's using the files it already has.

What does 'pack-reused' mean?

To be efficient, Git can take all the 'objects' and smush them into one big packfile (rather than having to keep track of hundreds or thousands of separate files). Since Git keeps track of files based on their contents, two identical files are just stored as one copy referenced twice, so it's possible that the file that you're pushing already exists in a pack file and can just be reused rather than having to push another copy.

Melatonic6• 1 month ago

The fact you took the time to explain this I think shows the problem. You did an excellent job.

Maybe what we need is one of those infographic things (or several - but not too many) that summarize the main terminology, actions, and basic architecture of how Git works. I've found those usually, when done right, can strike an ideal balance of reaching the largest amount of people. I personally prefer reading tutorials but I know plenty of talented people who instead prefer videos (which I personally hate) and a simple but visually pleasing graphic is right in between.

y1n02• 1 month ago

I’ve started using JJ vcs, mainly because some people were saying it was great and I didn’t really get it.

I’m starting to come around though. From a UI perspective it’s a major improvement on git. The branching workflow is something that has taken a bit to get used to though.

squirrellous• 1 month ago

The last time I tried JJ it plainly refused to work with git LFS, which is kind of what OOP here is about (large files). Has that improved?

lkjdsklf3• 1 month ago

Jj is weird. In general it’s a better ui and model than git, but there’s a few common operations that are just dumb.

It also recreates a lot of the stupidity of mercurial (which makes sense)

The fact that everyone needs the same stupid “tug” alias because jj wants to use “bookmarks” instead of just named branches is just classic silly shit.

We’ve had named branches for 40 years. Everyone who has vcs experience knows what it is. We don’t need bookmarks. Just stop. Everyone who used mercurial used plugins to turn them into basically branches and same thing with jj.

bartvk• 1 month ago

I bet most people don’t use the commandline client.

justinhj3• 1 month ago

I see Git as a tool aimed at experts that spend time to learn the tool. Asking non technical people to use it is a mistake. You can build guard railed apps on top of it for them, but probably it's the wrong tool.

dansmith19195• 1 month ago

This has bothered me ever since I was using git for the first time: what do you mean I have to 'add' and 'commit' and then 'push'? I just want to save my stuff, this is SO many steps.

yjftsjthsd-h2• 1 month ago

You don't have to do add as a separate step unless you're adding it to the repo for the first time. You can (and I do) just `git commit file1 dir2` or whatever. I don't conceptually think commit+push should be combined; committing a version is separate from pushing it to a remote.

abustamam2• 1 month ago

When I was getting started with programming I often forgot to do one of those steps and often ended up losing my work because I was working from a library computer or something.

When I got more experience I finally understood why it is like that, and it makes sense now, but its still a lot of steps for someone learning to remember.

j_w2• 1 month ago

These criticisms of git always seems so shallow to me.

'add' tells git to start tracking some file(s)

'commit' tells git to save the currently tracked files

'push' says "upload my changes to some other location." Git isn't dropbox magically 'rsync'ing the directory to some server.

'pull' says "download any changes from some other location." Same deal as push.

That should satisfy the majority of git casuals that get frustrated with it. You should learn the tools of your trade, and version control (specifically git) is one of the tools of the software trade. If you work adjacent to software why is it so hard to learn a little about git?

minraws2• 1 month ago

If you can't understand atleast 70% of those lines of Git, you really should be using a git web app or something.

Or maybe try jj. Either is better than you using Git's raw cli as someone who doesn't really know or care how it works.

Git cli is very much made with rough edges and is generally expected to be in hands of an advanced user, these days lots of commands have been made simpler and stuff, but git cli is just still very raw.

qwery2• 1 month ago

I think your criticism of git's output is fine -- fair enough.

What I will say to that directly is that everyone's different and expecting a UI to match every user perfectly is unreasonable. I don't think it's fair to call output that's more verbose than you'd like "user-unfriendly" without qualifying who the user/s is/are.

The thing missing from most of the responses to this is: you can only get this information once. It's effectively impossible to have any of this information repeated -- you can't run it a second time with `--verbose`.

I don't mind things telling me what's going on, and git's push output is perhaps verbose, but not by a margin that causes me trouble. Maybe that comes with always running less than ideal hardware and having second rate internet connections. I want to know why a command took longer to run than I expected.

arendtio• 1 month ago

Actually, it isn't that complicated. If you have a basic understanding of how git works (a little beyond just clone, commit, and push), this can be quite helpful when working with a slow connection or large repositories.

throw2ih0206• 1 month ago

For context, since a lot of people on HN haven't worked on games - this is not intended to compete with Git for general software development. This is a competitor with Perforce for game development.

Git is fine for text based files like code, but it's really bad at stuff like textures, 3D models, audio files, and other non-text files that game developers need to collaborate on. For example, one artist might need to obtain an exclusive lock on some art assets while editing them, because there is no sane way to merge two artists' async edits.

The SOTA in this area is Perforce (https://www.perforce.com/products/helix-core), a proprietary system. From what my gamedev friends tell me, when Perforce works it's great, but it hits enough snags that you need a tools engineer to manage it and occasionally fix issues manually. Git LFS is an alternative, but my gamedev friends all prefer Perforce especially when working on team projects beyond like 3-4 people.

rootlocus3• 1 month ago

Git LFS has file locking, and no VCS can provide you with the tools for diffing binary assets. I don't see any meaningful difference between Perforce, Diversion or Lore and git + LFS + file locking. Unless there's a meaningful performance impact for large projects (I only work on small / medium projects), the capabilities are the same. However, I get excellent git support for code in any editor, as opposed to Diversion or Lore which have none.

regnerba2• 1 month ago

Git LFS for example does not support file chunking. So a single byte change on a large (100s of gigs) file means downloading the whole file again. Lore does chunking of binary files which means faster downloads and better de-dupping on the backend.

throw2ih0206• 1 month ago

I haven't made games for a long time so I can't speak for my experience, only my friends. From what I understand (1) Perforce has decent integrations with the game engine editors my friends work with, so editor support is no factor for them and (2) it has better delta support for the file types they work with - I believe Git LFS mostly uses a generic xdelta diff which is kind of mediocre at everything versus Perforce can understand different file types and be extended to support custom types.

lentil_soup4• 1 month ago

don't think it supports branches

it's also tough when you have 1TB of data, over 1mm files and you might want to lock hundreds files in one go

zipy1243• 1 month ago

Git LFS breaks so often that it can't be seen as a serious professional tool tbh. I've had nothing but trouble with it. If it wants to be serious it needs to be built into Git, not added as some after thought.

flohofwoe3• 1 month ago

I guess you never worked with anything but git? The devil is in the details, and those details generally suck more in git (or generally: distributed version control systems) than in traditional centralized VCS's.

Also git-lfs is a crutch that breaks more often than it works :/

(I agree though that for small game projects, git is mostly 'good enough', even without lfs).

Tiktaalik• 1 month ago

There's also the tooling. Game teams have artists and designers where baroque command line incantations are headwinds to their workflow pace.

For the longest time Git tools were really poor. In recent years there's a few ok ones, like Git Fork, though I wouldn't know if those tools scale to the level of a AAA team size repo and not fall over.

chadgpt33• 1 month ago

Jonathan Blow found it convenient to represent all assets in a large number of text files, to enable merging. For instance he'd have one text file per entity on a map. The game and editor could read either this or the compiled binary version.

Aurornis3• 1 month ago

Jonathan Blow works with extremely small team sizes relative to the big studios. When you only have a couple people working on a project you don’t need all of the same coordination features.

pton_xd• 1 month ago

How do you merge changes to a texture, mesh, audio file, etc?

LugosFergus5• 1 month ago

You really can't merge binary data, such as textures, meshes, audio, etc. It doesn't matter if you base64 encode the data and stuff it in a text file: it's a jumble of data (assuming this is the implication of what Blow did).

superdisk• 1 month ago

He uses SVN and specifically has stated that Git isn't suitable for the work he does due to big binaries in source control.

flohofwoe3• 1 month ago

That's fine for database-like meta-data (e.g. game entity properties), but not for images, videos or audio files. Just writing those as hex dumps into text files doesn't make them any easier to merge.

LugosFergus5• 1 month ago

Something else that git isn't good at: permissions. In gamedev, you might have proprietary work that you want to restrict to certain users. In P4, you can add restrictions to certain directories for only those who have signed the required NDAs. That's not something that you can do in git: it's all or nothing. Maybe you can set something up with submodules, but that's going to upend your repository if you hadn't planned for it.

stevefan1999• 1 month ago

That's by design of git, you can't forget that git is first developed for a bazaar model of information flow, especially with a big decentrailized project like the Linux Kernel, not the silo and isolated corporate NDA and closed model you described. Git prefers open information and discourages information closures and segregation of information by placing restrictions exactly like this.

Git enthusiast would often tell you to do this separately with a submodule, and set permission on the version control forge software level (which means Gitea/Github private RBAC access to certain repos for cloning), sure, but that is also painful as hell.

But my point is that all of this is exactly by design from Linus Torvalds's need for Linux Kernel to replace BitKeeper. Git simply isn't the tool for everything, it was developed for a software project with liberalism in mind, but corporate stuff is monoculture and prefers proprietary, shut-in model, and the eat your own dog food mindset, and no wonder it is so painful to deal with.

rowanG0772• 1 month ago

Doesn't git crypt solve this? You can have encrypted blobs in a repo that will be auto decrypted if you have a working key.

MrDresden• 1 month ago

I once worked in a git repository that required those kinds of restrictions.

This was within a bank and the code in question was related to enabling Apple Pay from within the banking application. The consequences of that information and code leaking or being seen by anyone who had not signed the NDA were very serious (don't remember the details but it made the lawyers were extremely stressed about it).

Needing to figure out a way to protect those parts of the codebase it was decided in the end that the "easiest" way of doing this was to split the repository in half, with the actual artifact building taking place from the half that had the NDA code. The rest of the application (basically the whole application) was then used as a dependency by it.

Still didn't quite solve the issue, but access to that repository was heavily controlled.

PunchyHamster2• 1 month ago

That's not something that you can do in git: it's all or nothing.

That is partially incorrect; you can restrict writes via hooks but not reads; you'd need a workaround like submodules

iveqy• 1 month ago

The way I usually solve this is by using git submodules.

Freedom2• 1 month ago

I ended up writing my own layer over git for permissions for a specific client a long time ago. It has a huge amount of useful features - sadly, I never took the idea further.

Melatonic6• 1 month ago

That always seemed crazy to me about git. Permissions are a pretty basic enterprise offering.

Does Gitlab do better with this?

hhsecurity• 1 month ago

My teaspoons are terrible at peeling potatoes.

Git has no built in authentication or RBAC. Thats not what its for. Its flat file source control.

I swear loads of people havent a clue how git works or why it exists...most of the git based cloud services out there are 90% additional crap bolted on.

__d• 1 month ago

Try gitolite?

https://gitolite.com/gitolite/index.html

It has fine-grained permissions but works with regular git clients.

anitil• 1 month ago

Your comment and the parent comment are fantastic. I'd never have thought of them because I've never worked in the field

usrbinbash• 1 month ago

Something else that git isn't good at: permissions.

It doesn't have to be good at permissions. That's what DevOps platforms that integrate git are for.

TheBigSalad3• 1 month ago

That sucks, git is so absolutely horrible. It's crazy to me that nobody has made anything better yet. Although I could start that myself and yet have not.

VorpalWay• 1 month ago

There are some projects:

* https://github.com/jj-vcs/jj

* https://nest.pijul.com/pijul/pijul

jaapz• 1 month ago

turns out version control is hard

devin• 1 month ago

With respect, were you around to use any of its predecessors?

irishcoffee• 1 month ago

We did, mercurial just didn't win.

zer00eyz2• 1 month ago

Git is fit for purpose. That purpose is to host a monorepo, with out a lot of 3rd party dependancies, distributed, patch based.

Thats not how everyone else works.

We're all using package managers to help with massive amounts of 3rd party dependancies (why are you version pinning in any place other than your repo, why arent you pulling updates through your repo and reviewing them)

We're reliant on tools like artifactory to make sure those depedancys dont disappear or are not corrupted.

We use yet other tools to manage our binary files (this tool would fix that).

Github, gittea, gitlab, bitbucket... have all added piles of tooling around git, that are grafted on around its short comings.

It's crazy to me that nobody has made anything better yet.

Because our entire industry has fallen into the rut of "more tools", of stacking turtles (https://en.wikipedia.org/wiki/Turtles_all_the_way_down ) rather than fixing the real issues that hold us back.

Although I could start that myself and yet have not.

Because unless your a Google or a Linus, no one is going to look twice at your tool for something that is this important. Im not even sure that epic games has the good will, or trust to launch this.

I am going to give them the benefit of the doubt and take a long hard look at it, but my optimism is tempered. But unless it offers a LOT more than git, the extra overhead (lacking IDE support, deployment changes and all the other tooling in GIT's orbit) it isnt going to be a worth while change.

Decabytes• 1 month ago

I wonder how useful this could be as a generalized version control for regular user systems, as a way to rollback, or scrub through history. Presumably if this is designed to work at Epic and Big Game studio scale, it should work at home computer scale

827a2• 1 month ago

This presumption has destroyed far, far more companies and projects than the opposite assumption (that something built for small will scale to big, then doesn't).

ur-whale• 1 month ago

Git is certainly not great with binary assets, but calling perforce SOTA ... ouch.

If perforce is the best there is out there for large binary asset management, then there is a blue ocean worth of potential improvement for git.

Perforce is a piece of crap, a relic of the 20th century that must die in a fiery inferno.

maccard14• 1 month ago

I’ve spent more than a decade working in games and unfortunately perforce is the best out there for a variety of reasons. None of them are good.

asveikau• 1 month ago

15 years ago, both Google and Microsoft were on perforce. (The latter through a fork with a different name.)

danudey12• 1 month ago

It can be SOTA and still be garbage if there's nothing better (and there's nothing better, sadly). This is extremely exciting for anyone who's had to manage revision control for game devs.

debarshri• 1 month ago

I am building a small asset heavy game. Ran into a similar problem. Built a storage cost efficient tool for exactly this [1].

[1] https://github.com/debarshibasak/assets

GabeIsko3• 1 month ago

Man, I always hoped to find a project like this on hackernews. I starred it.

kvirani• 1 month ago

There's also a new player called diversion (diversion.dev) which I think may be a YC startup? Anyway it takes a different approach of being more like Google drive but bringing in VCS behavior making it more indie and designer friendly.

danudey12• 1 month ago

At my previous game-dev-company job we ended up splitting things up into:

1. Code - Git

2. WIP art, shared assets (logos, marketing materials, etc) - Google Drive (because things are often changing, getting passed around, etc)

3. Finished assets (PSD files you're done with, or you think you're done with) - SVN (because we wanted a log of who contributed to what, wanted artists to be able to pick up where someone else left off; having a log of who made changes to a given PSD)

4. Assets rendered out to PNG to include in the app bundle/publish to the static file servers - Git (because those files never changed after being published so the git history wasn't polluted with unneeded files)

I've also used LFS, which is... a fine workaround, but still not great. Users who don't have it configured can still commit binary blobs; users who don't have it configured will clone files incorrectly; if the LFS server is slow, unavailable, unreliable, then the system starts to behave oddly; you need a Git server that supports it.

It was a huge hassle to manage; having a system like this would have been a godsend at that company, and if I still worked there I would be spending all day importing our codebase and assets into it to see how well it works.

retroflexzy• 1 month ago

A significant part of my job, unfortunately, is helping people fix their workspaces when Perforce (p4) goes bad, or creating guardrails and wrappers to stop Perforce doing bad things.

In fairness, p4 predates most of the VCSes we consider "modern", so I empathize with a lot of the underlying architecture decisions. However, it has and continues to utterly fail at improving at a reasonable pace.

For example:

  - p4 tracks file metadata of client workspaces on the server (sync'ed locally, opened for edit, file revision, etc) and uses this as the basis to avoid doing unneeded work.  If this becomes desync'ed, a reconcile or force sync must be used.  A reconcile can take hours, potentially days; it tries do detect file moves by default, so likely at least O(c^n) for some c>1. I have never personally seen a default reconcile operation _complete_ over any modestly large game code base, and in practice, people accumulate a litany of workarounds and scripts to fix this for themselves.

  - Scripting p4 is a nightmare. Documentation is poor, schemas do not exist, and all the language-specific libraries are just thin wrappers over its C++ API.

  - By default, p4 "helps" you with text files by "correcting" line endings on sync or even converting between encodings. This works until you have a mixed-OS environment, and discover a part of the pipechain that _must_ have a certain style.  There are various levers to pull to make this better, but I've yet to find something fool proof.

  - By default, p4 keeps flies read-only, only unlocking them when explicitly marked as being edited.  This means, to avoid having to do this manually, every tool you use needs to be p4-aware.  Or, you can turn this off, and choose to contend reconcile instead. (See above)

  - Branching a modest game project, with, say, Unreal source code, can take hours.  And this is the quick version where you ask the server to simply create new metadata, with no file transfer to a client.

  - p4 is licensed by the user-account. Every user entity in p4 not intended exclusively for performing backups and maintenance operations counts toward this, including users required to integrate with other services.  Plus, often times, these integration users must have admin access to be useful. The security posture is horrific.

arka21474836472• 1 month ago

I’ll add some more

- The P4 cpp api was apparently designed before any modern Cpp std lib was available. And is at best archaic, and stringly to use.

- P4 encoding support is pain in the ass to configure. And ensist on adding or removing bom to files.

bryanlarsen• 1 month ago

p4 predates most of the VCSes we consider "modern"

p4 also significantly predates VCSes we consider obsolete. p4 is almost a decade older than SVN.

htr0waway2• 1 month ago

Scripting p4 is a nightmare

This is why I wish more command line tools were split into a library that does most of the work and a cli module for purely user interaction. Parsing stdout seems so unnecessary and could be avoided if a program could simply import a library.

seniorThrowaway• 1 month ago

I've had the misfortune of scripting an UE build with perfoce as the VCS, it truly is abysmal.

danbolt• 1 month ago

Reconciling is my least favourite part. It always feels like everyone’s checkout is a unique blend of local files and permissions you have to fix up now and then. It can be hard to keep track of how much one’s deviated.

I can understand how git isn’t always appropriate for less technical workflows and large file sizes, but p4 pain is its own character.

bananaboy• 1 month ago

I've been using p4 daily for about 21 years at various game studios (with a small break of around 5 years in the middle, where I used it only sporadically) and I think I can count the number of times I've used "reconcile offline work" on one hand. I think the only time my workspace has gotten into some kind of corrupted state was because of a crash that left it half-synced. If that ever happens I usually just blow it away and force sync the entire repo again rather than use reconcile (because it's so slow).

maccard14• 1 month ago

A quick reply;

p4 tracks file metadata of client workspaces on the server

Honestly, using P4 necessitates a plugin for $EDITOR of choice. The jetbrains IDEs do full on integration, but I managed a decade with an addon that just ran "p4 add/edit" on any save.

Scripting p4 is a nightmare

Agreed, but so is git, or plastic. My experience has been using the CLI with -Ztag is the way.

By default, p4 "helps" you with text files by "correcting" line endings on sync or even converting between encodings

This is a nightmare, and definitely one of P4's worst traits. Everywhere I've worked we have a presubmit trigger, to catch this.

By default, p4 keeps flies read-only, only unlocking them when explicitly marked as being edited.

See point 1, but honestly you go very very far with just p4 edit a whole folder.

Branching a modest game project, with, say, Unreal source code, can take hours. And this is the quick version where you ask the server to simply create new metadata, with no file transfer to a client.

A new stream including submission at my current job is about 20 minutes. Even at $BIG_CORP with 800 people working on a project, a branch didn't take hours. (Assumning by branch you mean streams. If it's a branch I have no idea, we don't use them)

p4 is licensed by the user-account

P4's licensing is predatory. It's very much "talk to us", but you'll find out that "talk to us" pricing is exactly the same for absolutely everyone. If you're really lucky, and get someone who is feeling generous, they might give you 1-2 extra automation seats. Given P4's security model is so old, I don't hate shared credentials for this. A bit like IAM roles, nobody should really have access to the credentials even if they were scoped appropriately.

throwaway81523• 1 month ago

Git is fine for text based files like code, but it's really bad at stuff like textures, 3D models, audio files, and other non-text files

Git-annex ?

throw2ih0206• 1 month ago

You gave me a flashback to that time I tried to use Git Annex on a moderately sized dataset and it resulted in a sysadmin at rsync.net personally contacting me to see if I needed help with whatever poorly written script was hammering their service.

In other words: No, absolutely not.

LinearIO• 1 month ago

P4 is also really well integrated into IDEs and UE Editor so that I don't need to think about it as much as I need, compared to Git. Locking assets, releasing them, merging into streams etc., is overall pretty streamlined. When it works, it's great, but when it doesn't work though, it's pretty hard to diagnose issues.

PaulDavisThe1st9• 1 month ago

One of the many points of git's design is that most of the steps you need to do in P4 are not necessary at all. You do not "lock assets", you do not "release them", there is no "merging into streams" equivalent.

The entire workflow with git avoids huge amounts of the cognitive load of using P4, which in turn means that integration with IDEs becomes much less important.

I worked with P4 around the time I first started learn git (coming to both from SVN). P4 struck me as "what a for-profit corporation would imagine a VCS should be like, if they'd never seen git". So glad to be far, far from that particular tool now.

jayd165• 1 month ago

P4 is more "industry standard" than "state of the art"... But it does handle large files and partial checkouts without feeling bolted on.

Hendrikto2• 1 month ago

Git has had native partial checkout for ages.

manoDev• 1 month ago

Git LFS is a major PITA, and if you use GitHub is even worse since there are quotas and rate limits that are charged separately.

ifwinterco• 1 month ago

One of those ideas that sounds clever in theory but in reality doesn’t work very well

arka21474836472• 1 month ago

Its important to understand that in Game Dev a ’git clone’, aka ’p4 sync’, can be a terabyte of stuff.

Git is bad at such volumes of binary assets, textures, models, sounds, etc.

cylemons• 1 month ago

Yikes, does that mean each dev has terabytes of stuff on their machine?

reactordev6• 1 month ago

Has nothing to do with Perforce being the Oracle of VCS because it’s baked into the big 3? Riiiight.

jhatemyjob3• 1 month ago

It's not even baked into Google anymore.

WorldMaker5• 1 month ago

Perforce is more the IBM of VCS. Older than it has any right to be. Has quiet "dark matter" support contracts with a lot of companies you wouldn't think need Perforce, but they've been using it for long enough they aren't going to change. Some of those support contracts included complicated forks and homegrown solutions that are so sunk cost as to be nearly black holes (and sometimes so different from baseline Perforce as to be evolutionarily different species).

computerdork7• 1 month ago

Well another factor could be that Perforce is a lot easier to use than Git - Actually, would like to think am good with git, but sometimes just wonder how it became so big considering the simple or important things (like check-ins and merges) are so complicated.

ultrahax• 1 month ago

Can confirm, we have a team dedicated to the care and feeding of p4.

raincole• 1 month ago

One thing I don't like about Git LFS is that there is no way to delete very old history. It's the 'git spirit' to not allow deleting history, but in the context of LFS it sounds horrible. Especially if you use Github.

If there is an asset that is updated very frequently in the early stage of development, you'll be charged for all the storage for the rest of the repo's life. That happens a lot in gamedev: most assets go back and forth early on but once it's done no one will touch them ever.

Marsymars2• 1 month ago

One thing I don't like about Git LFS is that there is no way to delete very old history. It's the 'git spirit' to not allow deleting history, but in the context of LFS it sounds horrible. Especially if you use Github.

At my dayjob we used Git LFS for a bit, but foud it unworkably clunky - we eventually found it easier to just make a separate "LFS" repository and add it as a submodule to the main monorepo. Now we can rewrite the history of the LFS repo on an as-needed basis.

paulddraper2• 1 month ago

that there is no way to delete very old history

That's one of the features of Git LFS is separately managed storage and lifetime.

You are correct that GitHub does not offer that feature.

doctorpangloss• 1 month ago

Git LFS is unusually bad. I think the maintainers are trying their best but it is one of those things where there are so many bugs and so many users that they will claim they are overwhelmed with LLM authored PRs instead of conceding that they own a very valuable piece of real estate (being installed alongside git by default) and totally mismanaging it.

a1o3• 1 month ago

I think you can delete the history but it’s hard (more than three steps), at least that was my conclusion from the e blogpost below

https://janetgilbert.net/staging/2026/04/27/dealing-with-old...

IshKebab7• 1 month ago

this is not intended to compete with Git for general software development. This is a competitor with Perforce for game development.

Well, it is intended to compete with Git for version control. It's just that Git happens to be so bad at some aspects of version control that it isn't used much in those cases.

There's no good reason that Git couldn't be good at versioning binary files, or splitting up large projects. I mean people have tried - there's LFS and submodules. It's just that those both suck balls.

I was kind of hoping Jujitsu or Pijul would take a stab at these major Git deficiencies but unfortunately it seems like they are content to do them as badly as Git does.

WorldMaker5• 1 month ago

It feels like a Pareto Principle problem. 80% of source control is text files that can be three-way merged as text files, but a lot of hard problems are in that 20% that isn't.

Git does very well at the 80% and with tools like custom merge tools and git lfs/annex and git sparse "cone" checkouts can get pretty close to hitting the 90 or 95% case.

But yeah, so many of those extra tools in that 80 to 90% area are awful to work with because they aren't the default, aren't out the box, are hard to configure and get right. Partly because it always seems like there will be a gap in that 95%-100% window and partly because the use cases that need that 80% to 90% often are only "just 10% of use cases".

(Which is also to say that to survive Jujitsu and Pijul and others seem to have to work to make sure they handle the 80% base case extremely well just to compete with git, they haven't necessarily time to think about the 90% or 100% problem.)

(ETA: And also relates to why game development seems to feel the 20% cases more, because by volume of data game development is certainly closer to a flip of the 80/20 sides with 80% or more large binaries by volume.)

j1elo• 1 month ago

There's also the virtual streams (branches). A mapped overlay over the actual contents of the repo, which you can check out and get a subset of the contents. Or even can provide different files that have been mapped in the server for each particular virtual branch.

This is used so e.g. an artist gets a repo that contains sources for the art assets, while a programmer gets the same repo but instead of art sources, it downloads the already produced binaries. As a SE, you just want to build the code, and don't care about 800 GB of art asset sources.

slowmovintarget• 1 month ago

Seems to me that this would be great in general software development for just checking in your dependencies as binaries. No more build-time supply chain attacks with vetted binaries.

throw0101c• 1 month ago

Git is fine for text based files like code, but it's really bad at stuff like textures, 3D models, audio files, and other non-text files that game developers need to collaborate on.

How well does Mercurial work in this situation (or even Subversion, given that Perforce is non-distributed like SVN)?

minraws2• 1 month ago

Mercurial is a bit better, but not by much it had slightly better large file support historically but I wouldn't recommend it.

DonHopkins• 1 month ago

Git LFS causes nothing but pain, suffering, and regret.

GabeIsko3• 1 month ago

I braced myself for this, and instead found that it worked surprisingly well for my uses. I only manage 4 gigs though.

Calgaryp• 1 month ago

I totally agree. I worked with Unity for a small indie game and it was sooo hard to use git with my team and merge things like game scenes etc.

bc1000003• 1 month ago

Confirmed. Perforce is (was?) the industry standard for most larger game teams. If you've been in the business for any length of time you're probably pretty familiar with it and all of its many warts. But Perforce does get the job done and it's reliable and stable.

However combining some of the flexibility and workflows of git with the ability to deal more efficiently and effectively with large asset files is something that a hell of a lot game engineers would be interested in. And having the virtual checkout as a feature out of the box for folks used to half terabyte repo sizes is definitely a huge plus.

This announcement is definitely a big deal, and if the promises for lore actually measure up, we could be seeing the beginning of a switch over to open source version control for larger asset heavy games where git was still not a great fit.

Will be interesting to see what the public code hosting platforms do (github), and whether there will be any major structural changes to git (I'm thinking not likely). I wonder if Epic will make a play to take business away from github or gitlabs.

fsfod• 1 month ago

There is work being done on Git to add pluggable object backends[1] by some GitLab devs that could changes things up a bit and make large object handling not suck Maybe Lore could act like a promisor remote to pull large objects from as well for interop.

1: https://gitlab.com/groups/gitlab-org/-/work_items/15061

maccard14• 1 month ago

Perforce does get the job done and it's reliable and stable.

Well… perforce does get the job done, but it does need a lot of hand holding from an admin (and you need someone who knows p4 server if you’re using it). It’s also stable in the sense of “it has exactly the same feature set as it did 15 years ago, and branches (streams) are considered modern”.

MayeulC• 1 month ago

Ah, that's interesting, the requirements are similar in the CAD industry. Dassault Design Sync is used a fair bit for semiconductor design databases, for instance. An open alternative would be welcome!

Edit: I do feel a bit uneasy about epic games, though.

apatheticonion2• 1 month ago

Another thing git isn't good at is massive codebases with long histories. IMO git should have a configuration option to pull commits lazily without the need for `--depth` and the `git fetch` dance that goes along with it.

woctordho2• 1 month ago

There's `--filter=blob:none` and it allows to automatically fetch blobs when needed.

woctordho2• 1 month ago

It's 2026. Historically the way for large binaries in git was git LFS. Now the way for large binaries in git is just git.

StellarScience• 1 month ago

10 months or so ago I believe HN posted "The future of Large Files in Git is Git" : https://tylercipriani.com/blog/2025/08/15/git-lfs/

So is it the future now?

left-struck• 1 month ago

Can this be used for Solidworks models? PDM is so awful

throw2ih0206• 1 month ago

Yeah, engineering is another use case for Perforce, will be interesting to see if Epic pursues that market too. I know UE has some use in industries like architecture and professional simulators.

toobulkeh• 1 month ago

Why do you need to lock anything? To prevent someone else from working on it? To prevent duplicative work? That sounds like a management/coordination issue. Not a VCS issue. So you have version A and version B. Why are 2 artists working on the same asset without coordination? Do they not use task tracking or project management? Even if that’s ignored, then one person’s work gets wasted. That happens all the time in text based dev too if it’s lacking coordination.

Should coordination live in the VCS layer? That sounds like it creates more issues than prevents.

lentil_soup4• 1 month ago

Because it's not so cleanly cut. Editing a level might touch several files without you realising, you might move a mesh file which causes 100s of references to update across 100s of files.

For example, you want several people collaborating on a level. Say a level designer, an artist, a lighting person, audio. Usually you split this into several files to allow for the collab but you're bound to hit moments where you need to edit across the board. The level designer moves an area which causes the light, audio, meshes to move. If you don't have a way of knowing someone else is working on those files you're going to step on a lot of toes.

Whether this should live separately from source control is a fair question, but it's just a simple place for it as it's tied directly to the files.

Also, the outcome of a mistake is quite high, it could mean a whole day or two (a week?) of wasted work because you didn't realise someone was also editing the same file.

tikotus2• 1 month ago

I like putting it like this: VCS can be centralized or decentralized, and it can version per file or per commit. For games you want centralized per file versioning, like Perforce. Git is decentralized per commit versioning.

maccard14• 1 month ago

I mostly agree. I think for games you want per commit versioning 95% of the time. You don’t _really want developers building with cherry picked old revisions etc, but you don’t want an artist to have to pull down the 8TB of new art content that was submitted in the last 48 hours to update a texture.. the only way to do that is per file versioning!

adamnemecek• 1 month ago

This looks really good. I have been using git to store some PDF (tens of GBs) and git is really not well suited for this. No GFS is not a solution.

20k• 1 month ago

The incredible laggyness of that website does not inspire confidence. Much of the text selection is also broken, and chrome consumes nearly a full core trying to render.. something?

Its remarkable that anyone thought this website was fit for release, and it gives off strong slop vibes

I also have absolutely zero trust in a product like version control being provided by a for-profit company. It seems like a terrible idea to tie your software stack to Epic Games of all people, given their track record

jon-wood2• 1 month ago

This is primarily going to be targeted at Unreal Engine users, for whom the source control tooling they use is the smallest of concerns when it comes to being tied to Epic Games.

dankobgd2• 1 month ago

never trust epic

CamperBob2• 1 month ago

Why not?

dankobgd2• 1 month ago

They will kill you like they did unreal tournament. Imagine if valve killed counter strike, literally they are named after that game (unreal gold, ut99, 2k4 etc) and they killed the ut4 version. Like it's so hard for them to hire 5 people to work on it, but guy can buy whole forest for himself to "save trees"

interpol_p• 1 month ago

Their docs seem entirely LLM written. It seems especially obvious in the FAQ. While I'm not against using LLMs for writing assistance, they've left a lot of the unnecessary language and typical stylistic choices in there, which erodes my trust in the project a bit. Perhaps it's a very good game-oriented version control system, but the lack of human attention on the docs makes me wonder how much they care

fwip• 1 month ago

Yeah, I hope I'm wrong, but it feels a little bit like "planned project no longer has internal support, let's see if we can make it open-source to garner goodwill and recoup some of our investment." Which isn't the worst thing that can happen.

coldpie• 1 month ago

I reluctantly agree. I was interested to read the system design doc[1] but it's so many pages long and so full of redundant statements and needless details[2] that I gave up a couple sections in. The numerous "it's not X, it's Y" constructions give the game away. If they can't be bothered to read these docs themselves, why should I?

Anyway it's probably fine software and I am genuinely going to give it a shot for a usecase I have involving large image files. But the LLM-generated docs don't inspire confidence.

[1] https://epicgames.github.io/lore/explanation/system-design/

[2] They literally have a section header "10.1 Revision state as a 320-byte fragment". The byte size isn't even relevant in the code as an implementation detail, much less belongs in a design doc. No one read this doc before publishing it.

tom_3• 1 month ago

Yeah, I agree about the docs. I started on the system design page and my head started to swim after about 5 minutes. So exhausting to read!

On the flip side, I expect the project itself will be workable - well, assuming they're actually using it themselves! UE is a big pile of Stuff on its own, and Fortnite must have god knows how much additional crap in there, so if this is (or will be) their replacement for Perforce internally, then it'll be getting a good deal of testing. (If they're just chucking it over the wall, though... well, sheesh, you first...)

(Perforce is the standard thing for games, and pretty well it works too, and hopefully this will deliver it a well-needed kick. It was sold to private equity about 10 years ago, and it feels like they've been coasting ever since. (Perhaps users in other sectors are happier though?))

BoggleOhYeah• 1 month ago

It’s great to finally see a possible alternative to Perforce.

Lucasoato2• 1 month ago

Git’s content-addressed revision graph is excellent, but it treats binary files as second-class citizens—large files require bolted-on LFS rather than first-class chunked storage, sparse checkouts have sharp edges in offline use, and there is no native multi-tenant isolation.

I'm trying to figure out what Lore can accomplish that git+LFS can't. I've read about big binaries chunking, native interface and permission, is there anything else? Weren't those problems already solvable in the git+LFS ecosystem?

zipy1243• 1 month ago

If you've used git+LFS for any extended period of times, you'd know how often it breaks, especially when used with forges like GitHub. Both GitHub and Git treat LFS as an after-thought and second class citizen.

Lucasoato2• 1 month ago

Can you tell me where did it break? Or what feature you wished it had? I’m just curious, still trying to form an opinion on this.

frollogaston12• 1 month ago

The fact that it's even referred to as git+LFS instead of just git... If I needed to work with large files frequently, I wouldn't want such a basic feature bolted on. Not a criticism of git, just can see why Epic doesn't use git.

bob1029• 1 month ago

I've used git+LFS for unity projects without much issue. If I was working with more than 2 people, I would definitely reach for something centralized.

iceweaselfan44• 1 month ago

fully open source look inside Lore Desktop Client is available as binaries only, download the installer for your platform here:

Someone• 1 month ago

There’s a “Repositories” link at the top of the page that links to https://lore.org/#repositories, WhyHow links to various GitHub repos, including https://github.com/EpicGames/lore, which claims to have code for the CLI. I see no reason to suspect that claim is incorrect. The code likely lives in https://github.com/EpicGames/lore/tree/main/lore-client)

lentil_soup4• 1 month ago

to be fair, that's just the desktop client. You can use or build on top of the CLI

they do say they will open source it, but who knows:

"It isn’t open source yet—it currently depends on some proprietary components, including Epic’s internal design system—but we’re committed to open-sourcing it in the future"

KomoD• 1 month ago

to be fair, that's just the desktop client.

I expect fully to mean fully, though.

O5vYtytb• 1 month ago

https://epicgames.github.io/lore/roadmap/#desktop-client

Open-source the desktop client so the community can build on its full graphical experience, not just download it. An early desktop client already exists as a binary download, but it isn't open source yet — it depends on some proprietary components, including Epic's internal design system. We're working to make all of it available in the open so that the client can ship as source alongside the rest of Lore. Lore is an open project, so it is important that the desktop client — which will be one of the main ways many people will interact with Lore — is also fully open so that the community is free to review, extend, and shape it.

Wowfunhappy3• 1 month ago

Thanks. I still think it's a bit weird to say "fully" open source while your flagship client is currently closed. I realize they're referring to the VCS itself, but—well, if they just dropped the "fully" for the time being, I wouldn't bat an eye.

dwroberts2• 1 month ago

This is just going to become another way to lock developers into UE. Then they will start charging for licenses, same as Unity did for its versioning feature. It might be open source but that doesn’t stop the commercial use of it being charged for.

shepardrtc• 1 month ago

It has an MIT License

dwroberts2• 1 month ago

It doesn’t matter

https://en.wikipedia.org/wiki/Open-core_model

penciltwirler• 1 month ago

The premise is that Git-LFS sucks, so we need to build a new data versioning system (in Rust, from scratch). While I mostly agree with this premise, but there are already lots of existing (mature) data versioning systems with the same tricks under the hood:

- Pachyderm (Go): https://github.com/pachyderm/pachyderm

- XetHub (acquired by HuggingFace): https://huggingface.co/blog/xethub-joins-hf

- LakeFS (Go): https://github.com/treeverse/lakeFS

- Oxen (Rust): https://github.com/Oxen-AI/Oxen

I guess with AI, anyone can vibe code a content-addressed, chunk-level deduped, versioning system in Rust these days...

But jokes aside, Lore seems really cool! What's interesting is the realization that different domains/industries have similar problems, but they don't seem to be cross-polinating. In this case AI and Gaming both need a storage system that can version control large binary files at scale. I think there's lots of opportunities to share ideas here, but perhaps the lack of idea sharing (currently) creates opportunity!

LtdJorge6• 1 month ago

I don't think the needs are exactly the same. I believe in AI the big binary files are normally written once, while in gamedev, they are constantly updated.

That already warrants different storage architectures.

dimatura2• 1 month ago

There's also git-annex and iterative DVC. I used xethub a fair bit (was the earliest user, in fact) and I thought it was better than git annex, git-lfs and DVC, but still did start to struggle past a certain size. I think part of the problem was just git itself, and the compromises required to have a hybrid repo. So I'm happy to see this vcs doesn't use it. xethub did start shipping a version of their product that did not use git but I didn't get the chance to try it. I've also tried oxen and it wasn't bad at first, but soon ran into some weird issues with the repo state which I didn't really try to debug. It is clear to me at this point, given my experience with all these systems -- none of which I've been 100% happy with -- that "git for data" is a nontrivial problem.

gschoeni• 1 month ago

one of the oxen engineers here would love to hear about anything you ran into on the os product or platform! we've grown the team a bunch and are eager to learn what your perfect vcs looks like

ozkatz• 1 month ago

Curious if you’ve had a chance to try lakeFS?

It was designed with large-scale environments in mind. I’m aware of several deployments managing hundreds of petabytes of data and billions of objects, which is why lakeFS does not use Git’s Merkle tree / directory tree approach.

Disclaimer: I’m one of the project’s co-creators.

kardianos• 1 month ago

I like everything I've read on this site so for, for it is also something I've been wanting.

If the roadmap's "Web client and code review tools" could replace gerrit for me, this would be a easy switch.

Moreover, it looks like they designed both the mutable store and immutable store to be able to easily store their state directly on an s3 like system.

There are a number of features that would greatly speed up CI/CD system operations I belive.

eblanshey• 1 month ago

This looks very cool! I maintain a FreeCAD workbench for 3d model version control[0], and it currently uses git as the VCS, simply because that's what I was already using. Thinking long-term, I see it eventually morphing into a broader PDM (Product Data Management) system, perhaps even PLM (Product Lifecycle Management). Lore has a lot of the requirements already built-in, like centralized locking (better than SVN), and it's better suited for for binary files. I implemented the git backend as a protocol/port so it'd be pretty easy to swap it out. I'll be watching Lore closely.

[0] https://github.com/eblanshey/HistoryWorkbench

ex-aws-dude2• 1 month ago

Interested in this as perforce is pretty terrible a lot of the time

It’s like anything you do has to talk to the server

Even something as simple as diffing a file will just hang if there are server issues

frollogaston12• 1 month ago

"Full-surface API" is a feature nobody here has mentioned. Is that a dig at how git intentionally has no linkable library? I saw this earlier https://news.ycombinator.com/item?id=48470604

ibejoeb10• 1 month ago

I don't know if it's a jab at git. Git does have `porcelain` to accommodate programmatic interaction. It's not linkable, but it's still an API.

wky• 1 month ago

The link to Architectural Decision Records is empty, but they're present in the repo to look at[0]. Curiously the decision with the most deciders is the implementation of JavaScript bindings[1].

[0] https://github.com/EpicGames/lore/tree/main/docs/developing/...

[1] https://github.com/EpicGames/lore/blob/main/docs/developing/...

nyxtom• 1 month ago

I came here hoping Epic Games somehow had launched a reliable alternative to GitHub, but saw their code is hosted on GitHub

manifoldgeo2• 1 month ago

Yeah, they should have posted it to LoreHub! I just checked for the availability of lorehub.com, and you can buy it for only ~$13,000 and start a competing business to GitHub.

yablak• 1 month ago

does anyone have a proper comparison of binary control systems like lore, xet, etc? i'd love to see how it handles mixed case workloads.

gonomodagast• 1 month ago

Why not just use Alienbrain?

advisedwang• 1 month ago

Nice, this seems sort of like Git-but-for-giant-monorepos. That has been a gap in the opensource VCS market

noopprod• 1 month ago

Kind of funny that it's on GitHub no hate.

Missed opportunity for Lorehub.

runevault• 1 month ago

Not really. Lore is for gamedev at companies and so not a distributed VCS. Using it for an open source project if you allow outside PRs makes no sense.

tlahtinen• 1 month ago

This is a very promising announcement for Unreal game development specifically. For any other purpose I wouldn't care as much.

Perforce definitely needs a challenger. It is not the incumbent because it is particularily simple to use or administer. Git is actually way simpler when it comes to branching operations for example.

The reasons why p4 is often preferred in gamedev have already been mentioned in other comments: large project support, permissions, file locking and so on. Another key reason p4 is the king for Unreal dev is just how well it's supported inside the engine. Not perfect, but it's the best supported VCS because it's what Epic uses. Even the Git plugin is painfully unfinished, because Epic does not internally use it. So with Lore I expect them to give it first class support. I'd recommend Git a lot more if the support in Unreal was better.

(background; I've been in gamedev for almost two decades now, 2-200 person companies, every kind of engine and version control system. I prefer git where I can use it: for Unreal that means small projects and/or tech savvy team members. Pick the tool that is right for the job and the team.)

jayd165• 1 month ago

For what's it's worth I've been brushing up the plugin a bit. The odds of it getting merged in are even less likely after today, tho.

https://github.com/EpicGames/UnrealEngine/pull/14630

boppo1• 1 month ago

What about blender?

applfanboysbgon3• 1 month ago

I'm in the market for a better VCS designed for gamedev-specific concerns, but reading the system design doc, it's LLM-generated. Not exactly confidence inspiring.

gavinhoward4• 1 month ago

As someone who has thought a lot about VCS design [1] [2], the chunking approach is the wrong one and will still waste space.

[1] https://gavinhoward.com/uploads/designs/yore.md

[2]: My WIP VCS has been named Yore for at least two years; I did not copy Lore's name.

MattRix• 1 month ago

This is a very long document that says nothing about chunking at first skim. If chunking is actually wrong, then just explain why, here. Wasting space is not actually a problem if it’s optimized for other purposes instead.

gavinhoward4• 1 month ago

When it comes to large assets, wasting large chunks of space is a problem. If your chunks are 64 kib average (from the Lore document), but changes only average 1 kib (which could be a high estimate), then you will still run out of space 64 times faster and need to read 64 times more data off of the disk for certain operations.

It also makes diffing hard, as well as diff viewing.

qznc• 1 month ago

I wrote down some thoughts about a "next generation VCS" in 2019: https://beza1e1.tuxen.de/monorepo_vcs.html

Back then I concluded that it will probably never be built because OSS projects don't need it. Maybe this is changing now as AI allows for larger OSS projects.

hparadiz5• 1 month ago

As long as Epic Games is anti Linux I will never use any Epic Games product.

Count on it.

TiredOfLife• 1 month ago

They are so anti linux that they ship a linux version of Lore.

LtdJorge6• 1 month ago

Well, that's easy for a Rust binary. But they don't put any effort into having a great UX for Linux devs with Unreal Engine, for example. It barely works on Linux and is almost impossible to run under Wayland.

ibejoeb10• 1 month ago

I'm not a gamer so I'm entirely ignorant of the situation. What's the root of your objection? The games are proprietary, right? So I figure it's not a Stallmanesque position. Is it a big problem to run games on Windows?

hparadiz5• 1 month ago

I don't have a Windows install anymore and won't set it up just for one game. Furthermore Epic Games has a history of acquiring studios and them crippling Linux support on games that were already purchased. I have a long list of games that I happily play on Linux with no issues.

pkasting• 1 month ago

The idea sounds good, even if Epic's recent track record of tools is not inspiring. But the commit messages etc. are very clearly products of vibe-coding. And version control is not the situation where "works 97% of the time" is a good-enough bar.

Passing for now.

frollogaston12• 1 month ago

About the recent track record, is there some technical problem or just drama with Unreal Engine that I'm not aware of? (I already have the same opinion about using AI-coded VCS.)

boredatoms• 1 month ago

Would lore be good for a tech company monorepo?

Imustaskforhelp• 1 month ago

What sort of scale are we talking about when mentioning tech company's monorepo though?

rustyhancock• 1 month ago

Guess this is announced as part of epics state of unreal and if so this is already off to an amazing start even if this is all there is!

Surac2• 1 month ago

Isnt that what SVN is good for?

sedatk• 1 month ago

Not immutable. Not good with binary files.

Surac2• 1 month ago

Interesting. Was‘nt aware of that immutable part. I use it with big bin files for year. Maybe i was lucky

ksec• 1 month ago

Turns out it is not really new but only open sourced it now. From the FQA.

Lore, formerly called Unreal Revision Control, is the built-in version control system for UEFN (Unreal Editor for Fortnite), where creators have been using it to version their islands. It is also seeing progressive adoption by internal Epic teams, and is being implemented as the backing store for UEFN’s cook pipeline, where it replaces traditional intermediary storage layers—eliminating redundant file transfers and significantly reducing the time between publishing changes and those changes being playable.

Surprised it is in Rust and not Epic C++ or Verse. I wonder why.

wrwills2• 1 month ago

I suspect the use of Rust rather than C++ might have something to do with the fact that Simon Peyton Jones and Lennart Augustsson (both of Haskell fame) both work at Epic and there would have been a strong internal push to do this in a language with some functional programming features. Rust rather than Verse because that would probably not be the right tool for the job (even if Simon works on it). Rust rather than Haskell probably because of performance -- DARCS never caught on partly for performance reasons.

frollogaston12• 1 month ago

I don't see how Rust is more functional programming oriented than C++

LugosFergus5• 1 month ago

It's a separate tool from Unreal Engine, so you don't need to constrain yourself to its conventions. You're also not going to use UObjects, its reflection layer, serialization, and all that other stuff for a tool that's strictly for version control. Plus you're going to tie yourself to all the issues and slowness of the engine. Epic made this mistake with the Epic Games Launcher, which is essentially an instance of the engine running, and a huge source of various issues.

As for Verse vs Rust, Verse is used in UEFN experiences, but it is _far_ from any kind of production state. It's also intrinsically tied to UEFN. It's not like you can download a standalone compiler or REPL for Verse.

yjftsjthsd-h2• 1 month ago

Honestly it inspires rather more confidence in me to see a tool that has actually been used for a while and is just now being released for public use as well. Unless, of course, it's being released because they've decided to stop development, but I don't think that's the case here.

aayushprime• 1 month ago

I must say lore is an awesome name for a version control system. Much better than git in that regards.

wxw• 1 month ago

https://epicgames.github.io/lore/explanation/system-design/#...

Helpful page that gets closer to the details

armchairhacker• 1 month ago

Game engine, programming language, VCS…will Epic launch a brand new OS?

gregschoeninger• 1 month ago

We're also working on an open source large asset versioning tool called "oxen" - https://github.com/Oxen-AI/Oxen

Would love any feedback on it or contributions if people are interested :)

rane• 1 month ago

The excessive emojis in the README don't inspire confidence

swiftcoder3• 1 month ago

I don't see a workflow for locking assets while they are under modification. This is kind of important for assets? Since we don't really have great merge workflows for meshes/animations/sounds/etc.

I also don't see any sort of GUI client? So the whole art team is going to have to get up close and personal with the CLI

ibejoeb10• 1 month ago

Locks: https://epicgames.github.io/lore/faq/#does-lore-support-file...

GUI: https://epicgames.github.io/lore/faq/#does-lore-have-a-deskt...

swiftcoder3• 1 month ago

Oh, that's good news

akurilin5• 1 month ago

We had to use Perforce (Helix Core Cloud) at my last game studio, and it is the de facto industry standard that most of your creative staff is already familiar with. The programmers don't love it, but they don't rule the roost in games. It's also the safe, verified default for working with Unreal Engine 5.

It does show its years though. We were one of the first users of the Perforce cloud offering, as we were small and didn't want to self-host ourselves, but it was a bit of a rickety experience. You had to register an Azure account in order to be able to access the service, and you had to ask support to modify things like triggers. Coming from the world of GitHub and other SaaS products, you could tell it was an attempt to retrofit an older model into a new skin.

The Git LFS path has some unofficial support as well, but you are on your own when things go poorly. Epic doesn't provide much help there.

Competition in this space is welcome, especially if they're planning to make it fully officially supported by the Engine.

I wrote about why merging files isn't as common in the world of game dev for folks coming from the world of text: https://www.kuril.in/blog/why-game-devs-dont-merge-files/

dijit9• 1 month ago

ditto; and UE5 on anything that isn't perforce is a lesson in pain.

I just took over a team that was using Git, and yes, I know it's everyones favourite VCS but for Games it's just about the worst thing available. I could measure art reviews with git in hours, now with perforce its seconds. I wish I was joking.

All the interesting tools that UE5 uses (Horde/UBA for a clean example) will require perforce.

but, Perforce hasn't done anything with its industry position. It's expensive as hell and they don't have operational costs related to hosting (you have to host it yourself, and honestly, for performance reasons you really want to- despite it being a real pain to maintain passed first install). There's some echoes of stuff they try but they have absolutely no solid direction and nearly everything they've been doing cuts against common sense or their userbase, while their core product keeps getting renamed but no actual improvements.

It's a lesson in how proprietary software is really a prison.

I wish I could use a better code review tool than Swarm.

I wish I could integrate SSO without weird LUA hooks which cause segfaults on my machine (enough of them causing a perforce deadlock).

and I wish I could run a distributed storage backend instead of relying on a big fat SSD and journalled backups that can't be restored because licenses are tied to the IP address of the main server.

It's forgotten technology, and the company that operates it is a zombie.

maccard14• 1 month ago

I completely agree with every single point you've said here. Running a P4 server is an exercise in self torture, but it's the only option with UE5.

I wish I could use a better code review tool than Swarm.

With one caveat. I think swarm is a great tool. It could do with being easier to configure, but as a code review tool it's simple, straightforward and gets the job done.

I wish I could integrate SSO without weird LUA hooks which cause segfaults on my machine (enough of them causing a perforce deadlock).

this is about the only thing that Perforce has actually managed to do in the last... 5 years? [0] It's also woefully awkward to set up and operate, as is everything P4. But it does work.

[0] https://help.perforce.com/helix-core/integrations-plugins/he...

akurilin5• 1 month ago

Couldn't agree more. I wonder why the company stalled as much as it did. To be fair they've been around for three decades, maybe it's just hard to keep momentum for that long, especially if you're in a dominant position with no competition coming after you.

s1ncere• 1 month ago

This a lot of very old and out of date information and opinions, respectfully disagree

jayd165• 1 month ago

UBA mostly doesn't need Perforce and Horde isn't as good as the GitHub, gitlab etc.

UGS (the tool that pulls down the latest engine version for artist) is pretty perforce centric.

munificent4• 1 month ago

I really like your article. It does a good job of explaining not just the technical differences but the way those affect the surrounding development culture.

akurilin5• 1 month ago

Thank you! It was a bit of culture shock for me when I first experienced it, coming from the world of business software, and I figured others would find it equally fascinating to learn more about.

superxpro12• 1 month ago

Git LFS and git's (relatively) new sparse clone features i suspect are their answer to this sort of thing, although my understanding is that was more focused on monorepo operations in general. I'm not entirely sure permissions were sorted, or this sort of mixed-mode dvcs/cvcs operation model with file-scope checkouts interacting with traditional branch-mode operation.

ex-aws-dude2• 1 month ago

Its not just about the tech, you have to have non-technical artists using this thing every day

bentt2• 1 month ago

There was and still is a very nice competitor in this space called PlasticSCM. They were bought by Unity a few years ago. Unity has not been a good steward. They should have done what Epic is doing and open sourced it. But instead they chose to give it P&L responsibility. Curious what it's contributing to their financials.

gmueckl3• 1 month ago

Seeing PlasticSCM wither like this is sad. I had rather high hopes for this system to successfully compete with Perforce, but history worked out differently.

robaato2• 1 month ago

Plastic is usable on surface, but doesn't scale well. If you've ever looked at what Plastic does when you use its git fast-export compatible feature, the results for merge commits are horrible to behold, e.g. delete of a directory followed by add of a file into same dir, then delete of it then rename.

bentt2• 1 month ago

I’m not picking up how its git export feature maps to its scalability.

goolz• 1 month ago

Fantastic. Every time I mix git and games I end up almost having an aneurysm.

jbverschoor2• 1 month ago

Too bad it does not support fully local/detached (without server). It would be nice to be able to run it similar to a local repo without remote

dijit9• 1 month ago

to be totally fair, perforce doesn’t either.

There are some awkward workarounds (officially supported ones) which create a local server and you then merge with a remote server upstream, but as you might expect: its awkward and practically nothing supports the workflow.

ibejoeb10• 1 month ago

The unique feature is universal locks (like cvs checkout) for opaque binaries like art, which requires universal state, so this seems to be orthogonal to the goal.

jbverschoor2• 1 month ago

Myeah I understand, buy i generally enjoy just creating a local repo. Of course, not what Epic does ;)

adastra222• 1 month ago

Why?

MarvinYork• 1 month ago

I wonder how many websites ChatGPT did by now...

the_duke• 1 month ago

Does this support using S3 as the backing store?

That would be very powerful for various use cases.

samgranieri2• 1 month ago

How about the last pre-enshittified version of Minio?

ibejoeb10• 1 month ago

lore stage covers adds, edits, and deletes — you use the same command for all three. Stage a deleted file and Lore records the deletion for the next commit. Moves and renames are tracked too, through a dedicated subcommand: lore stage move <from> <to> records the rename so the file keeps its identity and history across the move instead of registering as a delete plus an add.

Oof. So this isn't compatible with any tools that move or rename files. I can't see how this will be acceptable for real-world use.

xboxnolifes2• 1 month ago

Isn't this just the same thing as `git mv`?

ibejoeb10• 1 month ago

I don't think so, but I'm not sure. It seems to imply that renaming a file without using `lore stage` subcommands will record a delete and an add, meaning that changes made in a branch prior to the rename won't automatically propagate. Git will detect this without using it's own mv command.

IshKebab7• 1 month ago

They're actually saying it's better than `git mv`, because it actually records the move, unlike Git.

samgranieri2• 1 month ago

I may not be the first to bring this up, but seeing the name of this program reminds me of Data’s evil brother

nicebyte• 1 month ago

"Github Repo" "Find Us on GitHub"

????

wilg• 1 month ago

I've tried very hard to use Git (with LFS) in both Unity and Unreal and its just been a total disaster of problems for me and the team. Hopefully this can help.

MaikuMori• 1 month ago

I've been working on something like this for internal use for the last 2 month or so. Very high overlap, CAS, version chains, branching available in storage, but not implemented (not as high value for us), file locks, chunked storage on S3 api. Focused on assets only with more focus on build/export (think gh actions for blender files) and QA processes.

Interesting to see where this goes.

Qiu_Zhanxuan• 1 month ago

Is it me or I read in the docs "Design System" section that it was made in Scala then I went to double checked again an hour later and it said Rust

Melatonic6• 1 month ago

Guessing Tim Sweeney is a Star Trek fan?

Melatonic6• 1 month ago

Has anybody used Lore (or the older Epic internal version they were using on Fortnite) for also managing code?

As someone who came from VFX I always found the way Git works to feel very limiting. I feel like it was one of those things that was just so much better than the previous option (SVN, etc) that it blewup without actually being ideal.

nogbit• 1 month ago

Any multiregion deployment requires DynamoDB. Shame this solution is specific to AWS.

dijit9• 1 month ago

yeah, that's actually crazy.

I don't know a single AAA (or even AA) studio that hosts Perforce in Cloud at all.

Perforce usually starts out life in a gamedev studio as running on "a spare desktop" which then eventually gets promoted to a real server in a closet.

The most important thing for Perforce is disk, memory and above all: bandwidth (of those things and the network too!); it's also quite latency sensitive since all operations touch the server. So running it far away is never a good idea.

There was some complicated perforce architecture inside Ubisoft to account for this: AAA tends to involve a lot of co-dev studios, so the lead studio would get the commit server and others might get replicas (and you could feel to pain)- but all the perforce instances were actually on the same machines and would replicate to one-another anyway: each project was given a port for the perforce fleet. Really bizarre.

but, that's besides the point. Even the tiny studios run their own locally hosted VCS machines.

HellDunkel• 1 month ago

Why is this ranking #1 on HN? Seems quite niche to me.

justinhj3• 1 month ago

One of the reasons I left the game industry, or at least one the benefits of leaving was to not have to use Perforce any longer. Albeit a necessary evil, hopefully lore is better.

I always wondered if a combination of a database to index assets and asset versions with the data being in S3 style storage would be better for game assets; code can be in Git. Then your workflows are easy to build on top of that.

thsbrown• 1 month ago

Isn't this what git lfs is effectively?

justinhj3• 1 month ago

The main difference in my proposal is separating git completely from asset management and replacing it with a flexible, scalable system built for that purpose

scosman• 1 month ago

Top right: link to the Lore Repo on Github

armcat• 1 month ago

Can this be used for ML/AI projects as well? I'm thinking for version controlling LoRa finetunes, finetuning data (which can consist of text, images and audio), safetensors, etc?

dimatura2• 1 month ago

Of course it can. Is it good at it? I guess we'll find out, for sure people will try (I might if I get some spare time). At a glance some of the tech used resembles that of xethub (of which I was a user, until they got acquired by huggingface), which was marketed towards AI/ML/data science (to version control data/models/etc).

forrestthewoods2• 1 month ago

Frickin sweet.

Perforce is a necessary evil in games. Git is simply not suitable. So this is already a huge win.

I main that Git is - at absolute best - simply mediocre. It's not a "good" VCS tool. It is merely adequate.

I would love to see Epic go for a full Git killer.

It looks like Epic has a created a virtual file system interface that is cross-platform. FRICKIN SWEET. This is a critical component to making a VCS tool that doesn't suck. And there has not been an existing open source solution to it.

rkagerer• 1 month ago

Is it ironic the first thing I noticed on their landing page was "Find us on GitHub"?

They need a FAQ entry "Why isn't Lore hosted on Lore?"

sasham• 1 month ago

The issues they want to address are already solved in Diversion (https://diversion.dev). Granted it's not open source yet, but that isn't usually a priority for game/entertainment studios - having a complete platform is.

After tearing it apart for 2 hours - it seems like a solid beginning, but definitely not a fully-functioning product (especially on Mac, I hope it works better on Linux/Windows).

Disclaimer - I'm one of Diversion's founders.

schaefer• 1 month ago

Pricing for on-premise deployment is “talk to us”.

No thank you.

david_shi• 1 month ago

Has anyone built LoreHub?

a1o3• 1 month ago

I think the main reason of the flame wars is the fear the employer will decide to change things and erase all the years of knowledge on the existing tools, by something new. I believe this is the advantage of letting people use different tools for different things.

fg137• 1 month ago

Is any company going to actually deploy this system, in, let's say, two years?

You can trust Helix and Perforce because they have been doing this for decades, and this is part of their core business. You know they will keep maintaining the products for a while.

But for Epic, they could give up on this project tomorrow, and nothing will happen to their business. If anything, they'll get some dev resources back which helps their business.

It's like why anyone would trust Cloudflare to continue investing in EmDash over the long run. Cloudflare doesn't care about CMS. Their business is not providing the best experience for readers/authors/site owners. It's just a side hustle that is almost irrelevant to their business.

paganartifact5• 1 month ago

Lore is copying us with "git for gamedev" messaging, and their entire brand.

Epic interviewed me asking all sorts of questions about my customers and use cases.

"Pagan" "Lore" lol Our site used to say "Folklore" at the top of projects.

They still don't really understand the customer or use case, so have fun.

exac• 1 month ago

I tried your website, but it just says "LOADING".

paganartifact5• 1 month ago

It's a regular loading state.

Keep slowly copying other people's ideas without a clue why you are doing it.

Keep losing Fortnite to Roblox and random indie devs. You think copying another person is going to help you? Lol

paganartifact5• 1 month ago

What is it like to be a lying retard?

You guys are pathetic, just lie at best lol.

Fucking loser.

applfanboysbgon3• 1 month ago

Respectfully, "VCS for game development" is not a novel idea in any way, and you have literally zero claim to it. It's obvious that Git is poorly suited for multimedia binaries that are integral to games, so obvious that there's Git LFS which is still just a bolted-on hack when you'd ideally have a system designed for them from scratch. It's also explicitly not specifically "Git for gamedev" -- it's "Perforce for gamedev", as they opt into the authoritative server model rather than Git's decentralized model.

Epic has also been working on this for longer than the three months your repo has been around. This is the launch of the open source project and independent GUI, but it was already integrated with their Fortnite game engine before now.

Your VCS repo is called "Artifact". It seems like a stretch to say that somebody using "Lore" is copying from you. Even if it were called "Pagan" that's still a stretch, but "Artifact" especially so.

I think a VCS written in Javascript is also inherently a nonstarter for any kind of production-grade use case. The idea that nobody else can make a gamedev-oriented VCS because somebody in the world made a little hobby project along those lines is frankly ludicrous.

(Granted, I'm not exactly keen to use Lore myself given the heavy signs of vibe coding throughout the documentation and repo. A vibe coded VCS is also inherently a nonstarter. Here I'm hoping some third party will come along and do VCS for gamedev right!)

I'd also add that there's some element of hypocrisy to criticising Epic here when your own marketing pitch is "I copied Git".

paganartifact5• 1 month ago

I think a VCS written in Javascript is also inherently a nonstarter

That's because you have no clue what it is, or why it's in JavaScript.

Nothing else you said is interesting or valid.

Keep copying other people without a clue why you're doing it. Let's see how many customers you get.

Fortnite

The only cool thing they did (technically they didn't) - a game that is becoming a platform they can't handle.

The idea that nobody else can make a gamedev-oriented VCS because somebody in the world made a little hobby project along those lines is frankly ludicrous.

They dropped the "folk" and just made it "lore", it's too close to our branding. It makes no sense for Epic - their site is basically a vibe coded trash version of our real one lol

All Epic does is copy other people, that's why Fortnite sucks now and the company is going downhill.

mosfets• 1 month ago

I needed it!

DR_MING• 1 month ago

Can't believe that new version contorl system is coming out, very promising.

paganartifact5• 1 month ago

Epic is trash. The whole company is basically Tim Sweeney crying on Twitter.

calebm2• 1 month ago

I am a math artist (https://gods.art), and I’ve had trouble with hit because I want to be able to check in images and similar artifacts, but got is clearly not designed for it. Very excited to try this!

kurhan• 1 month ago

Why development of lore happens on git/github ?. Subversion is hosted on subversion server. Same applies to mercurial or fossil. I'd expect that VCS project is self-hosted.

kreco• 1 month ago

Maybe because they want contributions.

They have probably enough _bigger_ projects that dogfoods it.

Juliate• 1 month ago

I understand the website doesn't address users first, but it really feels like a missed opportunity to show a typical, simple use case in the front page (as in, "as a user, what's in it for me?").

A quickstart link is buried in there, and that's good, but the value of the product is multi-faceted and doesn't address data architecture only.

hosteur• 1 month ago

I do a bit of amateur photography and I have my photo collection in subversion today. Would Lore be a better fit for a photo collection with hundreds of thousands of binary files and almost a TB in size?

I have considered git many times but i have effectively one user and what I have now works. And I did not see any benefits in git for this use case.

fastcrw• 1 month ago

is there any comparision table or so with git, jujutsu or so?

minton• 1 month ago

I know this is targeted at game dev, but I wonder if this would work well for a huge MS Access app.

calebm2• 1 month ago

Any lore hosting services available?

jkl5xx• 1 month ago

Random naive thought, could I use this to backup and version all of my user data? E.g. just point Lore at my ~/Documents folder and call it a day? I'm sure there's some scalability constraint but I have music projects, game projects, markdown notes, photos, videos, etc. and have thought in the past it'd be convenient to treat all my files like a monorepo.

pixelneon• 1 month ago

Nice Website

aussieguy1234• 1 month ago

Since this claims to work well with large binary files, would this be a good solution for versioned PC backups, or would it be overkill?

Let's say I had multiple games installed, lots of videos downloaded, 3-4TB of data.

Lore – Open source version control system designed for scalability

Related Stories