On Bazel and Open Source

http://julipedia.meroh.net/2015/04/on-bazel-and-open-source.html

by shoover • 11 years ago

64 14 11 years ago

julipedia.meroh.net

zaphar• 11 years ago

Blaze is the one thing I miss the most from Google. However Blaze is very heavy for most small open source projects as the article says so the startup cost is too high.

I think there is probably room for a blaze-lite that relaxes a few of the restrictions blaze requires and is written in an easier to deploy language. Go perhaps?

The build rule language and the correct dependency specification and tracking are the most important pieces to have in my opinion. You could drop the expectation that all the dependencies including the compiler toolchain are in the source repo without losing too much.

shoover3• 11 years ago

waf [1] is very close to what you describe.

It's all Python. It's built so you can include one Python file in your source tree and install nothing but Python.

A relatively small core handles dependency tracking and shuffling flags from the user to tasks. I can't believe how fast it is, considering it is reading the files and computing hashes.

While user scripts have access to all of Python, you can limit yourself and write rules in a high level build rule style that's only slightly more Pythonic than Bazel, et al. Basically it's bld.program(source=['main.c'], use=['foo']) or bld(features='c cprogram', source=['main.c'], use=['foo']) instead of cc_binary(srcs=['main.c'], deps=[':foo']).

Adding rules can be done in various ways, including genrule-style shell commands, registering a task class by file extension, and a very flexible system of registering methods to hook into the "features" attribute.

With surprisingly little work, you can layer your own build language on top of the core modules. Demos are provided that parse makefiles or generate rules by searching the source directory with no user script [2].

Relative to the OP's concerns, a configure step is required and can search the system, call pkg-config, write config.h, etc. Arbitrary command line options can be added by shared tools and user scripts. Install, dist, and distclean are all there.

[1] https://waf.io/

[2] https://github.com/waf-project/waf/tree/master/build_system_...

chubot2• 11 years ago

FWIW Facebook's Buck and Twitter's Pants are both inspired by Blaze, i.e. have Blaze-like syntax:

https://buckbuild.com/

https://pantsbuild.github.io/

AFAICT, both are written in Python, which is a more palatable dependency than Java. Hm actually Buck is mostly Java, with a Python buck.py program.

mwcampbell• 11 years ago

Another build system that came out of Google, particularly the Chromium project, is GYP (https://code.google.com/p/gyp/). GYP is also used by Node.js, which wraps it in a conventional-looking configure/make procedure. I wonder how similar GYP is to Blaze.

chubot2• 11 years ago

Well GYP stands for "generate your projects". And I think it has actually been replaced by the "gn" build system, which does look a bit like Blaze, but with non-Python syntax.

I think GYP is more like a build system generator than a build system. It was more like autoconf (although quite different semantically). The original goal was to generate Visual Studio and XCode projects. And it also generates Ninja files.

Blaze/Bazel is not as portable, since it doesn't do Windows.

vhbit• 11 years ago

afaik Buck is Java

shoover3• 11 years ago

AFAICT they both have similar restrictions relative to the OP's concerns. I found the article because I was searching far and wide and for to deal with uses cases where one would typically use configure or pass a custom user command line option. They all have slick systems for organizing and reading built in options, but I couldn't find anything for adding configuration.

BJBBB2• 11 years ago

As an older and more cynical person, hard to be impressed by Yet Another Build system. Build systems just keep coming, one after another. New stuff, such as a new or different programming language or OS, is typically fun stuff. Another build system is never fun stuff. Please yell at me (have poor hearing) when you find a logical, useable build system that is fun.

FWIW, have used autotools, but mostly use scons.

vezzy-fnord• 11 years ago

I would classify djb's redo as being "fun", in large part due to its high implementation terseness and conceptual elegance permitting equivalency with make while ensuring target file atomicity, proper incremental rebuilding and a highly straightforward way of expressing build dependencies/relations based purely on metadata from the file system namespace, all while being persistent and driven solely through the shell.

Build systems can be eye-opening. tup, redo, memoize.py and Bazel all demonstrate different, interesting paradigms.

cbsmith2• 11 years ago

A lot of build systems focus on trying to make it "easier" to define a build, and invariably that notion comes with a certain naiveté about what is needed for a build.

Build systems are never fun, agreed. That's not the point. The point of Bazel is having one that is correct and efficient. That makes a huge difference, though not with the same out-of-box-experience as the "easier to use" build systems.

We'll see where this goes, but I'd encourage you to recognize the difference in what is being attempted here (admittedly, not for the first time, but this design has a pretty established track record... note the comments here of people actually missing the Google build system).

BJBBB2• 11 years ago

Will suppose that bulk of you people are corporate minions doing desktop or web stuff. At home and at work do 'deeply' embedded code. Assembly and C, with some Python honey poured on top to fill in the cracks. So perhaps it is a POV thing - huge company stuff done by several programmers vs small little black box stuff done by one or two EEs.

But a good tool, however mundane the task, will always be fun. This is my second Law of Engineering.

shoover3• 11 years ago

I'm surprised how little scons is mentioned in all the chatter around the announcements of Bazel, Buck, and Pants. We use it with swtoolkit and it's really slick how multiple environments can be built up. The only problem is it's slower to start building than I would like (because it has to set up the OS environment for msvc every time, which takes 400+ms).

Unfortunately swtoolkit seems to be dead since a couple years ago.

pjmlp• 11 years ago

Java is also a really annoying dependency to have in a project. Java virtual machines are not particularly known for their portability: the "build once, run anywhere" motto has always been a fallacy. By using Java, one closes the door to pretty much anything that is not x86 or x86-64, and anything that is not Linux, OS X nor Windows. Support for Java on other operating systems or architectures is never official and is always unstable for some reason or another. Heck, even most interpreted languages have better runtime support for a wider variety of platforms!

I guess the OP needs to update his Java knowledge.

Out of my head without searching the Internet, I am aware of JVMs for Aix, HP-UX, Solaris, GNU/Linux, Windows, OS/400, z/OS and their respective processors.

Not counting the amount of hardware vendors offering commercial JVMs like IBM, Atego, Aicas, IS2T

Which sums it up to x86, x86-64, ARM (including Cortex ones), MIPS, PowerPC, RX and a few others.

But lets just bash Java for brownie points.

On Bazel and Open Source

Related Stories