Show HN: Metalsmith – A pluggable static site generator

http://www.metalsmith.io/

by ianstormtaylor • 12 years ago

174 46 12 years ago

mercurial• 12 years ago

It appears to be indeed both extremely simple, and extremely composable. How well does it handle large collections of files?

ianstormtaylor10• 12 years ago

I haven't tried it on crazy amounts of files, but it's just using node's basic async I/O under the covers, and reading once. If you notice any sluggishness let me know!

It's also greatly impacted by what plugins choose to do. I had an extra clone call (literally cloning the buffers for each file) in the templating plugin at one point that like 50x'd the build time :) Everything I've done so far though has sub-second build times—quick enough that I've been building on every request[1] in development which makes things super simple.

[1] https://github.com/segmentio/metalsmith.io/blob/master/serve...

aram2• 12 years ago

Did anyone else manage to install it? I'm getting a "shasum check failed" error.

The project scaffold generator part sounds pretty interesting because I needed it pretty often and eventually had to build that for myself.

ianstormtaylor10• 12 years ago

Just republished! Can you let me know if you still see it? Sorry about that :$

BrandonSmith• 12 years ago

Got the checksum error, too. Successful after the republish.

aram2• 12 years ago

Everything is alright now, thanks :)

fredsters_s• 12 years ago

Awesome.

sdegutis• 12 years ago

So simple, yet so powerful.

dangoor• 12 years ago

Could be compared to assemble.io and stylistically reminds me of Gulp.

justarandomanon• 12 years ago

Gulp was the first thing I though of when looking through the examples.

Edit: In fact, could this whole thing just be a gulp plugin?

andyfleming• 12 years ago

I don't see why not. My thought was "why even have this when you could just build the plugins for gulp?".

xianshou• 12 years ago

If this really works, I would love to see it replace the hellish jumble of team-editable documentation. I've seen Confluence, PBWiki, Google Sites, and a smattering of others used to no good ends...can we please switch to this now?

ianstormtaylor10• 12 years ago

I'm actually in the process of converting our Segment.io docs to use it right now :) makes it way nicer for everyone* folks to just be able to edit Markdown, but still have the power to do lots of custom things to make the experience better.

* I was going to say for "less-technical" folks but then I realized that even technical people shouldn't have to be subjected to our current tangle of Jade files!

sixdimensional• 12 years ago

I agree, nice for writing/generating specs! Plus, check the raw documentation in markdown/markup/whatever you want into source control, right alongside the code, and reap all those benefits too.

cristianpascu• 12 years ago

One thing I don't like about jekyll (unless I'm missing something), is that on site generation, the last modified time stamp of the files gets updated too even if the file content hasn't changed. This way a FTP program like Transmit will not be able to synchronize only the modified files.

sneak2• 12 years ago

Just about everything that supports ftp supports ssh+rsync. I'm not apologizing for the bug, but rsync+ssh is a sane default for synchronizing everything everywhere these days and sidesteps the problem almost entirely.

8ig82• 12 years ago

Thanks. Looking forward to trying it out this weekend. My current generator, which I'm generally happy with, is DocPad.

http://docpad.org

sgdesign• 12 years ago

This looks very cool. I like the focus on plugins, it would be pretty awesome to have a flexible static site generator with an active plugin ecosystem.

sneak2• 12 years ago

I like that static site generators are basically the Cups And Balls of our craft.

It's so well understood and constrained of a problem domain that we can now ignore the practical considerations and go all-out with the art itself. I feel like this design and api is a great example of that, much like Penn and Teller's Cups and Balls with clear cups[1] - wonderfully creative innovation within a completely and totally solved problem domain.

[1] https://www.youtube.com/watch?v=O_n3Zb3bW3g

shortformblog• 12 years ago

This is really intriguing. I also recommend HarpJS (https://www.harp.io/) as well, which has some impressive pre-compile features.

andrewflnr• 12 years ago

I like this design. It kind of goes in the same direction as my github.com/andrewf/filtdir while being significantly more refined. This seems to make the whole directory structure available to plugins, while my tool only works one file at a time. It might even convince me to switch.

Touche2• 12 years ago

This is a nit, but non-constructors should be lowercase.

ianstormtaylor10• 12 years ago

It actually is a constructor, just lets you omit the `new` keyword if you choose since I think it's nice not to have to do that sometimes.

Touche2• 12 years ago

One problem I see. On the one hand you say:

All of the logic in Metalsmith is handled by plugins.

But on the other hand, you say this:

Each plugin is invoked with the contents of the source directory, with every file parsed for optional YAML front-matter, like so…

The YAML parsing should be a plugin as well, some of us have existing JSON front-matter files.

ianstormtaylor10• 12 years ago

Hmm yeah that's true. Tough call since that is the front-matter is a 99% case, and the idea of file's being able to have their own metadata is kinda core to the functionality. I wonder if I could just add support for both JSON/YAML by default. Would get annoying if people started asking for more front-matter types thought...

sturadnidge• 12 years ago

It's cerainly as sane as sane defaults come, but I agree that if the philosophy is 'everything-as-a-plugin' (another excellent choice for this kind of tool) then everything should be a plugin :)

Now to build something with this, I can see a 'dual write' plugin in my immediate future...

icebraining• 12 years ago

A JSON file is a YAML file, so what's the problem?

asb• 12 years ago

I'm not sure why this is getting downvoted (well, maybe the tone), it's true that YAML is a superset of JSON http://en.wikipedia.org/wiki/YAML#JSON

8ig82• 12 years ago

You could have 'reading' and 'rendering' plugins. Reading plugins obtain the content and meta-data.

A possible use case would be generating pages from an RSS feed. You'd have to specify the meta-data separately in some config though.

(Sorry, really don't want to over-design something that seems simple and elegant. Just a thought.)

Kiro• 12 years ago

How do I get this working on Windows? I've installed it with npm install metalsmith. Now what?

roryokane2• 12 years ago

You’re right, the “Install It” section should make that clearer. Anyway, there is an explanation of basic usage if you follow the link to “CLI” (https://github.com/segmentio/metalsmith#cli) in that section. It says you can create a `metalsmith.json` file that lists source, destination, and plugins in the described format, and then run `metalsmith` from the command line to build your pages according to the configuration file. And I think you will probably also have to install any plugins you use beforehand. Plugin installation instructions are all in the plugin READMEs – they are basically all just `npm install <some-package-name>`.

But it’s harder to figure out how to create my own local plugins and make sure Metalsmith is able to see them. And the documentation should make it clearer how to use the JavaScript API, in the context of a static file generator, where most people are not thinking about writing a program. It took me a bit of thinking to realize that you would have to create a `whatever.js` file inside the directory containing JavaScript code with `Metalsmith(".")….build()`, and then just run it with `node whatever.js`.

ianstormtaylor10• 12 years ago

Sorry about that! Just updated the Readme and website to hopefully make that clearer. And I've added Readme's to all of the examples[1] now too, some of which use the Javascript API and some the CLI.

[1] https://github.com/segmentio/metalsmith/tree/master/examples

esquivias• 12 years ago

Somewhat relevant plug:

I have a similar weekend project that is aimed towards generating static markup. The syntax looks a bit like haml with simple to use mixins, includes, and variables.

http://rubygems.org/gems/aml

roryokane2• 12 years ago

You should have linked to its home page https://abstractmarkup.com/. The RubyGems page does nothing to sell me on why I should bother installing your gem.

ricardobeat• 12 years ago

Not just a nice tool, but a great implementation. Simple, lean code, no promises or anything fancy.

tobobo• 12 years ago

The overall structure of the thing looks pretty promise-y to me.

caiob• 12 years ago

I fail to see the big advantage of this over its competitors.

ianstormtaylor10• 12 years ago

Yeah it depends on what your use case is to begin with. If it's just the simplest blog with a running series of Markdown files, then really any of the static site generators will do. But once you get into trying to implement some more advanced features then you run up against the limitations of most (if not all) of them because they assume way too much up front. A couple real-world examples from us at Segment.io are:

Documentation - for our docs[1] we want to be able to use the same simple static site generator without having all of the blogging logic. Basically the nesting of the files should result in the nesting of the URLs. But we also want to be able to tie in metadata that we have in our database about all of our integrations. And we'd also like to be able to write custom handlebars helpers that turn a simple JSON object into a widget that renders API calls in any of our supported languages.

Academy - for our academy[2] I really want to get to the point where we can generate PDFs for each of our articles and being to re-distribute them that was as eBooks (or potentially for a collection of articles) because that kind of thing appeals to enterprises who are looking for guidance. And we could even end up doing the same thing with our docs pages. And then we also want to have custom handlebars helpers for

Blog - for our blog[3] we want just the most basic implementation, although maybe with some niceties about author metadata to load in avatars and such.

Whenever you try and get into additional features that weren't considered by the original "static site" (or worse "static blog") generators, you usually end up building really cludgey code, if it's even possible.

So with Metalsmith we avoid all of that, because the plugins can do whatever they want, and it's super trivial to add local plugins to the mix if you're cooking up something which you know is unique to just you.

And the last thing was that we were sick of having a Ruby dependency (with all of the associated slowness) just to build our blog with Jekyll. Basically was increasing build times by an order of magnitude.

[1] https://segment.io/docs [2]: https://segment.io/academy [3]: https://segment.io/blog

rayshan2• 12 years ago

With sooooo many static site generators (all very well done too), would be awesome to have a side-by-side by-feature comparison table.

rayshan2• 12 years ago

Apparently there are many aggregation efforts, but no by-feature comparison. https://github.com/jaspervdj/static-site-generator-compariso...

borplk• 12 years ago

Very nice. Well done.

rainburg• 12 years ago

Couldn't choose between Jekyll and Middleman, but now I think I'm going with Metalsmith. No ruby, understandable plugin structure… I'm sold!

jon49• 12 years ago

A functional type approach. It will be interesting to look more into it. It would be nice to have a .map, .filter functions (if they don't already exist). So, if you don't want to rebuild everything you could do a .filter(htmlDate < mdDate) type workflow.

Show HN: Metalsmith – A pluggable static site generator

Related Stories