Makesite.py – Simple, lightweight, and magic-free static site/blog generator

by staticwebdev • 7 years ago

121 72 7 years ago

dmortin3• 7 years ago

Are static site generators only for small sites? E.g. if there is a site with 100,000 pages and the site owner wants to change something in the template (adding something to the sidebar, or footer, etc.) then does he have to regenerate and redeploy all the 100,000 pages every time when he makes a global change?

hanniabu• 7 years ago

Depends how it's setup. I remember reading something one time about how you can set it up where each individual build step is cached which would cut down on a lot of build time. So for instance, in this case it would only remind the sidebar.

vages• 7 years ago

I would guess that depends on whether the change affects the HTML. If it's only a CSS change, you could probably get away with redeploying the CSS file.

mxuribe• 7 years ago

I can not answer for large sites (nor for makesite.py specifically), but in general to your scenario, yes, all pages would need to be regenerated. I guess you could think of it as a build server. But instead of building code for an app that you would be deploying, you are generating your content for your static site/blog. That being said, you could then create a build server optimized to generate your content, and/or try a static site generator whose forte is the build/generation process (I think hugo fans tend to highlight this feature for that platform?). Maybe instead of generating all content on your local machine, you could keep all your content on a beefier machine, and generate it from there - which could process stuff much faster, etc. Even still, it is probably unlikely that one would have to alter things like sidebars or footers that often, right? Also, worse case scenario, something tells me that even if needing to regenerate that many pages, i mean, would it be ok if it takes just a few minutes - probably not the end of the world, right? (I guess it depends on the timeliness of the single post or footer change that needs to wait for 100,000 pages to get regenerated, eh?) I hope my comments helped a little.

DoctorOW• 7 years ago

Yes changing the global template would involve regenerating all 100,000 pages in most static site generators. The only exception being those rare ones which offload template rendering to client side (there's a couple that I've seen). Still, some static site generators like Hugo, are built for bigger sites specifically by being able to generate massive amounts of pages very quickly. Even in your 100,000 page example a site probably would be completely rebuilt within a second or two.

earthboundkid• 7 years ago

It depends on how you do it but generally speaking, the break even point is “will I be generating pages faster than they are accessed?” Eg if I have a site like Wired.com with a long tail of old articles, it might be that by generating it, I refresh old articles 500x daily for the 500 new story creation and edits, even though the average old article gets less than one view per day.

cameronbrown• 7 years ago

Maybe an incremental generator would solve this problem?

softwaredoug2• 7 years ago

Based on our experience (n=1) I would say, yes, it works best for small sites. We have a medium site with on the order of 1000s of pages. Jekyll has become a bit unwieldy to rebuild a full static version of the site (takes about 5 mins for a full rebuild). I would certainly say an order of magnitude higher we’d definitely want server side Wordpress or similar.

jacquesm5• 7 years ago

Try Hugo if you're using Jekyll. I switched and never looked back. Blazing fast and very solid, and very near to a drop-in replacement.

jjjbokma2• 7 years ago

My blog, https://plurrrr.com/ consists of 291 pages and is generated within less than 1.5 seconds. I use my own static site generator: https://github.com/john-bokma/tumblelog

dikaio• 7 years ago

5 minute+ rebuilds are enough to move you to WordPress? 5 minute rebuilds does seem high but the benefits you get with static site generators (security, push “anywhere”, performance, etc) far outweigh having to manage everything else coming with a wordpress site. I suppose if you and your team are pushing new articles multiple times a day the builds could become annoying but even then I’d build/deploy and go grab a cup of joe; come back all done

z3t4• 7 years ago

The alternative is to generate the page per request. So it depends on the read/write ratio.

shakna3• 7 years ago

It may require regenerating all pages, but depending on the generator that may not be particularly expensive - they may cache partial assets, so you replace part of the build asset and then the build is basically a bunch of cat-like calls, which tend to be extremely cheap.

The largest static site I've managed was only a couple thousand pages (2138 pages from the last build log), but the build time was around 8 seconds (8.1 from the last build log) with cached build assets.

I'd be happy using a static generator for most sites.

dmortin3• 7 years ago

The build may be cheap, but you still have to upload the 100,000 files every time the global template changes.

dmortin3• 7 years ago

It may require regenerating all pages, but depending on the generator that may not be particularly expensive - they may cache partial assets, so you replace part of the build asset and then the build is basically a bunch of cat-like calls, which tend to be extremely cheap.

I've been thinking about this, and you can only do quick dependecy checks if the input is in files, right? So if the actual data is in a database then you do have to have some process to dump the changed data into files from the database, so the build system can pick them up.

Otherwise, if the build system takes data directly from the db then it doesn't know if the data is changed, so it has to retrieve everything from database which won't be efficient.

brenden2• 7 years ago

Cool, but I'm so tired of Python that I've started to avoid any projects written with it. Dealing with dependencies, 2 vs. 3, pip, virtualenv, etc is such a pain.

I think the Java/Go/Rust model where all dependencies are included in the binary is the way to go.

idoubtit• 7 years ago

In my opinion, the worst problem is the lack of a standard way to handle dependencies. I recently had a hard time deploying a small Python project over a few heterogeneous Linux machines.

For example, this "simple, lightweight" program has dependencies, as one can see in its .travis.yaml file. That's why the readme has the command `pip install commonmark` (plus more dependencies for dev). But, wait, doesn't `pip` install as root in /usr/local/ by default? (BTW, the documentation of `pip install` is mute on the subject.) So I should use pipenv, but last time I tried it was broken and the ticket had low activity. So virtualenv + pip, but it's a mess since I like using many terminals. I've heard about poetry, but that's not standard at all...

jacquesm5• 7 years ago

Python deployment is a shitshow. Unfortunately mission critical stuff (looking at you letsencrypt) is also written in it which means that your systems are built on very shaky foundations.

mixmastamyk• 7 years ago

It is not hard, it is just folks desimplifying things to solve largely theoretical problems.

Just pip install to the location you choose. sudo or —user is fine 99% of the time. Venvs are only needed for big work projects.

kalia354• 7 years ago

Why is virtualenv a mess with multiple terminals ? pip install for root unless you activate a virtualenv in which case it installs in it. If you make an alias of . env/bin/activate it is very simple, no ?

marmaduke• 7 years ago

You should use pip install --user, which pip will tell you about if you run pip install -h.

Virtualenv is usually excessive unless you have to handle lots of version problems

tedmiston4• 7 years ago

Dockerizing Python apps provides a nice isolation that makes managing dependencies across multiple apps much easier.

drunken-serval2• 7 years ago

Or go to the dark side and install Wordpress.

It’s actually not as bad as I thought it would be. I modified one of the twenty-x themes that come with it and installed a few plugins to make a decent personal site. It took less time that it would have to create anything with remotely similar functionality.

I run it off a single vps that has no other purpose. If it gets hacked, nothing of value is lost. I can revoke the api keys for the external services I use and restore to a new server from my backups.

x3haloed2• 7 years ago

More like the vulnerable side.

skittleson• 7 years ago

wordpress is great for quick sites. Long term maintenance is where I have been bitten.

suyash• 7 years ago

I would have considered if this project was using Node.js instead Python as well.

dastx• 7 years ago

Sure, let's install 1 million dependencies instead of 5.

cxr• 7 years ago

I think the Java/Go/Rust model where all dependencies are included in the binary is the way to go.

This is how triple scripts are designed[1] to work.

One proof-of-concept just so happens to be a minimalist static site generator called triickl[2]. You can drop it onto (virtually) any machine, double click, and point it at a source directory for your site—subject to the expectation that it follows a certain jekyll-like structure—and then you're off to the races. I'm using it as the build step for the triplescripts.org home page itself and my personal site.

1. https://triplescripts.org/format#self-contained

1. https://releases.triplescripts.org/triickl/

diminoten• 7 years ago

Have you tried writing a web app in Java/Go/Rust vs. Python in the past year?

joshklein2• 7 years ago

You raise a fair general point about Python dependencies, but that issue is usually related to poorly considered use of third-party dependencies or the need to distribute binaries and leverage C bindings.

But here we have exactly the kind of project that should be written in Python, forked and cloned by you, then tweaked and maintained as if you wrote it yourself. From a brief glance through the code, and taking the submission title at face value, this project is a refreshingly appropriate use of the right language. I offer this as someone who writes Python and Go, generally preferring Go anywhere possible.

I plan to poke around some more in this repo to see if I can easily replace my use of Hugo with this, since I use only a tiny fraction of the features in Hugo and have found it challenging to hack on it as a low-priority weekend project.

For turning personal blog posts from markdown to HTML with some general convenience functions around maintaining the front-matter and templates, I can’t think of something more appropriate than a Python script sitting inside your blog directory with all of its dependencies except the interpreter itself checked in with your templates and content.

RobertRoberts2• 7 years ago

I won't bother with Python again because these conflicts happen on your first time even trying out the language. (running linux, any better on Windows?)

It was such a mess that I can't even describe the frustration. And I even got Apache working with SVN! (40 pages of docs and no testing points from start till it's fully working, yuck)

EDIT: Since I got downvotes so quick, tell me why anyone should choose Python over the many alternative out now?

theamk2• 7 years ago

I’ll by Go or Rust, but Java? I have spent too much time fixing java software recently.. it’s dependencies are a complete pain.

tedmiston4• 7 years ago

You can create and distribute all-in-one Python executables with PyInstaller.

I've used it in multistage Docker builds to produce very clean, minimal production images like what people commonly do in Go.

https://pyinstaller.readthedocs.io/en/stable/usage.html

m0002• 7 years ago

I don't want to bash or put down anyone, but frankly I don't see any point with Makesite.py. But, why do we have a toy project that openly admits that it reinvents the wheel on the first page of HN?

Makesite.py is "sold" as an alternative to Jekyll for people who prefer Python. Yet, it completely fails to mention Pelican [1]. Why should I bother with this instead of going with the featureful and well-tested Pelican?

[1] https://github.com/getpelican/pelican

foo1012• 7 years ago

Makesite.py is "sold" as an alternative to Jekyll for people who prefer Python.

That's a really strange argument to make. If you see the README, it mentions, "But then did you yearn to use something even simpler to generate your blog? Do you like Python? Perhaps the thought of writing your own static site generator crossed your mind but you thought it would be too much work? If you answered "yes" to these questions, then this project is for you."

Makesite.py is being presented as a do-it-yourself blogging solution with makesite.py serving as a good starting point to start hacking. It is very different from Jekyll. How does it make sense to mention Pelican which is, if anything, similar to Jekyll and not similar to Makesite.py?

m0002• 7 years ago

In fact I have written more than one DIY static site generators for small sites in the distant past. The bottom line of my experience was: they don't worth the effort. You will start missing features sooner than you think, and it will be entirely up to you to implement them from scratch. Instead of actually adding content to your site, you'll end up adding features to your feature-strapped hack.

In the presence of excellent tools like Jekyll or Pelican that are also fairly easy to use and extend, I'd need a concrete use-case before going DIY. If there is no concrete use-case, the DIY static generators is just another version of the "not invented here" syndrome. Just my 2¢.

georgecn• 7 years ago

I don't use Makesite.py but if you see the network graph [1], obviously there is an audience for this kind of tool. I see many more results of people using and modifying makesite.py here [2].

I think the appeal of a tool like Makesite.py is that you can make it what you want [3]. It is about a 100 lines of code and written very neatly. Instead of wading through documentation and configuration to figure how to get Jekyll or Pelican running, you can make your own Jekyll or Pelican. I can see why many individualist programmers would like such a thing.

[1] https://github.com/sunainapai/makesite/network

[2] https://github.com/search?q=%22makesite.py%22&type=Code

[3] https://github.com/tuchandra/tuchandra.github.io/blob/master...

x3haloed2• 7 years ago

Crap. You beat me to it! https://github.com/x3haloed/afsw

welder• 7 years ago

Good job, but why would I use this when it takes less code and less time than it takes to read Makesite.py source code, to build my own with Jinja2 [1], PyQuery [2], markdown2 [3]?

Python is so easy and powerful there's no need for a static site generator.

[1] https://jinja.palletsprojects.com/en/2.10.x/

[2] https://pythonhosted.org/pyquery/

[3] https://pypi.org/project/markdown2/

jjjbokma2• 7 years ago

Nice, I wrote a simple, lightweight and magic-free static site/blog generator as well [0]. Example site: http://plurrrr.com/

[0] https://github.com/john-bokma/tumblelog

Makesite.py – Simple, lightweight, and magic-free static site/blog generator

Related Stories