zackelan

That's a good point, I suppose it'd be better to frame it as what you can run on a $1k workstation vs. a $10k rackmount server, or something along those lines.

As a software engineer who builds their own desktops (and has for the last 10 years) but mostly works with AWS instances at $dayjob, are there any resources you'd recommend for learning about what's available in the land of that higher-end rackmount equipment? Short of going full homelab, tripling my power bill, and heating my apartment up to 30C, I mean...

The version I've heard is that small data fits on an average developer workstation, medium data fits on a commodity 2U server, and "big data" needs a bigger footprint than that single commodity server offers.

I like that better than bringing racks into it, because once you have multiple machines in a rack you've got distributed systems problems, and there's a significant overlap between "big data" and the problems that a distributed system introduces.

attrs also has a feature that dataclasses don't currently [0]: an easy way to use __slots__ [1].

It cuts down on the per-instance memory overhead, for cases where you're creating a ton of these objects. It can be useful even when not memory-constrained, because it will throw AttributeError, rather than succeeding silently, if you make a typo when assigning to an object attribute.

0: https://www.python.org/dev/peps/pep-0557/#support-for-automa...

1: http://www.attrs.org/en/stable/examples.html#slots

I've been very happy with Wallabag as a Pocket / Instapaper replacement. Self-hostable, with an option to pay 9 EUR/year for a hosted version.

Just discovered that earlier this week, very happy with it so far. Thanks for your work!

Note that it's not either/or - you can dispatch work from an event loop to a thread pool (or a process pool) with loop.run_in_executor [0], while loop.call_soon_threadsafe [1] can be used by worker threads to add callbacks to the event loop.

This means that the "frontend" of a service can be asyncio, allowing it to support features like WebSockets that are non-trivial to support without aiohttp or a similiar asyncio-native HTTP server [2], while the "backend" of the service can be multi-threaded or multi-process for CPU-bound work.

0: https://docs.python.org/3/library/asyncio-eventloop.html#exe...

1: https://docs.python.org/3/library/asyncio-eventloop.html#asy...

2: Flask-SocketIO, for example, requires that you use eventlet or gevent, which are the "legacy" ways of doing asynchronous IO: https://flask-socketio.readthedocs.io/en/latest/

If you want a middle ground between hand-written shell scripts and full-blown Kubernetes, we use Hashicorp's Nomad[0] on top of CoreOS at $dayjob and are quite happy with it.

Similar use case - self-hosted VMs, for low-traffic, internal tools, and no need for autoscaling.

I can't speak to how well it integrates with Gitlab's Auto DevOps, but Nomad integrates very well with Terraform[1] and I'd be surprised if there wasn't a way to plug Terraform into Gitlab's process.

0: https://www.nomadproject.io/

1: https://www.terraform.io/

The key difference between "classic" RDS and Aurora is that classic RDS really only automated the control plane. That is, RDS spins up an EC2 instance (or two, for multi-AZ) on your behalf, attaches an EBS volume of the appropriate specs, installs Postgres, sets up security and backups and replication etc.

Under classic RDS, when your application makes a SQL connection (the data plane) it's talking to a more or less stock Postgres instance, the same as you would have if you ran it locally.

Aurora, on the other hand, is involved in both the control plane and data plane. Your SQL connection is to a Postgres instance that's been forked/modified to work within Aurora.

what do you do about people moving to a community for its desirable character but killing that character in the process?

Here's what I think is the central (and flawed) assumption in this line of reasoning - people move to an area because of its "character". And that "character" is an intangible, immeasurable quality, but it is somehow diminished if more people move to the area.

I grew up in Seattle. Both of my grandparents, when I was a kid, lived in Seattle's Fremont neighborhood. I live in Fremont today. From one perspective, the Fremont of my childhood is completely changed. On the other hand, it's still Fremont, with the Center of the Universe sign and the statue of Lenin and many other things I remember from childhood. Does it have the same "character"? Does it have a newer, different, but just as good, "character"?

Those are impossible questions and it boils down to a Ship of Theseus style argument. Either way, I can't bring myself to assert that the housing supply of Fremont should be artificially constrained by zoning policies, in order to preserve my ideal of what Fremont "should be" or "used to be".

At that point you're not writing JSON though. Once you start bolting on non-standard bells and whistles, why not recognize that JSON was never meant to be used for config files, and switch to something that was, like TOML [0]?

0: https://npf.io/2014/08/intro-to-toml/

I'm a very happy fish user but this is one of my pain points as well. If you want to define a function, the syntax is light-years ahead of bash, including named arguments (and closures!), but it took a fair bit of googling and eyebrow-wrinkling before I could figure out the way just to set a default argument for that function.

What I ended up with was (for a shortcut for generating a password on the command line):

    function pw --argument length
        test -z $length; and set length 16
        python3.6 -c "import secrets; print(secrets.token_urlsafe($length))"
    end

It's possible to have both randomness and reproduciblility - generate a random seed, log it, then explicitly seed the RNG. If you encounter a failure you suspect is caused by a particular random seed, temporarily hard-code that seed while you debug it.

The "that's an intermittent failure, just re-run it and it usually passes" attitude is in my experience more likely due to just plain old poorly-written tests, usually with a time.sleep() or something similar that makes them unreliable.

The reason for it, as well as the reason for it not being changed, is already explained in Redis' documentation:

https://redis.io/commands/slaveof

A note about slavery: it's unfortunate that originally the master-slave terminology was picked for databases. When Redis was designed the existing terminology was used without much analysis of alternatives, however a SLAVEOF NO ONE command was added as a freedom message. Instead of changing the terminology, which would require breaking backward compatibility in the API and INFO output, we want to use this page to remind you that slavery is both a crime against humanity today and something that has been perpetuated throughout all human history.

If you want to read about the theoretical underpinnings, Cassandra is derived from the original Dynamo paper[0] from Amazon, and Scylla is a drop-in replacement for Cassandra written in C++ instead of Java. Cockroach follows more closely the Google Spanner[1] approach.

For a more practical summary, compare the architecture overviews of Cassandra[2] and Cockroach[3].

0: http://www.allthingsdistributed.com/files/amazon-dynamo-sosp...

1: https://static.googleusercontent.com/media/research.google.c...

2: https://docs.datastax.com/en/cassandra/3.0/cassandra/archite...

3: https://www.cockroachlabs.com/docs/stable/architecture/overv...

If I'm reading the underlying docs correctly, previously you would have called ChangeResourceRecordSets[0] with a quite verbose XML document. It looks like you'd need to first query for the existing RR set, modify it, then update it, and deal with potential race conditions if two service instances are starting concurrently. Technically possible, but quite a bit of complexity.

Now with auto-naming, you create a service[1], then a service instance calls RegisterInstance[2] on start-up with a much simpler JSON payload.

0: https://docs.aws.amazon.com/Route53/latest/APIReference/API_...

1: https://docs.aws.amazon.com/Route53/latest/APIReference/API_...

2: https://docs.aws.amazon.com/Route53/latest/APIReference/API_...

For higher-end home connections (100mbit - gigabit) those older "Wi-Fi aisle at Best Buy" type of routers may or may not be able to saturate the connection.

By separating the router and wireless APs you also get the ability to place multiple APs throughout your home for improved signal.

https://twitter.com/githubstatus/status/935296653401247744

"We are investigating reports of elevated error rates."

Anecdotally, it seems to work if not logged in.

The cable bundling isn't scare-mongering, it's a real scenario (but a worst-case one).

It also serves the purpose of being something I can explain to less-technical relatives at the Thanksgiving table, and they immediately grasp the importance. If I started talking about Title II and common carriers and peering agreements it would have nowhere near the same impact.

If you're unconvinced, you should default to supporting net neutrality. It's the status quo, the null hypothesis. The burden of proof is on the people wishing to make this fundamental change to the structure of the internet.

From TFA:

So: don’t treat this as a fully general Redis/MySQL benchmark. It really, really isn’t.

I’ll repeat this one more time, so drive-by commenters look silly: this is a benchmark of our specific use case and no other.

The survey, as well as the article, is bullshit clickbait.

To me it reads like barely-disguised native advertising, with every single chart having both the company's logo embedded in it and a direct link to their homepage right below it.

It's sad to see such content appear so prominently on HN.

Sqlite is great but it is a poor substitute for a "real" database

This is true, and explicitly acknowledged by the author:

SQLite does not compete with client/server databases. SQLite competes with fopen().

https://sqlite.org/whentouse.html

If you're deciding between Postgres and SQLite, in most cases you're doing it wrong (there are a few exceptions, such as serving a low-traffic almost-entirely-read dynamic website can be accomplished easily with either). Usually you want to choose between SQLite and "anything else that involves writing directly to the local filesystem".

I'd argue that the more large/complicated/nested the JSON structure, the more it would benefit from using a "real" database instead of a JSON file sitting on disk. If not SQLite due to document-relational mismatch, then LevelDB, RocksDB, or one of the other embedded key-value stores.

Of course, SQLite can also emulate a key-value store quite well, with a table with 'key' and 'value' columns.

"SQLite As An Application File Format" [0] and "What If OpenDocument Used SQLite?" [1], both by the author of SQLite, may help to answer why you'd use it over other file serialization formats, such as CSV, JSON, Python's pickle, etc.

0: https://sqlite.org/appfileformat.html

1: https://sqlite.org/affcase1.html

Early releases of Chrony go as far back as 2006 [0] so the original codebase predates Rust, Go, etc.

0: https://git.tuxfamily.org/chrony/chrony.git/tag/?h=1.20

That strikes me as being comparable only at the most superficial level. I started that SQL Server container and it's using 825mb of RAM (according to systemd-cgtop), just sitting there idle with no data in it. I started a Postgres 9.6 container also using their official Docker container and it's sitting idle at 58mb of RAM.

Insult to injury, if you're using Docker in a VM (as you must on a Mac) the instructions on that Docker Hub page say you need a minimum of 3.25GB RAM dedicated to the VM. That's quite a heavy lift if all you want is a database instance for local development.

And then sure, you could run SQL Server on Linux for your production instances, but you're well off the beaten path. You're going to have to roll your own infrastructure rather than relying on AWS RDS, Google Cloud SQL, Heroku, etc etc. And the moment you have a performance problem the first advice you'll hear seems likely to be "try it on Windows and see if it reproduces there".

I've been very happy with Borg[0] combined with the B2 command-line tools for Linux backups. I switched from Duplicity + AWS and it reduced the cost as well as made the backup process simpler - Borg doesn't have the same full vs incremental difference to think about, you just take a full backup each time and it dedupes everything automatically.

0: https://borgbackup.readthedocs.io/en/stable/

The self-hosted solution I use for this (switched from LastPass) is pass[0] plus syncthing[1]. Passwords are just GPG-encrypted files, so they replicate seamlessly - much better than the "single monolithic database" approach of things like Keepass which is prone to sync conflicts.

Syncthing mirrors everything between my desktop, laptop, and phone (and there's an Android app[2] that works with OpenKeychain[3] so passwords are accessible from my phone). I haven't done this yet, but it'd be trivial to also run syncthing on a cheap VM somewhere, and replicate the passwords to it (but obviously not my GPG private key) for disaster recovery.

0: https://www.passwordstore.org/

1: https://syncthing.net/

2: https://github.com/zeapo/Android-Password-Store

3: https://www.openkeychain.org/

CPUs with hyperthreading typically allow you to disable it in the BIOS. If you're running a new, beta-ish CPU microarchitecture (such as Ryzen/Threadripper) you might wish to disable HT as part of troubleshooting a CPU freeze/hang.

Changing a single CPU-related flag in the BIOS should under no circumstances render a filesystem unmountable.

How is L2ARC not "true hybrid"?

I think what that may be referring to is that the ARC is in-RAM and obviously cleared on a reboot, so as a result L2ARC on an SSD is also not persistent. After a reboot, you have to allow the ARC to fill up, then as it evicts data from the L1ARC it's pushed to L2ARC. Until that happens the SSD is not used at all.