mcmatterson

The thing that really gets me as a small-time OSS maintainer is that none of us asked for this. The social and technical millieu where most of us started our projects is not the one we find ourselves in today, and the forces behind this are wildly asymmetric.

Security findings are one place where we as maintainers simply do not have the choice to not play ball, whether we like it or not. It seems likely that the only way that we meet the moment is to adopt these tools ourselves -- once again -- whether we like it or not. Reconciling this with the ground truth that 'OSS doesn't owe anyone a goddamn thing' is proving to be really hard for me.

As an HTTP server author, this doesn't surprise me.

We've ceded HTTP specification development to the big guys, and in so doing have made it more or less impossible to implement without resources on their scale. Have you looked at RFC 9000 et al? They're monstrously big, far larger than most independent shops could ever hope to economically pull off. The only way to comprehensively implement something of that scale is to have Google level resources to throw entire teams of engineers and years of focus at it.

I've long said that any protocol worthy of being foundational should be reasonably implementable as a fourth-year term project. It doesn't have to be production ready or ergonomic or even generally useful, but if a group of fourth year CS students can't pull an end-to-end implementation together in a semester, the protocol is just too complex. It's not perfect, but it's as good of a yardstick as I've found.

HTTP/1 passes this test easily; you can make a working version of it in about ninety seconds right in your terminal. HTTP/2 looks intimidating at first glance, but it's so much better specified than HTTP/1 that it's almost easier to get to a reasonable implementation. HTTP/3 on the other hand is...... well, weeks (if not months) of work just to get a QUIC foundation working reasonably well enough that you could hope to start iterating on connections from a 'real' peer, and THEN you have to start on RFC 9114. Not to mention that the way it's structured you end up doing most of that work in the dark, hoping that you line everything up just so so that your first Hello World actually works. It's a way of working that is completely at odds with the hacker ethos that the best foundational protocols have in spades, and ends up looking and acting like what it is: a tool by the big guys, for the big guys. The rest of the internet need not apply.

I'd just mitigated this exact thing in Bandit last month!

https://github.com/mtrudel/bandit/blob/main/lib/bandit/http2...

TBH, from an implementors perspective this is a super obvious thing to cover off. It had long been on my radar and was something that I'd always figured other implementations had defended against as well.

The fact that a critical piece of the evidence was cell phone photos sent between workers coordinating door re-assembly doesn't exactly instill a whole lot of confidence in their permit-to-work process. I didn't like it when it was medical teams doing shift handover via a Google Doc, and I don't like it when it's a matter of flight safety either. Or, as Homer might eruditely say: "guess I forgot to put the bolts back in" [1]

[1] (https://www.youtube.com/watch?v=IiNPLIauEig)

I have this exact kit hosting my house's RPi server!

A couple of things:

0. The physical quality of this build is out of the world good. PCB, plastics, switches, it's all amazing

1. The software as provided has a pretty old school build process (part of the charm?). I tightened up a bunch of it and dockerized it at https://github.com/mtrudel/pibox/tree/main/pidp11

2. I wish the build would have used something like an MCP23017 for IO instead of claiming so many RPi GPIOs. There's only a few (2-3 IIRC) GPIOs unused by the front panel, and the matrix LED/switch scan setup burns a ton of CPU

BMath (CS) 2002 here. This description is spot on the way I think about development, and is a bit of a superpower to be able to do well. I'm not totally sure it's a UW-ism though. I can certainly recall a couple of very formative Tompa courses where he impressed the importance of taking a data-first view of design, and I think we had a stronger bias towards data structures than most other schools whose grads I've worked with. But overall I think that sentiment grew weaker in my upper years, when a more conventional algorithms approach took over.

I will say though, that I've also noticed the contrast before with MIT grads, who tend to have a very strong LISP bent to their styles. It's true that each school has their own unique flavour, and much like accents it may just be that you don't notice your own.

Bandit author here. Correct! The byline of Bandit is ‘a web server for Plug applications’ and being able to focus on that narrowed set of requirements is a large part of where the perf boost comes from (less code, easier to reason about, fewer processes, etc).

The PR you linked to adds support for generating new phx apps with the relevant change already incorporated; it’s just a generator change. We’re waiting a bit to incorporate this change to ‘soft launch’ Bandit support.

Bandit author here. We’re fully supported on phx 1.7+; it’s a one line change to your existing app, described on the Bandit readme!

I couldn't stop thinking about this, so I went back and dug up an ancient (20yo+) copy of my OS implementation for the trains course (mat(t)OS). Sure enough, we did indeed dispatch the upper half of interrupts to processes, albeit via a dedicated blocking syscall rather than send/receive/reply semantics. Bottom half handlers (which were implemented behind task gates and had persistent stacks) just did the standard bottom half stuff: disabling interrupts & managing state so we could properly unroll after the interrupt had been handled.

What a throwback!

Interrupt code is at https://gist.github.com/mtrudel/c29fa60e5b2f3b6fdc46a9e3c65d.... I've been meaning for years to clean this stuff up and resurrect it. Maybe this is the kick in the ass I need to finally do so!

https://www.youtube.com/watch?v=031vKBPk5eA

Fascinating talk. The overall approach here strikes me as being extremely influenced by QNX's design. Send/Receive/Reply as a messaging primitive is too often overlooked, and provides incredibly powerful that (as Cliffe mentions) renders an enormous amount of scheduling complexity as moot.

Anyone who's done the UWaterloo trains course will recognize these patterns immediately, and (IIRC) interrupt dispatching is was done in a similar manner there as well.

Finally, the supervision patterns here strike me as being very similar to those within the BEAM, and remind me of the infamous quote from Robert Virding (http://erlang.org/pipermail/erlang-questions/2008-January/03...). Obviously a necessary reimplementation here, but humorous nonetheless.

Great project, great talk.

Fundamentally, Bandit just does a lot less work than Cowboy does. Particularly with HTTP/1 requests, Bandit uses a single process per connection in contrast with Cowboy's 2 per connection. This allows Bandit's handler code to be very 'linear' and free of coordination complexity; for the most part, an HTTP/1 connection will go from initial connection to Plug call and back again without having to leave the process at all.

More broadly, there's just less functionality in Bandit (by design; it's a very focused library that doesn't need to do a bunch of the stuff that Cowboy does). All that stuff ends up adding up when you're doing it 100k a second.

This correlates with Bandit's relative numbers being better in HTTP/1 (up to 5x faster than Cowboy) than they are in HTTP/2 (a paltry 2.3x faster than Cowboy).

In terms of the 32 concurrent connections peak, my hunch here is that it's due to the server the test is running on being a 32 core VM (see https://github.com/mtrudel/network_benchmark for details of the benchmarking process). Truthfully I'm no expert with benchmarking & this may not hold a lot of truth; at the end of the day the test is pretty apples to apples so I think it has comparative merit, but the absolute numbers are a bit murkier to me.

Bandit is an HTTP server, not a client. It performs the same job in the stack as Cowboy does currently (just up to 5x faster as this post's title alludes to).

The Bandit project shares a small amount of code with the Mint project (Andrea & Eric were kind enough to factor out their HTTP/2 header compression library for use in Bandit), but this is mostly due to HTTP being a largely symmetric protocol (ie: there is a lot of functionality used equally by both clients and servers).

Author here. AMA!

A number of the first trans-atlantic cables landed in the tiny village of Heart's Content, Newfoundland. I drove through there on a road trip about a decade ago & stopped at the excellent museum in the old cable station, and was excited to make my way across the highway to the beach to see this exact thing. It turns out that the old cables are just.... left to rust on the beach. It's really amazing that these cables, originally a technological wonder & a bridge between entire continents, are just left to the elements once their useful life is over.

https://goo.gl/maps/Ku9FtfbMupApZthJA

My goodness, you're right (well, your example isn't, but the feature I'm talking about has been present since at least 1.8 (the earliest install I have handy):

  > <<len::8, val::binary-size(len)>> = <<3>> <> "abc"
    <<3, 97, 98, 99>> 
  > val
    "abc"

(Your example demonstrates 'pinning' which has been a thing since always I think).

The use of prior matches in guards is super useful for parsing TLV style payloads where you first match on a size/length field and then match on the value itself as that number of octets. Does anyone have any insight into how / when similar support will make its way into Elixir?

As someone who writes Elixir full-time in a product with deeply complex date / time needs (spanning multiple timezones, sources of time, multiple meanings of 'month', etc), I can say with some conviction that the the simplicity of Elixir's DateTime module is most definitely a feature. The lack of something as transparent in Rails was / is a huge part of what makes our team so productive since porting the product to Elixir; NaiveDateTime alone has saved us from more bugs than I care to think about. Between Elixir core and Timex, they've definitely solved the harder corners of time as well as any other library or platform I've seen.

I did a lot of network work early in my career (high school job at a small town ISP in the early 90's), and have read deeply about / waxed nostalgia for the Arpanet era ever since.

Networking now is as different from my ISP days as they were to the days of IMPs. It's remarkable.

I designed and installed a complete hydronic heating system in our house last year, replacing a conventional forced air system. It being the first time I'd ever done such a thing (and indeed, the first time I'd ever really been exposed to hydronic heating at all) I learned everything through reading (mostly the excellent idonrics series at https://www.caleffi.com/usa/en-us/technical-magazine). I went really deep on the reading so while I had/have a pretty solid theoretical understanding I was/am a complete novice on the craft side of the field.

Overall, the practice seems to be very heuristic based, with every installation being a bit different and rules of thumb being the norm. Our system seems to work fairly well in practice, but I've been curious to get a professional's opinion on it (the only pro that's seen it was the gas guy who did the final hookup, who seemed impressed but didn't really offer any specific criticisms).

Repo at https://github.com/mtrudel/boiler

I've since added an elixir based graphing system to it, at

http://mat.geeky.net/boiler

For someone who doesn’t follow lower level browser dev that closely, this seems like an optimization pretty specifically for a react style ‘nested redraws’ use case. Is this at all correct?

When I was in grades ~4-8 I was placed in my school's special education stream, which really just amounted to a day a week of totally unstructured play in an incredibly well kitted out classroom (at the high school across the road, no less). There was a group of us that spent more or less every available second in that classroom playing this game on the Apple II, sharing secrets and strategies through the (surprisingly long) game. I credit a large part of who I am to today to it.

As a grown up I've watched video walkthroughs of it numerous times with my daughter, and have been meaning to figure out a way to actually play it with her. This.... is amazing.

Goodbye, Friday!

This is a perennial problem for the handful of Pi's I have deployed around the house (homebrew DAC, weather station, boiler manager (wip), workbench terminal). They're all 'purpose built' and sit almost entirely idle while fulfilling a single, usually non-interactive task. And after 18 months or so, they all start failing the same way:

* Random userspace crashes (most recent one was a reproduceable crash in the bowels of the python runtime) happen with greater frequency

* (Usually in the process of debugging the above) strange filesystem artifacts show up (files disappearing after reboot, etc)

* Booting ends up taking 5+ minutes, if it succeeds at all.

The solution to the above ends up being 'rebuild the box on a new SD card'. This is usually on 'brand name' (Kingston, etc) SD cards bought through proper retail channels, so while they're not high end cards, they're at least not (likely) counterfeit either. I still don't have enough data to know if simply dd'ing the old card over is sufficient, or if a full ground-up rebuild is sufficient.

Would love advice on this, especially around brands of cards with known good lifetimes in similar environments.

This reminds me a lot of the father in 'All the Light We Cannot See', building their neighbourhood in miniature for his blind daughter to learn from. As a novice woodworker and obsessive builder of all things small and useful, I find small scale construction like this fascinating. To have spent so much effort on something so devoid of practical use seems impossible for the era.

It's not just you. I noticed it a few months ago and ended up disabling cursorline entirely. It seems to be caused by an intersection of cursorline and specific syntax highlighting rules (it happens for me in Elixir, but not in Ruby, for example). Honestly, while line highlighting is nice, I can't be bothered figuring out the way through it.

https://www.reddit.com/r/vim/comments/6onaui/vim_getting_slo...

is at least one exploration.

No KiCAD support?

"Given the importance of getting the core ABI and the related fundamentals correct, we are going to defer the declaration of ABI stability out of Swift 4 while still focusing the majority of effort to get to the point where the ABI can be declared stable."

Do I read that correctly? Swift 4 will have a ABI under the hood but they aren't going to come out and formally lock it down until after Swift 4?

I'm facing the same dilemma with a hardware project of mine (http://tooner-test.moshozen.com). In the past month alone, I've been stuck on several things (public name, dealing with constant ID creep, finding a mill that can resaw, among others). Though they're contradictory, it seems that half of the roadblocks get solved through putting them off (and usually thinking of a better solution, or a workaround), and half get solved by #JFDI.

'Shipping' means something much different for hardware projects of course, but nonetheless I think the advice to ship on day one is really foundational. I've always been fond of the idea that 'if something hurts, you need to do it more often'. Make the game about iterating and not shipping.

That's 40gbps of voluntary (technically settlement free, but not in the usual meaning of the term) peering, not paid transit. Paid transit numbers are several orders of magnitude higher, and total traffic numbers are unknown.