ralph87

If you buy a matched OS + laptop in any other situation, why not also for Linux? Buy an XPS next time, or any other vendor (Lenovo?) with tested hardware, and just put an Ubuntu USB stick in.

"I bought the top result for 'wheel' on eBay and ruined my life trying to fit it to this aircraft engine, this is such a joke!" Shit has just worked for years if you want it to work.

The network stack integration was less than transparent too. Random places all over the system where e.g. a magical ioctl() just didn't work properly. Who really wants to spend their life reimplementing every last baroque detail of that stuff?

The argument about Windows not having a single, central dentry cache doesn't hold water

This was more about cache coherency, and it's not an argument, it's a trait shared by every other Linux-over-X implementation (UML, coLinux). It is fundamental to solving a problem users want - seamless integration, i.e. no opaque ext4 blob. Why doubt the reasons given by Microsoft, when they match observations not just for WSL but every other system in the same class?

Contra opinion: the VM option reduced the service interface between Windows and Linux to a single kernel implementation and a few drivers, rather than every possible userspace program ever written. It's an amazing and obvious trade off. My inner architecture astronaut appreciates all the ideas in this post, but I've been trying to kill that guy for over a decade now. The bottom line is WSLv1 design SUCKED precisely because it tried to cross-breed 2 extremely complex semantically incompatible systems.

Example: Linux local filesystem performance mostly derives from the dentry cache. That cache keeps a parsed representation of whatever the filesystem knows appears on disk. The dentry cache is crucial to pretty much any filesystem system call that does not already involve an open file, and IIRC many that also do. Problem is, that same cache in WSL must be subverted because Linux is not the only thing that can mutate the NTFS filesystem - any Windows program could as well. This one fundamentally unfixable problem alone is probably 80% the reason WSL1 IO perf sucked - because the design absolutely required it.

Solutions are rip out a core piece of kernel functionality, and in the process basically taking over ownership and maintenance for ALL code anywhere in the kernel assuming the existence of said cache, engineer something that is somehow better, and support this in perpetuity, including any semantic mismatches that turn up much later that were never designed for

The idea of merged ps output where Windows binaries would show up in the Linux /proc. How would you even begin to implement that without confusing EVERY process management tool ever written that interfaced with /proc? What about /proc/.../environ? On UNIX that has no character set. On Windows it is Unicode.

A trillion problems like this made WSL1 a beautiful nightmare. Glad it was tried and from watching the tickets, that team bled heroically trying to make it all work, but ultimately, I'm also glad it's gone, because the replacement is infinitely easier to engineer, maintain, and use, and that team earns its quarterly bonus much more easily. Everyone wins, except the astronauts, but as experience teaches us they never do.

Do what exactly? This is essentially a performance optimization for folk bagholding oceans of vulnerable hardware

This method was talked about since the Spectre/Meltdown days. It has often taken Google much longer to release patches they've been sitting on, e.g. many original containers patches were like this.

Chrome is not open source, it's a derived proprietary superset of the open source Chromium. They are not the same thing in very material ways

We could argue this ad infinitum, K8s of course doesn't remove all proprietary elements from a solution but it is a huge step up. Speaking as an ex-Googler, it took over 10 years but I'm so happy the rest of the world finally has a standard like this, the world is a better place for it even though I at one point had to unlearn all my traditional sysadmin habits and immerse myself in an environment practising it successfully to finally understand.

Your original question was what is the point. These are the points. As for why not Docker, k8s network effects and strategy of its sponsors mean Docker is on a lifeline, everyone knows that.

Ever tried to host multiple apps on a single machine? Oh look, a custom Nginx config only one person understands. Oh look, some hacked up letsencrypt config only one person understands, etc etc.

K8S on a single node does nothing for you network-wise

- Container IP auto-assignment

- Container security policy

- Container DNS management

- Ingress management ("custom Nginx config")

- "Environment that feels like a large network and doesn't change if moved to a large network"

What part of this is difficult to understand?

Kubernetes does a whole lot more than just restart jobs when a node fails, in fact of all its features this is one most people probably see in action the least.

Kubernetes centralizes all the traditional technical nonsense related to providing a robust environment to deploy applications to. I want Kubernetes even in a single node scenario because I want Kubernetes-like packaging, deployment and network services for any app I work on, as they are a net simplification of what was previously an ad-hoc world of init scripts, language-specific supervisors, logging, monitoring, etc etc., and rearchitecting an app deployment from scratch simply because its resource requirements increased.

If people want to continue trying to scale it down further, where is the harm in this? There are plenty of legitimate cases where it makes sense. There's no real limit to that work either, it's conceivable with the right implementation improvements (in k8s and the container runtime), it might eventually be possible to reuse the same deployment model even for extremely small devices.

Just for completeness, updating parent comment's Lambda estimates, not counting provisioned worker costs, and assuming no request takes more than 100 ms.

    Lambda requests: ((1.5e9 * 4) / 1e6) * .20     = $   1,200 
    Lambda CPU (1536 MB): 0.0000025000 * 1.5e9 * 4 = $  15,000
    API Gateway HTTP reqs:
        (count): 1.5e9 * 4 = (6 billion)
        (first 300m): 300 * 1.0                    = $     300
        (next 5700m): 5700 * 0.9                   = $   5,130

    LAMBDA MONTHLY TOTAL                           = $  21,630
    LAMBDA YEARLY TOTAL                            = $ 259,560

     NLB (2x)
        (NLB hours 1 month):
           2 * 0.0225 * 24 * 30.45                 = $      33
        (NCLU hours):
           2 * (2280/50) * 0.006 * 24 * 30.45      = $     399

     NLB MONTHLY TOTAL                             = $     432
     NLB YEARLY TOTAL                              = $   5,184

     EC2 YEARLY
        (if  1 req/vCPU)                           = $ 253,926
        (if 15 reqs/vCPU)                          = $  67,704
        (if 30 reqs/vCPU)                          = $  33,852

That's an 87% reduction in operational costs in the ideal (and not uncommon!) case, and a minimum of a 59% reduction in the case of a web app from hell burning 250 ms CPU per request.

I stopped reading at "3000 cores"; there is a lot of money to be made mopping up disasters like that, it's clearly even something of a growth industry. We had one machine push 2,400 requests/sec average over election night, without even touching 30% capacity, costing around $600/mo including bandwidth. Its mirror in another region costs slightly more at $800/mo. As a side note, it's always the case with those folk that they invent new employees to top up their estimates, that wouldn't be required in the serverless world, yet in every serverless project I've ever seen, they absolutely still existed because they had to.

Price-perf ratio between Lambda and EC2 is obscene, even before accounting for Lambda's 100ms billing granularity, per-request fees, provisioned capacity or API Gateway. Assuming one request to a 1 vCPU, 1,792MB worker that lasted all month (impossible, I know), this comes to around $76, compared to (for example) a 1.7GB 1 vCPU m1.small at $32/mo or $17.50/mo partial upfront reserved.

Let's say we have a "50% partial-reserved" autoscaling group that never scales down, this gives us a $24.75/mo blended equivalent VM cost for a single $76 Lambda worker, or around 3x markup, rising to 6x if the ASG did scale down to 50% its size the entire month. That's totally fine if you're running an idle Lambda load where no billing occurs, but we're talking about the BBC, one of the largest sites in the world...

The BBC actually publish some stats for 2020, their peak month was 1.5e9 page views. Counting just the News home page, this translates to what looks like 4 dynamic requests, or 2,280 requests/sec.

Assuming those 4 dynamic requests took 250ms each and were pegging 100% VM CPU, that still only works out to 570 VMs, or $14,107/mo. Let's assume the app is not insane, and on average we expect 30 requests/sec per VM (probably switching out the m1.medium for a larger size taking proportionally increased load), now we're looking at something much more representative of a typical app deployment on EC2, $1,881/mo. on VM time. Multiply by 1.5x to account for a 100% idle backup ASG in another region and we have a final sane figure: $2,821/mo.

As an aside, I don't know anyone using 128mb workers for anything interactive not because of memory requirements, but because CPU timeslice scales with memory. For almost every load I've worked with, we ended up using 1,536mb slices as a good latency/cost tradeoff.

The emphasis here being on the word "should". I wouldn't buy just yet until some third party has published a sustained load/heat test. Apple have sucked at this for quite some years now, it's folly to touch a MacBook for anything remotely compute-intensive

The use case is benchmarks basically, and maybe faster cold boots. About the only place the typical heavy computer user is likely to notice major difference between SATA and NVME is probably running something like "find /", or some kind of full-disk search scenario.

Even in the latter, full text indexing which every platform has had for years now makes it much less likely that the full directory tree will even get walked, and differences even less likely to be noticed. As a side note, every desktop platform's full text search seems to suffer software performance problems that are largely independent of the underlying disk

Even in the full-tree enumeration case, since Spectre/Meltdown mitigations landed, system call overhead is so high now that even with a lightning fast disk, a large chunk of total time taken to walk the directory tree is lost basically twiddling the CPU mode securely. You can definitely still see the difference between SATA and NVMe, but you can also definitely measure the amount of time during the NVMe run that is spent in software -- incrementally faster NVMe will have quickly diminishing returns.

"What about databases!" This was my original interest in SSDs to begin with. It turns out, despite being a data monkey who loves large databases, since 2013 any time I've worked with a giant dataset like this, it is always in the form of large scans (usually from something like a CSV or XML file), where SSDs don't really have a mind-blowing advantage over magnetic (but of course they are still 5-10x faster a seq io, its just that data parsing and processing is typically the bottleneck now).

Small but important nitpick: since at least 2005 there is no "on the NYSE", the price you're likely seeing is the best price consolidated across all national market system venues, the primary listing exchange is mostly only an administrative entity in recent times. Actually some of the largest premarket trading is occurring on NASDAQ and a CBOE-owned exchange called BATS

The average web page can be as nasty as it likes, but if I can deliver apps that _feel_ awesome and optionally can deliver metrics that provide a commercial basis for that awesomeness to whoever is paying me, I don't really care much about the average web page at all. Insert some truism here about using the incompetence of others to justify one's own slovenly behaviour, and some other truism about standing out from the crowd by applying common sense.

"I built an average web application and it attracted an average number of users, it more or less worked if you visited it from mobile" would look fucking awful on anyone's resume.

Effectively this is arguing for the project's attention to detail being low, we covered this already

Marketed as a single codebase platform where the most ubiquitous deployment target has a user experience that tells your customers you hate them

4.39 MB in 50 requests is a showstopper on any mobile network, especially when alternative solutions do not have that problem. It is fair to assume some first time experiences will involve 7.5 seconds or more additional latency on 3G networks, double that for a poor signal areas.

It's still an instant pass

The gallery app looks to be authored using it:

- It needed to show a loading spinner on a 250 Mbit connection

- It hung the browser while 'booting' the page for a solid 3-5 seconds

- It downloaded 4.39 MB in 50 requests

- Opening web inspector in Firefox while reloading the page was sufficient to cause the boot process to hang indefinitely

- Page looks pretty, but at this point it barely matters

We could conclude either that the project's attention to important details is low, producing this experience, in which case what else might we discover once committed, or alternatively we could conclude that it is high, in which case, this is the best possible experience for any Uno app.

Instant pass

Hoping someone snarfed a git clone, revision history is waaay more interesting than the code itself

Running a site the size of the BBC on Lambda is nothing short of an exuberant waste of a government-subsidized budget, it's absolutely crazy.

Lambda VM time has a massive markup compared to regular compute. It only makes sense where usage does not exceed some threshold the BBC absolutely certainly do.

There are plenty of alternative options, even on AWS, that don't suffer from such huge markup without requiring any additional ops input. The thing that runs in Lambda is practically a container image already. Does it really cost tangible budget to have CI build a full image rather than a ZIP file that already contains a few million lines of third party JS/Python deps?

IMHO this is the epitome of serverless gone wrong.

This entire post is just unnecessary fear-mongering.

You perhaps missed the bottom 30% suggesting there may be better options.

Docker's registry protocol treats layers essentially as opaque slabs, as any user will no doubt be aware, where update of a single timestamp or byte of a file could mean the reupload (and download) of a potentially unlimited amount of data. While that architecture has its perks (simplicity, sequential IO patterns suited to the magnetic disks of 2013), there are obvious downsides.

We have plenty of systems just like Docker capable of managing Docker-like file counts without the obscene bandwidth abuse, we just don't call them registries. They have names like Perforce, Subversion and even (some Microsoft variant) of Git. Eliminating the data gravity angle is impossible, but reducing its potency by several orders of magnitude is not only possible but has already been done repeatedly in many popularly deployed systems.

This is not to suggest replacing registry with Git, but perhaps some graph-like manifest format combined with some new "fetch-multiple-objects" protocol verb would be all required to bring the cost of running a registry down to essentially a non-issue.

$45/mo would be the minimum payment, assuming the free limit announced today actually lasted. Let's say they begin crying 12 months from now and knock it down even by 50%, now you're looking at $810 minimum per year per client site. Multiply that up by, say, 1000 sites, and you're starting to look at a "free" service that might completely pay for its own development within a year or two.

500GB/mo. presumably per IP address for anonymous pulls. Amazon have absolutely no problem revealing the true cost of egress when it suits them. Also, "500GB/mo.", at launch. Let's not forget those generous free limits can be slashed whenever it suits, and why they might be so generous for launch at all.

Don't be mistaken, actually using this for public images is falling for another "data gravity" marketing gimmick. So they attract enough public images that some folk find one or more of their services absolutely must pull from AWS, pushing their corporate NAT address over the 41GB/day limit. Now your "free" hosting has become a reason for that company to start paying ~$45/month to AWS in egress fees just to use your images, and perhaps with a new ticket in the backlog to consider simply moving the cluster to AWS.

"But it'll always be like that regardless of provider", not necessarily. The reason that Docker registry consumes a ridiculous amount of bandwidth is largely an architectural issue, both in the software and the design of the client's network. Perhaps a better direction is addressing this immense waste of networking rather than turning it into another free marketing opportunity for a near-monopolistic cloud

It really has instant appeal to me as an adult, but I'm not so sure about target market any more. Kids these days are all glued to their mobile phones, it seems if you want to blow a kid's mind, something that turns their phone into a magic box they can tinker with would be so much better. Something maybe like a combo keyboard with IO breakout ports funnelled over bluetooth, and some matching software development environment closer to GWBASIC or maybe even VB3 than the usual Android toolchain

I'm also concerned this doesn't mention much about how accessible the internals are. Can a kid crack it open?

Meanwhile, totally digging the nostalgia factor. Probably won't buy one, but damn I'd love one

Docs and Sheets

Try out Office online, insanely more featureful. Scold anyone mailing Docs share links by saying you can't open it, insist on a file instead

PyPy is the awesomest thing created since sliced bread, but C extension interoperability is still a source of perf problems in a few scenarios AIUI. Glad to have Pyston in addition to PyPy

Too little for what and too late for what? Qt has a massive install base in embedded and desktop, and plenty of weird niches like kiosk and interactive media. Also try building out some specialized UI featuring any kind of heavy computation and IO, there are very few practical choices for this. Qt is one of them

This page feels almost like quackery, however Knuteson's background is part physics professor part quant. See https://www.bruceknuteson.com/

They release periodic delta torrents of the entire collection. I won't link it here, but it's trivial to find