vii

Even worse, it normalizes being a vindictive egotist. People need to be grown up about feedback especially if they're in powerful positions. Not ok to be offended.

After having interviewed hundreds of engineers, I agree with the premise that elite TopCoder competition programmers are great not just at brainteasers. It takes a lot of discipline to improve skill to solve that fast, so they are smart and hardworking and can ace the design interviews too.

However, the fact is these people are inundated with job offers and very generous swag - like laptops. Everybody wants to hire them. It's hard to compete for one of these candidates let alone fill an engineering team. Centering hiring around that is an act few can follow.

Some of this advice is great, like getting out of the way, guiding people with how to think about trade-offs and doing daily coding. But it doesn't feel like 'Principal' level advice, at least in terms of Big Tech and the blog notes the author isn't sure what the distinction is with Staff.

The author complains that it's hard to be in the critical path at a senior level. This lacks self-awareness. It's always hard to be in the critical path. Shipping on time is one of the toughest deliverables of the software engineer role, and one that many people struggle with. Accurately estimating development costs including wall time vs. actual time someone has to work on a project is a very important skill. It's not acceptable for senior engineers to abrogate responsibility for this, especially if they claim to be mentoring other engineers.

Senior engineers own the business outcome and must weigh costs of all kinds, from security risks to technical debt. As scope increases, the feedback loops get longer and longer. A new engineer can tell if they did well with a comprehensive unit test. A junior engineer can tell if they did well with a performance or integration test. A senior engineer can tell if they did well with an A-B test in the market. A staff engineer can tell if they did well by seeing market share grow.

In Big Tech, senior staff and principal roles carry the idea of doing something to 'shock the world' - that is, successfully shipping innovation that people were afraid to, for example, because it seemed risky. Greasing the wheels of communication between teams and helping people avoid common mistakes is fine and a good thing. But there is limited business value in building consensus around the latest "architecture" or framework or language or whatever, however nice it feels to enjoy the social status as the person turned to for this kind of question. Step change innovation is the real value add and this article hardly touches on it.

Is that a false claim?

The expectation that an SSD or even SD-card is infinitely rewriteable (practically speaking) is implicit in the marketing.

The idea that a drive can't be rewritten too many times should be explicit (like SMR HDDs)

Being considered the go-to person means you are first in line to make small tweaks (and config changes in code).

This increases the change count.

If there is a review process where another engineer has to approve work, this exacerbates the gap, as the go-to person can get their reviews done quickly. If they're trusted, the reviews might not be thorough.

This increases the rate at which changes go in.

These and other factors suggest that it's hard to split cause and effect here. Being seen as productive increases change count :)

Rob Pike wrote this about handling errors https://blog.golang.org/errors-are-values

This is like an option type but manually built.

Go is painful if you don't want to write high-quality heavily exercised production code. If you do, then you should be considering each error condition. Most code doesn't need to worry about error handling as it will run under close supervision on at most one machine over its lifetime. Go is a nuisance for that.

well that's the subject of the Linux kernel thread as the original patch was just to remove a warning from the use of the Intel MSR https://lore.kernel.org/lkml/20200907094843.1949-1-Jason@zx2...

I guess it's a question of whether you want drivers to be in the kernel source tree with supported interfaces, where interactions with other drivers can be mediated, or in userspace, where they can iterate without the kernel release process, which takes a long time to get to distros and end-users.

As this Intel MSR is not well documented, I would not argue strongly either way.

This write up covers a lot of ground from specific issues around language and protocol transport layer choice to general advice that organisational change without a strong senior sponsor is hard. If you feel that quality needs to be increased the first question you need to ask is whether that is a widely held view. People have different expectations of trade-offs and may even benefit from noisy failures - which can grant headcount and greater focus. It might really not make sense to improve something that will be retired soon.

In this video https://www.youtube.com/watch?v=DpO1Tfa4IZ4 keynote, Amin Vahdat explains how he led a transnational approach to reliability in a huge complex system. In response to a question about whether hiring should be changed to increase reliability he says no - just that it needs to be measured and emphasised as a priority.

Coming up with the idea of a portable personal computing device is easy. It's even possible to manufacture it (e.g. the Apple Newton). However it won't affect people's lives until it is quite polished. Similarly figuring out that people want to stream all their music is easy. The innovation in getting it to work, overcoming complex technological and social barriers is hard to explain to outsiders and less obvious. But it opens up these 'old things' to new people who couldn't access them before.

For another unicorn, clearly opening up new things to people: Oculus, though it wasn't the first virtual reality headset.

There were plenty of platform specific problems with Flash. Steve Jobs famously ranted about it https://en.wikipedia.org/wiki/Thoughts_on_Flash

That said, cross platform compatibility is always going to be hard. I'm still amazed that Adobe let that platform die. It has taken so long and so much effort to recover equivalent functionality, and in some cases we are still lagging what Flash could do.

For the recursive functor application example on a tree, normal meta-programming would be a macro taking the type as an argument, and then generating the resulting case by case application code, at compile time.

This would be much clearer and could automatically keep working if more cases were added (adding new node types is common when adding language features to abstract syntax trees).

This is a very awesome demo but my experience with other templating systems in IDEs (like autogenerated toString and other methods in Java) is that making boilerplate easy to write doesn't help make it easier to read or modify.

This paper is surprisingly accessibly written given the complexities involved and depth of expertise of the author, DJB, who has made great contributions to many fields.

Unfortunately nearly all decision making processes are vulnerable to this kind of attack: processes are never clear, selection criteria always up for debate, and generally people pick a special option to present preferentially.

This generally just leads to suboptimal decision making - we're lucky to have DJB's focus here to improve this one.

The power of the Apple brand is so extreme that the fact that many users experience near weekly or daily crashes from this bug (or series of bugs) does not convince them to buy other machines.

The CalDigit TS3 dock is pretty solid for me on Linux, though sometimes it doesn't power on the DisplayPorts and the dock itself needs to be reset. I have a smart plug to do that.

However - given the high performance and deep system integration (PCI,USB,video) that Thunderbolt provides it's fairly extraordinary that hotplug works at all!

Setting a good objective function is pretty hard. In this context of consumer goods, it is at the intersection of three difficult problems:

- equivalent to incentivising salespeople, which is known to be very difficult, as short term incentives often are in opposition to long term ones

- distinguishing and dealing with spammers, robots and crawlers

- and setting up a stable reinforcement learning behaviour even for the short term, which is tough even without the first two problems

For these reasons, naturally business partners, designers, and others will be very curious how the bandit affects the customer experience.

Many years ago to solve this I made a system that would emit a list of (suboptimal) rules to exploit the opportunities learnt from small A-B test groups (like an epsilon greedy contextual bandit). These rules were reviewed by relevant stakeholders and then explicitly deployed to production as a configuration change, which allows for manual consideration of issues in the three above areas that are hard to automate.

There are many alternatives for distributed tracing like Lightstep, Jaeger and so on but the ambitious level of integration with log searching (like ELK) and payload tracking is like an integrated in-house Splunk. Great idea and great to see the energy and enthusiasm put into making debugging tools better! One dream feature for a tool like this: code execution counts showing which version of the code and even which lines were executed - in aggregate is useful but ideally for each trace.

Unfortunately, the tradeoff of value gained saving debugging time against cost of infrastructure and development is hard to manage. The storage costs are very easy to measure so it is tempting to go after them rather than the more intangible benefits that rely on a counterfactual of how hard things would be to debug without it.

The title is a little confusing as DNS over HTTPS is the technology that comes to mind for DNS privacy. That hides the DNS queries in an HTTPS tunnel, which means fewer actors can see or spoof DNS results.

The QNAME minimisation technique described in the article is about showing only partial requests to intermediate authorities in the DNS hierarchy. DNS over HTTPS can protect the request until it gets to the resolver, and then the QNAME minimisation system can hide it from interested intermediate DNS authorities. I guess in practice this means the .com servers can't tell whether someone is going to xxx.substack.com or yyy.substack.com just that they're asking about substack.

As the article points out, most users ask a shared DNS resolver to perform resolution for them, and if you want to, the Cloudflare public resolver 1.1.1.1 uses this technique.

The recommendation algorithm generally tries to avoid showing directly similar content. It's surprising it is all cats especially if you ever swipe. Perhaps this is an A-B test and you are in a group with lower boredom avoidance settings :)

Can you explain a bit more about Elite TikTok? I found this article but it contains quite a jumble of ideas https://www.theguardian.com/culture/2020/jun/22/what-is-elit...

Same for the Nintendo Switch.

Older firmware has exploits, which allow installing Ubuntu, etc. To prevent exploiting, old firmware refuses to boot if the new firmware has ever booted. This is marked by blowing fuses.

Note - there is even a fuse that when blown, prevents other fuses from being blown!

https://docs.nvidia.com/jetson/archives/l4t-archived/l4t-323...

It's not surprising that a simple theorem doesn't deal with practical trade-offs. In my experience if you feel you are running up against the CAP theorem you need to take a moment to think about the real needs of your application and looking for a more detailed theorem isn't going to guide you well, instead you need to think about what you actually want to happen when various kinds of failure occur. In finance for example, often trading systems will choose to be available and accept transactions when there is a failure and then reconcile afterwards - swapping the risk of making mistakes but in a trusted environment to prevent the potentially catastrophic condition of not being able to trade.

The article claims that the CAP theorem only prohibits linearisability as a consistency model. The original CAP paper talks about "Atomic consistency" which is clearly at least linearisability. However as the proof formalizes, it's obvious that if you accept writes to totally split and separated systems which cannot communicate together (network partition), you cannot be consistent across the two parts. This is confusing because this kind of consistency only loosely matches the definitions in the ACID acronym which are geared around invalid database states in a single machine database, not about different requests to different database machines giving inconsistent answers.

Linearizability versus serialisability (and strict serialisability) are explained well by Peter Bailis http://www.bailis.org/blog/linearizability-versus-serializab...

From the underlying paper there are several interesting snippets https://arxiv.org/pdf/2009.01694.pdf

while maintainers are aware of that they sometimes intentionally bypass the process, they were surprised of the magnitude of unreviewed patches

Would love to see an analysis of these changes - are they just simple merge style fixes or rearrangements, or more significant?

And then there is the hard to define distinction between a security bug and a normal bug, which is then mixed into the the incredible productivity and pace of kernel development:

Koah-Hartman argues that only a small fraction of Linux kernel security fixes are assigned to CVE entries. From 2006-2018, 1005 CVEs were assigned to the kernel. He argues that, on average, bugs with CVE entries are 100 days fixed in mainline before they get a CVE assigned.

Seems there is long lag between the bug being introduced and the exploit discovered, so there must be many potential security exploits that are never discovered before they are fixed - and so are not practically exploitable as they never get into downstream distribution kernels.

Well it might blow up some software not expecting these tiny tick sizes but there is an economic motive for larger tick sizes.

Traders can compete on

- price

- queue position

Small ticks mean they compete on price and big ticks mean they compete on queue position - to trade they need to have an order on the book, offering to trade at the current tick, that is ahead of other orders.

It's easy to see how the market benefits if people compete on price. However, it also benefits if people show how much they are willing to buy and sell. No sophisticated trader wants to reveal that as they will be taken advantage of when they are wrong. By having bigger tick sizes you incentivise people to try to get into the queue at these artificially better prices - it pulls liquidity into the open.

The tick size pilot that concluded in 2019 shows how this balance isn't easy to strike https://www.finra.org/rules-guidance/key-topics/tick-size-pi...

It's definitely true that case-folding is generalisable to more generous equivalence notions - for example, more or less whitespace isn't a noticeable difference that people typically want two files for. People casually using their phones want to treat the filesystem as a search tool where fuzzy matches (across different directories and names) are reasonable responses to file system queries. There have been many attempts at it like WinFS, but that's a significant increase in scope from case-folding.

One incontrovertible justification for case-folding is to support Samba shares efficiently. Samba has to implement case-folding to clients and if it is not done in the filesystem datastructures you have to read in many directory entries to figure out if one would match with case-folding.

There is plenty of MS Windows and MacOS software that just expects case-folding from the operating system.

The article is old and this was released in 5.2. It can be turned on per (empty) directory with chattr https://kernelnewbies.org/Linux_5.2#Optional_support_for_cas...

that's what the waiters told us!

But here is someone who claims to have found the recipe: http://www.nogarlicnoonions.com/lentrecote-restaurant-famous...

* One chicken liver

* One shallot

* One sprig fresh thyme, tarragon, parsley

* 10 cl of liquid cream

* One tablespoon of Dijon mustard

* 20 g butter

* 3 cups of water

The practices identified here make sense. Testing pipelines on a small amount of data to make sure there are not simple typos, then doing aggregate statistical tests and specific example condition tests will catch many issues.

The paper "Making Contextual Decisions with Low Technical Debt" https://arxiv.org/pdf/1606.03966.pdf goes deeper. Testing and monitoring deployments are very similar. The idea of shadow testing new models (seeing how their outputs would differ from the production models on real data) has been very important for identifying issues in my experience.

This can be generalised to comparing models on historic data which greatly speeds up evaluation. This is different from cross-validation as it is not about correctness, just how different the new output is. This is like the pattern in UX development of a test harness that compares differences in a screenhot. If the differences look good, then ship it!

Single menu item steak frites restaurants are available in NYC and London and apparently Mexico City - https://relaisdevenise.com/ They import the secret sauce from France.

These restaurants also make it easy to split the bill as everybody has the same thing!

The idea of layering security is very powerful, acknowledged in this "Network Operations Division Cryptographic Requirements" document

https://wikileaks.org/ciav7p1/cms/files/NOD%20Cryptographic%...

Certificate validation must not be performed against any standard SSL root CAs.

implement an inner cryptostream within the SSL tunnel transfer

People often state that you should not roll your own crypto. Definitely, you look foolish for making a mistake doing your own thing. However, adding your own layer on top of a standard one seems safe and likely to slow an adversary down considerably. Adding a layer below to encrypt data before the standard algorithm gets it has some risks (e.g. could leak in some complex way like a timing attack) but it also protects against a compromise in the implementation of the standard algorithm.

Adding the Horcrux layer of multiple channels does seem to increase security at the cost of creating a new unvalidated magic wand that then becomes the attack surface - and another significant cost in that it is not user friendly and involves considerable effort per message. There are ways of implementing greater security at high cost, e.g. point-to-point communications off network. The question is if the extra effort confers any benefit. Sometimes just the fact that two parties are communicating is valuable knowledge and this Horcrux mechanism actually makes that easier to detect as it occurs across multiple systems.

they also estimate close to a petabyte of RAM; if all the RAM is spent in leaves that is about 6-7GB per leaf. I don't know that we can say it would be unreasonable to have 1 core per leaf replica; presumably some leaves have low utilisation so it might make sense to share the core with another workload. From a capacity planning standpoint, I think they leave this open though they do indicate that they are sometimes CPU bound so don't try to compress beyond delta compression. That might suggest multiple cores per leaf.

A single core design allows a very simple concurrency model, without having to worry about cache pingponging, false sharing, or myriad other issues. The parallelism is applied at higher layers, as there are multiple replicas for each leaf and obviously they can use many cores effectively overall.

I don't see that the paper gives enough information to help us prune the design space here.

Wow great demo clip!

Did you try a Sobel filter https://godoc.org/gocv.io/x/gocv#Sobel ? Curious if that could give more contiguous joined up edges than AdaptiveThreshold.

From https://github.com/Kadle11/GoFlip/blob/master/src/makeCartoo... looks like decompressed frames are all stored in an array. For larger videos or lower memory machines might be handy to process each frame through the pipeline rather than batching.

If it were just that, then overall the project would be dangerous trap, as there is a big cost of added complexity from the new high level configuration language with its limitations and own terminology (volumes, etc).

Adding a wrapper, and then eventually forcing you to learn all the abstractions that leak through, creates an attractive nuisance. The Hyscale project is at least trying to overcome this problem. Not sure how well they succeed.

Along with the high level config it attempts to help untangle common K8s debugging steps, which normally require using multiple tools to determine what caused an error condition like CrashLoopBackoff - see https://github.com/hyscale/hyscale/wiki/App-centric-Troubles...

Looking at the code they painfully enumerate different Docker and K8s options in Java, so it will be expensive to maintain and keep up to date - the host company Primati may have the resources to do this and that's exciting!

It's not ok to set goals and let people just get on with them. The result of that is the most extraordinary implementations; it's just not possible to lock down requirements clearly enough to have the right outcomes come out without massive communication of implicit context.

The contradiction in management is that you must somehow know what's going on, but it is not helpful to interfere constantly.

I don't see how this framework helps with what I think is the most difficult problem in management: how to deal with talented people who aren't connecting to impact in their current roles. If you don't reward them, they'll leave. If you reward them, everybody else will be justifiably jealous. Setting interfaces doesn't help you figure out how to unblock them and might make it worse.