HN user

owlbite

540 karma
Posts0
Comments191
View on HN
No posts found.

I'm still astounded my Ford's infotainment system takes 5-10 seconds after opening the radio "app" to present me with the station selection UI. Most of that seems to be some sort of startup delay. The mind boggles.

Or economies of liability and buck passing. I suspect managers and businesses will still want to be in the game of "not my fault, supplier is working on it, we can sue them if they don't meet SLA".

FORmula TRANslation, the clue is in the name. It's great at math, but yeah, strings and OS stuff is a PITA. The modern vector-based syntax is still really nice and I've yet to come across a C++ library quite as slick.

But I think what it was really missing last time I looked at it was good access to compiler intrinsics (or otherwise) to hit vectorizations and math optimization instructions. The OpenMP simd pragmas weren't really doing a fantastic job. I hope that's better now it's in LLVM.

I think the current generation of tools have a long way to go before I trust any numerical algorithm they implement, based on our recent experiments trying to make it implement some linear algebra by calling LAPACK. When we asked it to write some sparse linear algebra code based on some more obscure graph algorithms it produced some ugly stepchild of dijkstra's algorithm instead, which needless to say did not achieve the desired aim.

Have meetings to figure out how to interact with the other 9990 employees. Then try and make the skeleton app left behind by the team of transient engineers who left after 18 months before moving on to their next gig work, before throwing it out and starting again from scratch.

This book provides a high level overview of many methods without (on a quick skim) really hinting at the practical usage. Basically this reads as a encyclopedia to me, whereas Nocedal and Wright is more of an introductory graduate course going into significantly more detail on a smaller selection of algorithms (generally those that are more commonly used).

Picking on what I'd consider one of the major workhorse methods of continous constrained optimization, Interior Point Methods get a 2-3 page super high level summary in this book. Nocedal and Wright give an entire chapter on the topic (~25 pages) (which of course still is probably insufficient detail to implement anything like a competitive solver).

But it can be even worse than that. It's "we assassinated the phone", "algorithm says vehicle has suspicious travel history and must die". There's no real thinking human in the loop for some of this stuff, just some model decided the metadata has a high probability of being associate with an opponent of some flavor and then everyone in the vicinity is blown to bits as computer said kill.

Very true, but a lot of stuff builds on a few core optimized libraries like BLAS/LAPACK, and picking up a build of those targeted at a modern microarchitecture can give you 10x or more compared to a non-targeted build.

That said, most of those packages will just read the hardware capability from the OS and dispatch an appropriate codepath anyway. You maybe save some code footprint by restricting the number of codepaths it needs to compile.

Oh yes, from an actual implementation POV you can just apply some transpose and ordering transforms to convert from row major to column major or vice-versa. cblas is pretty universal though I don't think any LAPACK C API ever gained as wide support for non column-major usage (and actually has some routines where you can't just pull transpose tricks for the transformation).

Certain layouts have performance advantages for certain operations on certain microarchitectures due to data access patterns (especially for level 2 BLAS), but that's largely irrelevant to historical discussion of the API's evolution.

What I suspect he really means is that FORTRAN lays out its arrays column-major, whilst C choose row-major. Historically most math software was written in the former, including the de facto standard BLAS and LAPACK APIs used for most linear algebra. Mix-and-matching memory layouts is a recipe for confusion and bugs, so "mathematicians" (which I'll read as people writing a lot of non-ML matrix-related code) tend to prefer to stick with column major.

Of course things have moved on since then and a lot of software these days is written in languages that inherited their array ordering from C, leading to much fun and confusion.

The other gotcha with a lot of these APIs is of course 0 vs 1-based array numbering.

Why we need SIMD 10 months ago

Much better to burn the area for multiple smaller units, its a bit more area for frontend handling, but worth it for the flexibility (see Apple's M-series chips vs intel avx*).

So my experience with moving from a Cambridge undergrad to an Edinburgh postgrad, albeit a couple of decades ago now, was the expectations between the two were nowhere comparable.

Cambridge if you'd not done the homework before the tutorial, you got sent packing for wasting everyone's time, but in Edinburgh it was common for all but the best students to only start the homework at the tutorial (thus wasting their opportunity to ask questions on trivial stuff they could get by reading the course notes.

Equally on exams, the minimum standard at Cambridge was "regurgitate proof from course notes" with the other 2/3rds of the marks for iterating on it with unseen material, whereas the Edinburgh exams the regurgitation would get you 100%.

Unless things have changed significantly (or Edinburgh is that much worth than other redbricks), I'm not sure I trust these rankings in terms of student quality.

That and the speed limits are actually enforced, at least significantly more so than they seem to be here in California:

- Lots more speed cameras

- Average speed cameras especially make a huge difference vs spot enforcement

- Tolerance for enforcement is normally 10% rather than 10 mph (i.e. 30 limit means no more than 33mph rather than no more than 40mph)

Trying to reduce high end processor performance to "operation X takes Y cycles" likely confuses the uninitiated more than it helps once you get beyond "cache miss bad".

For the uninitiated, most high-performance CPUs of recent years:

- Are massively out-of-order. It will run any operation that has all inputs satisfied in the next slot of the right type available.

- Have multiple functional units. A recent apple CPU can and will run 5+ different integer ops, 3+ load/stores and 3+ floating point ops per cycle if it can feed them all. And it may well do zero-cost register renames on the fly for "free".

- Functional units are pipelined, you can throw 1 op in the front end of the pipe each cycle, but the result sometimes isn't available for consumption until maybe 3-20 cycles later (latency depends on the type of the op and if it can bypass into the next op executed).

- They will speculate on branch results and if they get them wrong it needs to flush the pipeline and do the right thing.

- Assorted hazards may give +/- on the timing you might get in a different situation.