deadgrey19

This is the trouble with submitting to OS conferences. Their networking knowledge is typically limited. I’ve seen the same thing happen in the past.

FWIW - I don't think it would be that hard to port VMA or ExaSOCK to run on top of DPDK, certainly less effort than making a whole new category of operating system!

But ... selling the hardware is where those companies make their money, so there's not much of a business case for making your free and open-source software run on other people's (cheap) NICs.

Fair enough. My guess is that the commercial solutions are simply going where the money is. If there was a commercially pressing need to port to other systems, I’m sure it can be done.

With that said, it's one thing to have a legitimate reason for preferring an alternative approach. But the paper doesn't even mention these solutions, which means that the authors are either ignorant of what's already been done (bad), or deliberately avoiding comparisons to alternative approaches (worse).

Yet another academic work in the OS space that should have been tested against the the state of the art in (commercially backed) networking space.

From the abstract:

However, none of these systems offer a general-purpose datapath OS replacement that meet the needs of µs-scale systems

One of the key points that the authors use to motivate their work is that using kernel bypass technologies is hard because it requires software engineers to rearchitect their code.

Call me sceptical of that claim. I can think of 3 (open source) systems that provide drop-in replacement network interfaces for existing software, but offer nanosecond (not microsecond!) scale performance.

- Solarflare/Xilinx Open Onload - https://github.com/Xilinx-CNS/onload

- Exablaze/Cisco - ExaSOCK - https://github.com/cisco/exanic-software

- Melanox/NVidia VMA - https://github.com/Mellanox/libvma

With each of the above systems, they are designed to offer low latency network stack replacement for existing applications, without code modification or even recompilation. Exactly the authors’ goals.

But there is not a single reference to any of these technologies in the paper. And crucially no comparison to any of them in the evaluation.

The authors seem blissfully unaware of what work has actually been done to solve these problems. Which makes the contributions of the paper … somewhat dubious.

Exactly this happened to me. We bought the TV and for years things were great. Really loved the functionality of the smart TV. Being able to stream YouTube at a moments notice was great.

But then the adverts started. We paid $3000 for this TV. Not a low end “ad-supported tv” The only way to “opt out” is to disable the “smart” TV function which I have done. Any smarts now comes from my Apple TV which does not thrust averts in my face. Needless to say, my last Samsung purchase ever.

Basically the entire internet is built on TCP. But TCP wasn't always very good. There was a time when congestion nearly killed the early internet. Congestion avoidance and control talks about how they fixed it. It's light hearted, accessible and one of my all time favourite reads:

https://dl.acm.org/doi/10.1145/52324.52356

This is a standard second year CS course at UNSW: https://cgi.cse.unsw.edu.au/~cs2041/19T2/index.html

Topcis included: Shell Perl Git Javascript Webservers Make Performance tuning

I think you're missing the point. I'm talking about the concept of mathematically modelling the program first, before writing it (as per the article). This process only applies to certain types of applications, ones that are rooted in algorithmic transformations (like compression), rather than in system and I/O operations, like copying a packet from a network RX buffer into a disk block, in the most efficient way.

What I'm discussing here is the general concept of modelling your program formally before writing it (as per the article). What I'm arguing is that this type of approach is only possible for a certain set of applications which take the form of y = f(x), where f(x) is some type of data transformation /computation operation (e.g. calculate the GCD of these ints, find the shortest path through a given graph, sort this set etc). There's another set of applications which I/O bound, are very important, and yet, the "computation" that they preform is limited to none. These applications are rooted in, and bounded by system parameters, like understanding how disks work, how network cards work, how CPUs work etc. This is an optimisation problem, but not one that can be modelled mathematically (in any reasonable way) because of the vast complexities of the system. Building a TLA+ "proof" of this system will reduce trivially to x = x, and yet, the system is still important, and difficult to write well.

How so? Can you give an example?

Thinking in math first has been the catch cry of functional programmers (and their formal logic/verification friends) for decades. And there's nothing wrong with it, unless the problem you are trying to solve actually requires performance. Then, you have to think in "system" first. For example: Write a program that captures network packets and stores them to disk as fast as possible. There's no maths to think of here. The complexity is all in the "fast" part, and to solve that, a deep understanding of the architecture of the system is necessary. Fancy algorithms (maths) will not help you here. e.g. Will compression help? Depends on the properties of the system. Will batching help? Depends on the system. Will threading help? Depends on the system.

+100 to this. I had him for H Comp 1A, literally changed my life .

Yes! This and only this! The posted "bit hacks" are little more than bit level manipulations exactly as intended. The Stanford hacks are truly excellent. My favourite is: https://graphics.stanford.edu/~seander/bithacks.html#RoundUp...

Congestion avoidance and control, by Van Jacobson looks at how TCP, the fundamental protocol behind the internet, works (or didn't work to begin with). It's a pretty easy paper to digest without too many dependencies, especially if you skip the maths (which is explained intuitively anyway). And it's a fascinating read. I highly recommend it!

https://nsl.cs.sfu.ca/papers/Jaco88.pdf

FPGAs are not a new thing. But FPGAs that are big enough and fast enough to outperform ASICs (in the networking context) is a new thing.

Re the title. It's perhaps a bit poetic, but it's not misleading. What else could the title possibly mean? It's either going to be about 3D printing or FPGAs. You can't seriously expect to actually download a physical piece of hardware.

A bit of ongoing work from me. Trying to make wiring C programs fast and painless.

https://github.com/mgrosvenor/libchaste

NetFPGA is a great development board with a lot of features. Unfortunately its size and research quality software/firmware do not make it a suitable device for deployment at scale into production environments.

Ha. Thanks yes. Too busy selling stuff to get around to fixing the website. It's on the list of things to do...

The company I work for builds exactly that. NICs, which are actually FPGA's with area larger than what is necessary to build the NIC alone. Our customers then either just use the NIC or add custom firmware for their own applications.

http://exablaze.com/exanic-x10

http://exablaze.com/exanic-x40

We also build switches this way too:

http://exablaze.com/exalink-fusion

This idea was the subject of a paper at a major systems conference. The paper is called "Scalability! But at what cost?" - It goes well beyond this simple example above to explore how most major systems papers produce results that can be beaten by a single laptop. Here's the paper and the blog post describing it.

http://www.frankmcsherry.org/assets/COST.pdf

http://www.frankmcsherry.org/graph/scalability/cost/2015/01/...

Without wanting to start a build system flame war. I've had very good success with this for both c and c++ projects.

https://github.com/Zomojo/Cake

For an IDE I use the eclipse CDT which has improved I leaps and bounds. The one thing I do is make sure I give eclipse enough memory (a couple gig usually so that its snappy)

IMHO, a good manager would communicate the importance of the project along with the spec of the project. "Because I said so, so do it" is not a good strategy for highly skilled, highly competent, highly intelligent people. The "why" is every bit as important as the "what". Presumably you wouldn't "waste" a good programmer on a simple task if it wasn't important. Similarly a good programmer would understand that business priorities means that some projects are complex and interesting, some are important and boring.

As an employee, if the manager is not a skilled communicator, it's not the sort of place I want to work for. As a manager, if the employee is cannot understand business priorities, it's not the sort of employee that I want.

Being asked to write yet another depth first search implementation (honestly, 5/5 interviews at a large corporation were the same question) that any second year student can do is humiliating when your daily bread is building complex distributed systems. I still did it, because (unlike the writer) I understand that you do need to establish a baseline. As I've said, I've sat on the other side of the table, asked the humiliating question and seen the candidate fail miserably. It's a necessary evil, but it's still humiliating.

A less humiliating but still valuable interview (IMHO) is a systems design question. It should still be a whiteboard exercise, but it's a better test of problem solving and engineering skills. I don't seriously begin any system work without first doing the very same thing on a whiteboard myself. It's a useful on the job skill and a useful way to test skills of communication and collaboration in describing the solution.

With respect, if you're putting your employees to work on "crapy little side project's" for a month, either the projects are not crappy and have potentially large impact on the firm, or, you're wasting valuable resources.

I won't give away the details, but the warm-up question should have taken a about 14 seconds for someone with the claimed skill set. They should have been able to speak the code at us rather than use a whiteboard. Think of something like "write a program to sum the values in a 10 element integer array". It was supposed to be a fluff filter so we could get on to higher level architecture/design questions. I felt embarrassed to even give it. On resume alone, this guy was a definite hire. The interview was something else entirely.

White board interviews are humiliating, especially if the level is pitched wrong. As someone with a career prior to starting grad school I'm sick of being treated like a child in the interview process and being given diddly little graph traversal problems when I used to building entire systems. The problem here is a mismatch in expectations. But the whiteboard interview does have its merits. I have sat on the other side of that table. We took a candidate with years of experience a masters degree from a prestigious university and a very very impressive thesis. We were almost embarrassed to give him the warm up questions. Until he completely utterly failed to complete them. We didn't even get past the warm up questions during the two hour interview. As a baseline measure of competence, it was a pretty good one and we (in a small startup) really dodged a bullet by doing this kind of testing.

Xeon Phi is around and used in various financial computations. The problem with GPUs is not TDP, but programming flexibility. Programming a GPU is very hard to do well for generic computations, whereas x86 like CPUs have millions of programmers who can do a pretty decent job.

Maybe they mean "effective" speed. 1000x 1.78GHz = 1.78THz.

Assuming a job is embarrassingly parallel it might be an ok metric. Realistically, probably not.

Citations give a ranking of how often work is mentioned, but not how valuable the work is. For example " Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children" , otherwise known as "Vaccines Cause Autism" has (according to google scholar) over 2000 citations, but is known to be wrong and based on bad science.

How is that different? Neither Scala nor OCaml rank in the top 10 programming languages according to several metrics/organisations [1-5]. Keeping in mind that programming language use is a heavily skewed distribution, OCaml and Scala have almost zero usage relative to the big ones. So, again I must ask, how is this relevant to the masses?

[1] http://www.tiobe.com/tiobe_index

[2] http://pypl.github.io/PYPL.html

[3] http://spectrum.ieee.org/computing/software/the-2015-top-ten...

[4] http://redmonk.com/sogrady/2016/02/19/language-rankings-1-16...

[5] http://www.codingdojo.com/blog/9-most-in-demand-programming-...

This sounded so promising until the OCaml part. The problem is, the masses don't program in OCaml. They program in C# or Java, or C++. So this is really lock-free programming for OCaml, which is a lot less relevant.