HN user

eeks

420 karma
Posts4
Comments133
View on HN
Verilog Is Weird 4 years ago

I had a similar experience with HardCaml back in 2015/2016. The benefits where overwhelming: higher productivity, higher reusability, less intractable bugs, tighter TTM, and no more slippage in deliverables timeline. The performance where comparable to Verilog-only project (density, path length). In the end, the effort was shut down by management because the approach was “too complicated”.

Some references to the mentioned works, missing from the OP:

DB2 BLU: https://researcher.watson.ibm.com/researcher/files/us-ipandi...

Wildfire "v0", the "failed" HTAP project: https://researcher.watson.ibm.com/researcher/files/us-ytian/...

Wildfire "v1", the shared-nothing architecture over cloud storage (EventStore 1.0?): http://cidrdb.org/cidr2017/papers/p123-barber-cidr17.pdf

DB2 EventStore (EventStore 2.0?): http://www.vldb.org/pvldb/vol13/p3299-garcia-arellano.pdf

You may be right about the DE -> Fellow promotion, although I only heard of it once, and that person was pretty spectacular. The Fellow -> DE path is unheard of. As is the RSM -> DE. All the DEs I know came from SWG and were STSM.

Re: my “dilution” comment, i was not thinking about DEs. I was thinking about non-technical people (management, law, accounting, ...) that have been creeping up the Fellowship lately. Also the bluewashed executive that were given a fellowship package to seeeten the pot.

The IBM ranks are inaccurate. Fellows are not above DEs. Both are the highest ranks indeed, but of two different tracks. DE is the highest rank of the engineering track, while Fellow is the highest rank of the research track. Fellowship have also be granted to non-research people over the past 20 years, diluting the merit of the title IMHO. Examples of fellows are: John Backus, Gene Amdahl, Ken Iverson, Benoit Mandelbrot, etc.

Both levels may reside in different pay bands, but they are both at the executive level. And DEs cannot become fellows, and vice versa (some exceptions to the rule may have existed).

I was in undergrad and grad school in Grenoble where I had F. Maraninchi and P. Caspi as teachers for the synchronous programming and VLSI classes (embedded system track). We did a lot of Lustre. It's a pretty great way to get acquainted with synchronous programming and all the fun stuff that derives from it (cf. SDF, code generation, soft/hard real-time, and FPGA programming). Fun times.

It certainly is not "the French". It's the "French Republic". Local dialects lived happily in France for over a Millennia under the Monarchy. The French dialect, mostly spoken in the Jacobin circles in Paris, was imposed by the revolutionaries as a mean to achieve "equality".

Niklaus Wirth is one of those unsung geniuses of computer science. He invented Pascal, the Modulas, Oberon ...

Then he woke up one morning wanting to reboot his life's work for his class and, undeterred by the fact the original targeted processor did not exist anymore, settled to write his own computer architecture on a Xilinx Spartan.

He then successfully ported the Oberon system and compiler to this architecture to demonstrate to his students.

Just wow.

I think using TCP, but I don't recall for sure

It was UDP.

Which is to say, you're saying "detrimental impact on performance" and yet this seems to be a significant win over BSD sockets.

The point I was trying to drive home is that, while 30% overhead reduction compared to BSD socket is nothing to laugh at, a fully user-space UDP/TCP network stack combined with memory-mapped buffer sharing usually gives performance improvement measured in the "x" and not in the "%".

Now, there is not much in terms of experiment protocol in their material so its very hard to tell. You mention uncompressed video so they could be sending humongous frames, which I doubt as no one ever would send raw frames over the network (that would be several MB per frame for a 720p front camera). But if that is the case then the data copy operation becomes the predominant bottleneck and getting rid of one may justify the improvement. But that is not a realistic scenario.

The more realistic scenario is that they sent compressed delta-frames across the network (H264 or HEIF), which would then considerably reduce the transferred payload size. In that scenario, data copy is not the predominant overhead anymore and 30% overhead reduction is underwhelming, telling me that they still are calling expensive operations like syscalls/uIPC on the critical data path.

I doubt they would have explicitly put TCP if it was not true

I am not doubting they have some sort of user-space TCP implementation. I am trying to understand how much of TCP they moved out of the kernel. From what I can gather they still have enough of it in the kernel such that the related syscall/uIPC overhead does not allow more than the 30% performance improvement announced.

Re: your clipboard issues, which platform are you running on? I am using vim+iTerm2 on OSX and vim+st on OpenBSD and the clipboard integration works flawlessly.

You should probably substantiate such a bold claim with a few comparative examples. I have yet to encounter a situation where vim doesn't "just work".

This implementation looks a lot like HW TCP offload engines where the kernel handles session creation and termination and the HW takes care of most of the state machine. Apple then must have found some way to hide the handling of ancillary tasks from the user in a way that does not cripple the protocol. They may have added hooks in the main application loop but my guess is that they are running separate threads to avoid having the application hanging the main thread in a non-returning loop and prevent these ancillary tasks from running (some are time sensitive). This means that their TCP user-space session management is most likely multi-threaded, which has a detrimental impact on performance due to the use of locks and the consequential cache pollution.

It is limited to UDP for the moment (at least AFAICT). I would be very surprised if they ever offer user-space TCP. It's a much different beast than UDP to handle right in user-space.

Also, the 30% less overhead is underwhelming. I would have expected much better improvement than that. That being said, if their measurement includes the encryption layer then the I/O benefit may be overshadowed by the encryption overhead.

Still, pretty cool stuff to make publicly available.

I will certainly gets dozens of downvotes for obvious fanboyism, but I'm prepared to take the heat.

For the doubters and the disbelievers that have been wondering what is the relevance of IBM in this day an age: this. This is what IBM is all about.

And it's not just about PFLOPS; each node has 1/2 terabyte of memory, globally addressable across the entire cluster using RDMA over Mellannox 200Gb/s EDR.

It's also P9: 44 cores per node; but most importantly each node drives a couple of V100 through NVlinks, which allows the GPU to share the system's main memory.

That could not be more true. Mai 68 is now considered in some thought circles to be the first “color revolution”. The main figureheads come from French upper middleclass bourgeoisie (Cohn-bendit, Sauvageot, Geismar, Goupil) and derailed legitimate blue collar upsets to push their liberal/libertarian agenda in order to shake up the conservative government led by General de Gaulle.

At that very moment, losing ground, de Gaulle flees France for Baden-Baden to regroup with General Massu. After that meeting de Gaulle decides to resign, leaving the room to the banker Pompidou, much more open to the Anglo-saxon “weltanshauung” that the General spent his life fighting.

Like many others said it highly depends on where you work and for whom you work. Speaking from experience in NYC greater area, you can expect starting slighlty above $100K as a junior, which is really not much for the location. Seniors often go between $200 and $250K. The finance industry tops these with 50% to 100% bonuses. Large attractive companies like GOOG or FB top these with other form of bonuses. Also speaking from experience I would equate a $100K salary in greater NYC with a €25K salary in a average town like Lyon or Grenoble in France.

The absolute number is really meaningless. I remember a Ph.D prep talk about salary expectation in grad school in France mentioning $105K in the US, and I remember thinking “wow!”. What they don’t tell you is that a third of this will go to taxes, half of the reminder will go to housing, and because of the outrageous deductible and out-of-pocket caps of most corporate medical insurances you can be hit anytime with a $2K or $3K medical bill, which can be quite common if you have kids.

In any job you must be given the opportunity to grow. The best employers will give you that chance over multiple dimensions: knowledge, wealth, reknown, responsibilities. Most employers will give you only one dimension. As long as you keep growing along that dimension all is well. But that single dimension gives a lot of leverage to your employer. He can use it to coerce you and force you back into line. When that happens, it’s time to go.