jcbeard

Couldn't find too many simple alternatives, so, wrote my own while on vacation for fun. Hopefully it's useful as open source, will fix-up CI over weekend and add a lock-free (not atomic free) allocator while I'm at it.

yes...yes they would.

Depends. Right now they're in two different spaces. One still hanging on to graphics/gaming, the other coming from dense compute space. We'll see how long it takes them to converge.

Better programmable gather/scatter (like described here: https://www.nextplatform.com/2017/09/14/shedding-light-dark-...) can definitely open up a wider range of applications to vectorization.

It'll be called SPiDRE (pronounced spider) at MEMSYS. Name coined by a colleague, I can't take credit.

Thanks for reading! You'll have to wait for the presentation and follow-on papers for some of those answers :). If you read the Dark Bandwidth paper, there are some solutions mentioned there and in the presentation (http://www.jonathanbeard.io/slides/beard_hcpm2017.pdf) that could apply to what you suggest.

This might be interesting for you then: http://www.jonathanbeard.io/pdf/b17a.pdf

Seems very much "back to the future." Systolic array processors were used to accelerate neural networks in the 1980's. Great for matrix math too. (ref: http://repository.cmu.edu/cgi/viewcontent.cgi?article=2939&c...). These aren't quite the systolic array processor of old, but too close to be considered new arch/micro-arch. The formula is simple, have low precision MM to accelerate, drop in a matrix multiply unit that can be blocked for and high bandwidth memory to feed it and let it go. I'm waiting for more new takes on old arch....as fabbing chips becomes more economical, I hope to see more retro chips. Especially things that didn't quite make the jump from research to production b/c of scaling (or other reason), might now make sense.

At ISC17 they had these at the Gigabyte booth....with multi-GPU configs (see GPGPU on top).

https://drive.google.com/file/d/0B4tfh1u3NMxnTThSdEswR2o1SXc...

I can confirm...I didn't make a dime :( just reporting on on the goingarm.com workshop.

If you're interested in seeing more about what people are currently doing with ARM, check out the slides at http://www.goingarm.com/#2017schedule

For buying a machine, it depends on what you want. Desktops are coming out now, for example: https://softiron.com/products/overdrive-1000/

Server/HPC, there are quite a few coming online. The Cavium presenter @goingarm listed (https://www.packet.net) on their slides, might be decent for getting a cloud instance to try out. There are many others. A good place to start finding them is: http://arm-hpc.gitlab.io.

A nice thing about ARM is that you get lots of different micro-arch under a single ISA. Once the ball gets rolling there will be many processor choices to choose from, which I think is a great thing (however, in full disclosure I'm one of the authors, and I work for ARM Research for our HPC program..although all statements above are my own opinion and not those of ARM).

Officially I think the only thing that can be said, has been said by Paul Messina: "I believe that the Aurora system contract is being reviewed for potential changes that would result in a subsequent system in a different time frame from the original Aurora system. But since that’s just early negotiations, I don’t think we can be anymore specific on that.” source: https://insidehpc.com/2017/06/told-aurora-morphing-novel-arc...

C++Now is branching out. I gave a talk this year on FIFO communications and a tutorial on RaftLib. One not so C++ focused and the other definitely C++ focused (C++ library). I enjoyed the wide variety of people and topics. Will try to go again next year.

In general, with current systems, it's all a partitioning problem. Speed-up is a trade-off between granularity of the compute kernel placed on a resource, the communication between said resources, latency at each level of the memory hierarchy (cache line utilization and reuse have a direct impact on this), and also (eek) protocol overheads between all the abstraction layers. Current systems are largely built around single CPU abstractions pasted on to make them multi-CPU abstractions. Working with accelerators, RDMA, etc. is quite a challenge both for the programmer manually making things work and for systems developers beating the abstractions into submission.

Many companies in the US are providing gyms for their employees, making staying in decent shape that much easier. I definitely take advantage. Even when I don't have time for a full workout, I can hop on treadmill for a 30 minute run and be back to work in 40 minutes (assuming showers aren't all occupied).

You might want to check out "A Systolic Array Optimizing Compiler" -> https://books.google.com/books?id=6nTjBwAAQBAJ&lpg=PA188&ots...

Systolic array/vector processors have been used since the late 60's for neural and other gemm heavy workloads. The Warp architecture from CMU is an excellent example.

So...to answer directly.

TBB was in alpha at one point. So was OpenMP, MPI, ...

Just because you have an established library doesn't mean something else can't come along. Said another way, just because new doesn't mean to dismiss it. <rant> Do you have a quill pen? Hmm, didn't think so. If somebody came along and said: "Hey, look they both write, why use that fountain pen." Then we'd likely be writing with feathers today. </rant>

thanks!, and meh.

Yup, it was quite a bit better on the upper end especially. Looking at the snoops on the bus using PAPI RaftLib does a better job at keeping the cache lines from bouncing.

The benchmarked version also has a dynamically resizing FIFO which uses utilization of the queue itself to guide the sizing. This means that the FIFO can better adapt to dynamic behavior found in most applications run on top of an operating system (most all these days outside of HPC). Looking at load stalls, the RaftLib version has fewer, but not quite enough to account for the results.

If you look at the single worker thread case, then jump to two threads..you can see a fairly big jump. RaftLib by definition is a pipelined programming system. The read file and compress are done perfectly in parallel. The bzip2 code doesn't quite pull it off in a perfectly pipelined fashion. It's close, but not quite. This results in less overlap of execution and communication. If I'd run on Linux (thread affinity on OS X is well, fun last time I checked..if not impossible to do manually), I'd also add thread affinity to the list which most people don't bother to optimize. Hot caches and synergistic cache accesses are quite beneficial.

Actually, the reason I didn't show distributed was in fact the naive original implementation of the TCP stack support. The platform minimizes data movement over network pipes, however due to time constraints on me finishing a PhD I chose to get the more interesting mathematical modeling aspects worked out rather than focus on engineering something that is pretty well understood (especially using techniques directly developed for HPC with MPI). Will push better support soon.

On the multi-process..yes, that's easy. I've removed it for the current main-line branch given the lack of demand. IfDef'ing the code made it much easier to proceed with getting it ready for alpha. The FIFO mechanisms are well tested using SHM and the forking code will be added back in soon. Another thing I commented out to get it working on multiple platforms is the NUMA placement code, now that I think hwloc will work on all platforms I'll get it added back in. Helps out on cross-socket communication quite a bit, as well as placing buffers closest to PCIe root for data transfer to accelerators.

In reality the data movement is no worse than any OpenMP or other parallel program. In as many places as we can, the data is left in place vs. pushed. Between nodes, it gets more fun...however it's still a rather well understood problem. Thanks again for the interest! I'll see if I can do a ShowHN before CPPNow 2017 for the beta release.

Hi! I'm the primary author and maintainer of the library. Thanks for the interjection. The intended application for RaftLib is to make something that will scale, just as you mention. I wrote this post a long time ago when I was just trying to get people interested in using it. It's a simple example that shows you can take many lines of standard parallel code, and write a much easier to read (smaller) version in a very short time that performs just as well or better than the manually managed parallel code.

A long time ago I was a biologist, then bioinformaticist. I wrote some code that would scale to a single node, and to a few dozen cores. In doing so, I realized how much I hated writing the same boilerplate code over and over again. TBB, c++11 threads, OpenMP, and MPI all basically have the same level of boilerplate and gotchas. I wanted to make something that was relatively easy to use and easily integrable with C/C++ code. Go was the only thing that came close, but it was brand new at the time.

It occurred to me while working on the AutoPipe system as a grad student that I could do something even better than a simple coordination language and at the same time subsume the functionality of a lot of parallel libraries. With stream/data-flow processing, I can do the exact same things I can do with OpenMP and MPI, but I can do more. The state encapsulation allows a whole host of cool optimizations, like identifying bottlenecks and duplicating actors dynamically (there's a whole host of reasons we'd be limited in OpenMP, c++11 threads). You can also compile an encapsulated function to another hardware platform entirely, or use high level synthesis tools to go to an FPGA (I'll be going there again soon too with RaftLib). The only thing that has to be constant across optimizations, is the connectivity of the DAG. By maintaining a port interface, just like you would hardware components (see Arvind's work from MIT...he's famous enough I just have to say Arvind :), we can compose really complicated applications. The port interface, it turns out, is also perfect for distributed compute.

Awhile back, I also had the realization that iostreams were perfect for this paradigm. Once you get your head around the concept, it seems quite natural. If it doesn't take off as a library, oh well. I enjoy working on it, and using it so I'll likely keep developing it in my spare time.

In the interim, I'll get back to exascale hardware stuffs :).

Why would anyone want to remove public access to impartial facts? Hmm.

I'm not sure that is quite correct ( http://www.businessinsider.com/the-states-the-most-and-least... , http://blogs.wsj.com/economics/2014/03/27/which-states-take-... ) are just a few of many references that show where the tax dollars are distributed in the United States. Here, states that tend to be more fortunate/productive subsidize the other ones (note: most of the charts specify return on their own tax dollars, and are likely sourced from same data). This has come up in several recent elections, blogs, etc. Not to the point of the EU debates, however it is definitely a source of animosity to some. I don't have data to back up why each state is the way it is (a very complex web to unravel). We can speculate all we want (I'd rather not since political ideology invariably trumps factual discourse, pun intended)...however, if somebody has a definitive answer, they should definitely write a paper :).

Same basic trend we saw with industrial revolution. People lost their somewhat skilled (for the time) jobs to steam powered machines. As the recent election also put front and center, we're also competing with the entire planet for jobs. How do we compete with the Terminator and the foreigner bogeyman? You don't directly. Nations build a social safety net that cushions the churn of jobs, universal healthcare so that the stress of loosing your job and healthcare doesn't kill productivity, cheaper (or free) access to entry level college education (community college), and lastly nations need to provide geographic mobility (getting from the place without jobs to one with jobs isn't easy when your bank account balance is -$40 USD). The net effect of all these suggestions would likely be a good bump in long term national GDP and growth...we could likely implement most of them using less than 1/4 of our $600 billion yearly defense budget.

slightly less editorialized from Top500 news: https://www.top500.org/news/arm-buys-hpc-software-toolmaker-...

Cool idea, but any idea on the quality of feedback?

TL;DR: Best way.... (other than completely disable icloud calendar...) make new temp calendar, move invite(s) to temp calendar, delete temp calendar. Then change settings to send to email first vs. straight to icloud cal (see article).

Hopefully Apple will get on making some actual solution...this is a horrible hole in icloud's calendar setup.

ooh, ok. you might also check out openshmem (http://openshmem.org/site/) and openucx (http://www.openucx.org). I'd been planning on integrating both in RaftLib just not enough cycles to get it done yet. These combined would make it much easier to maintain a relatively portable yet performant back end. My thesis research was all about locality, memory placement, and throughput for big data systems. Current work is similar but I'm not neck deep in the hardware dev world. There are current research efforts on my part outside of work, most center around the raftlib.io platform. Before I forget, you might also want to check out the graph partitioning frameworks like metis and scotch...both are used in some MPI frameworks for more optimal partitioning. To get topology data you might want to look at the hwloc framework, it's cross platform and provides input for things like NUMA/cache/PCIe topology for optimization. I haven't had a chance to integrate this hook into RaftLib, however it's just a few lines away once I find the time. If you're wondering...I started out writing my own fiber library for RaftLib. Had ports for both IBM Power and Intel, but it gets a bit tiring maintaining/optimizing for every new architecture. Qthreads and the like have been used in HPC circles for quite awhile, so it made sense. There was no way I was beating them for dev time, so might as well join them.

Based on your auto-tuning discussion...RaftLib aims to do something similar, but for big-data stream processing workloads. Here's my 2016 IJHPCA paper: http://hpc.sagepub.com/content/early/2016/10/18/109434201667...

It looks like it's behind a paywall so if you don't have access I'll update my website with the "author archive" copy sometime today...will be at ( http://jonathanbeard.io/media ) once I update it. Bottom line if there's intersected interest, definitely open to collaboration :).

How would something like this differ from something like Sandia National Lab's Qthreads (http://www.cs.sandia.gov/qthreads/)? Seems it's a tried and true solution in C that also works with C++11 (committed a test case for C++11 myself)...It is also an optional underpinning for some relatively big-name frameworks like Kokkos, Chapel, RaftLib, etc.

From the article: He said that in the aviation community every pilot in the world learns from investigations and that the results are disseminated widely.

“When a plane crashes, we don’t say this is confidential proprietary information the airline company owns. We consider this part of public safety. Hospitals should be held to the same standards,” Makary said.

Seems like this number would be higher if they also included HAI's (healthcare associated infections, about 75K additional deaths): https://www.cdc.gov/hai/surveillance/

And it's right, the rates of what are essentially malpractice haven't changed. Arguably despite spending more, we get worse care: http://www.npr.org/sections/health-shots/2013/09/20/22450765...

If infections are included in the numbers then the US is only slightly above average: http://www.euro.who.int/en/health-topics/Health-systems/pati...

The WHO suggests by reducing HAI, the EU could reduce deaths by about 95K/year vs. US's 75K (actually normalizing by population makes the US's number look really bad..assuming stats apply to entire EU's pop of 508-ish million and the US's ~308 million).

I can't find in the cited study (original article) if they included hospital acquired infections like the WHO does. If they didn't then we've a lot of work to do. Likely will never be an issue though, unless we treat the med profession like any other professional service provider with the power of life and death. We need to open the books on investigations, share data publicly, and likely provide more oversight. Also...seems like a wonderful area where big data and ML could contribute to meaningful improvements...

And many others, because it presents them with an economic opportunity they would never otherwise have had: college, vocational training, and increased social status.

You should show empathy, not sympathy for Soldiers. We don't want your freaking sympathy(from former Soldier), most just want opportunity. If you want to stop people from signing up for the US Army then here's what you have to do: 1) End inequality in America 2) End the spiraling healthcare costs (adding a single payer would likely go a long way) 3) Provide a means for formal education and social support for everyone. It's not enough to teach people, there's life skills a vast swath of America misses out on. 4) Fix the credit rating system. It amazes me that this is even allowed. Security check, sure...credit? Give me a break, just another form of discrimination.

Do all those things and we can start to level the playing field. Until then we'll likely need public jobs programs (and the military is indeed such a program, even if it is also something much more) to throw folks a rope to climb towards the American dream.