jhokanson

"Firstly, there are three main Python concurrency APIs, they are:" asyncio, threading, multiprocessing, ... oh, and concurrent.futures

All kidding aside, I used the multiprocessing module lately and it was a mess. Do I want 'map', 'starmap', 'imap', etc.? All I wanted was to run a function multiple times with different inputs (and multiple inputs per function call) and to fail when any launched process failed rather than waiting for every input variation to execute and then telling me about the error(which honestly I didn't think was asking for too much).

Spreadsheets.com, for example, lets users dump almost anything into a cell. Drop a photo or a PDF into a cell and the product will immediately create a thumbnail, which you can then expand, as if the spreadsheet were some sort of blog content-management system.

Yes please! Can I get this for Google Sheets? :/

Speaking of math pages on Wikipedia ... and math text more generally

Is it just me or are we horrible at teaching advanced math? Where are the examples (with actual numbers)? Where is the motivation? Where are the pictures?

Does AMD have folks that you can reach out to regarding this? I know Intel has MKL and all the work around its own compiler for maximum speed. This seems like it should be trivial for someone at AMD to put together as an example of how to do things like this correctly ...

My c++ is not great (so it is hard for me to tell what is going on) and I'm used to OpenMP where my understanding has always been that you tend to get a single thread per processor (or per hyper-thread) -- not sure if that is guaranteed with the way your code is laid out? Perhaps it really is a NUMA issue as others suggest. I will note that one other variation I had (as it looks like you are already splitting across threads) is that the chunk sizes were actually smaller than the # of threads which meant a faster thread would take more chunks rather than waiting on the slowest thread. Good luck!

It is not exactly clear to me what is going on with threads (I guess you are using all of them?). I haven't done too much in this space but anecdotally I've had better luck if my summation is explicitly split into sub-summation tasks. It is not clear if that is being done here. It looks like a single summation loop that the author is expecting the computer to magically split across multiple threads. I'd be interested in seeing what this looks like if instead the task were to add chunks of the original dataset into results per thread (e.g, first 8000 samples on first thread, next 8000 on 2nd thread, etc.), with a final accumulation loop across all threads. Again, the author may be trying this and this is not my area of expertise but I've had decent luck saturating the memory bus with a similar approach.

Yes please! I like to search for examples of how to use libraries and often times the results are all the same exact call in forks or copies of the same code in multiple places. Perhaps deduplication could be optional when searching?

I thought this would be referencing work by Nathan Myhrvold using a new type of reactor that supposedly runs on "spent" nuclear waste and in the event of power failure just stops running safely. Not sure of the other logistic issues involved. The one thing I remember about transitioning to this approach is that Nathan said the US isn't very good about building new things, so they were going to build in China. But then it got shut down right as the anti-China trade policies started a few years ago. Not sure if there are big problems with this approach, but it sounded promising ...

As someone that rarely works with Python lists, as opposed to numpy arrays, I was pleasantly surprised to see numpy does what I would expect in providing a mergesort option. I'm surprised Python doesn't, other than via heapq and only implemented in Python (according to my reading of the post and a very quick Google search).

Oops, just for fun the numpy documentation currently states: "The datatype determines which of ‘mergesort’ or ‘timsort’ is actually used, even if ‘mergesort’ is specified. User selection at a finer scale is not currently available." Awesome ...

Also, apparently mergesort may also be done using radix sort for integers "‘mergesort’ and ‘stable’ are mapped to radix sort for integer data types."

Interesting take. I did not interpret their statement as vindication of being correct but rather that this version of events is a possibility that can't currently be dismissed.

Agreed. However my reaction when first hearing about the lab leak (middle of last year?) was that the leak stories were meant to be malicious/propaganda against China. I didn't take any of this seriously until an article in Politico a week or two ago.

But here's the kicker. Let's say this was a lab leak and as a reporter (which I'm not) I thought the evidence was good enough to warrant reporting. I'm not sure I would share it. The previous occupant of the white house did a great disservice in giving this whole thing a racially charged tone. I'm genuinely scared by the increased acts of violence against southeast Asians in the US and worry that stories like this will make it worse. I'm hoping that the new US government is secretly taking steps to help prevent what may have happened in that lab -- in addition to the large effort needed elsewhere to improve our handling after things had begun to spread.

Anyway, main point is that this was the first time in a long time (ever?) where I really wondered whether, given the circumstances, if it was good to share "the whole truth" (as best we know it) given that we don't know what happened and the potential real-life implications to many people in the US.

Maybe somewhat off topic ... I don't sell anything on any App stores currently but I do like the idea of having some way of creating "applications" that are a bit more discoverable. Last I checked GitHub search is pretty awful for discovery (maybe I'm wrong on that?). I could imagine an interface where it would be easier to browse "apps" that do something specific.

Mathworks (MATLAB) has a decent version of this: https://www.mathworks.com/matlabcentral/fileexchange/

People can provide feedback and ratings which makes it easy to see when projects are dead or when there are big issues that aren't being fix. You can also reference other projects - "This project is like that project but it is faster ..."

I don't think PyPi is all that good for this. Also, I've never really invested the effort to learn how to deploy an application to PyPi whereas something that simply points to my repo may be valuable?

Anyway, the main problem I want solved is not having people discover my code (although that would be great too), but being able to find other code more easily so I don't need to constantly reinvent the wheel.

Wow, those are big performance differences (660 MB/s for fast-double vs 1042 MB/s for the 'newer' fast-float), although most of the numbers (for the different libraries being tested) are all over the place, and even 'strtod' more than doubled in speed between the two tests (70 MB/s fast-double vs 190 fast-float MB/s). It wouldn't surprise me if those two code bases are essentially the same.

That highlights the complexity of benchmarking in general and the importance of comparing within the same benchmark. I haven't looked at this in a while but I thought some of the newer JSON parsers were standards compliant (maybe not?).

Anyway, that other blog post answers my question as it looks like the big insight is that you use the fast approach (that everyone uses) when you can, and fall back to slow if you really have to. From that blog link:

"The full idea requires a whole blog post to explain, but the gist of it is that we can attempt to compute the answer, optimistically using a fast algorithm, and fall back on something else (like the standard library) as needed. It turns out that for the kind of numbers we find in JSON documents, we can parse 99% of them using a simple approach. All we have to do is correctly detect the error cases and bail out."

Again, I swear I've seen this in one of the other JSON parsers but maybe I'm misremembering. And again, good for them for breaking it out into a header library for others to use.

I'm curious as to what the biggest win in terms of speed was here (in terms of an approach, good lookup tables?). Also I'm curious how this compares to the many (?) JSON parsers that have rolled their own number parser because everyone knows the standard library is so slow ... (just more accurate?, faster?). Regardless, kudos to the authors on their work!

I thought my Garmin watch already did this and I thought Apple had differentiated itself with some sort of advanced ekg analysis. I feel like I'm missing something with this article? Is this truly different than the competition or just Apple catching up?

That was a great story about actually managing to debug a wireless problem. I wish there were better diagnostic tools commonly available that would clearly explain these things. I'm not sure if in the author's case there is some software that would listen for extensive polling and flag that as an issue - even better if it could log running programs at the time and try and guess which one is causing the issue.

I'm constantly having problems with my mac laptops and after many hours (10+) of internet searching I still have no idea why the wifi doesn't always work reliably. Some days or weeks (months?) it is great. Other days every hour it is disconnecting. Sometimes resetting the router helps, sometimes it doesn't. The whole situation is extremely frustrating.

Some issues: Why am I getting a DNS issue when my wired desktop never has a wireless issue?

Sometime I swear that the laptops becomes much less reliable at the opposite end of my house as my router but the signal strength is generally still excellent (4 bars).

Anyway, I'm not asking to have my particular issues solved (although that would be much appreciated!!!!). The real issue just seems to be that debugging these issues is extremely difficult and not based on principles but just random things that people can try (e.g., delete your plist files).

My introduction to Python (a long time ago) and "batteries included" was going to some random guys website to download all the compiled packages I needed. Thanks Christoph!

Besides focusing on language 1 vs language 2 I think the author is also highlighting just how hard it is for people to get started in any language. You mentioned R/Rstudio ....

I just wanted to work through an example in a book that I'm reading that has corresponding R code. I type require(package_name) into the command window and that doesn't work. I look at the help for "require" and there is nothing about how to install packages - I would think a reference to the install packages command would be useful (they link to how to check, but nothing obvious on how to install, maybe I missed it). After some searching I find the command and it prompts about whether I want to use the source code or not (after a few reads I understood what was going on but it could be improved). Things started compiling and then I tried to run my package again, no luck. So then I try reinstalling and this time I noticed the error - something like "exited with non 0 status". Awesome. Looking a bit closer I noticed one of the dependencies was not installing. After trying to install the dependency manually I somehow realized it was only for R version > 3.6, I'm on 3.5. So I figured updating RStudio would fix the problem, no luck. It's not like RStudio advertises what version of R they are shipping with Rstudio .... Now I'm on someone's Linkedin post (wtf?) looking at how to upgrade R from within RStudio. They indicate that on my mac I should updateR (or wait, they say that turns out to be not good, so just install from CRAN). Hopefully that works. Do I try my luck with R v4? Nothing's going to break there right ...?

This isn't meant to be a "R is bad" rant. Maybe C# or some other language really does avoid these problems, I don't know. However my experience has been the problems the author mentions are a problem everywhere.

Thanks i

Apparently this has nothing to do with the cable company ...?

The function bit has always confused me. I tend to write code with lots of functions/modules/classes for handling various aspects of the analysis and I just don't understand how that's supposed to be integrated. Instead notebooks seem better designed to handle small code snippets that rely on well known libraries. I'd be happy to jump on the notebook bandwagon but I'm having trouble seeing how I could adapt my code to the notebook style.

Yikes! Good to know. I just wonder if there has been any official mea culpa announcement or if I (along with many other people) are just suffering from sort of selection bias. Did the screen material change at all?

Has the iPhone screen durability improved at all? It seems like they are (or were?) extremely susceptible to damage from dropping and I always took that as a sign that Apple wanted to force people to upgrade.

Edit: Here's a link discussing the issue: https://www.fastcompany.com/3063238/why-wont-apple-fix-the-i...

What about learning? Would an update with more data and updated coefficients require going back to the FDA?

I can't personally speak to the accuracy of the linked claim below, but it suggests that the coverage of residency spots is controlled by a physician led advisory board called RUC that intentionally limits the reimbursement so as to control wages.

http://www.motherjones.com/kevin-drum/2013/11/its-doctors-wh...

The parent isn't talking about reducing quality, but rather waiting to pick "winners and losers" so to speak until after training.

Does anyone know what the tuition is for the Abbott preschool program that is mentioned in the article? My impression has always been that one of the reasons that teachers get paid so little is that class sizes are relatively small (e.g. 4 infants per teacher in NC). I personally think preschool teachers are not paid enough but I'm curious how Abbott is able to pay their teachers so much.

But isn't it possible to design multiple models, where the judgement of the "best model" is dependent on your goals (e.g. reduction of false positives vs false negatives)?