JangoSteve

As a software developer who spent nearly 10 years founding and building a genomic startup, this is a good start, but does have a lot of vast oversimplifications and a few inaccuracies. The people making this know what they're doing, so I'm sure these are known shortcomings they likely deemed necessary for a quick introduction. You'd need a further study at the end to start being able to do some real-world work.

Anecdotally, I've noticed an uptick in my eBay feeds recently of items being immediately relisted after having supposedly sold. This has always happened occasionally, but within the past couple weeks, I've noticed it happening for like 25% of my watched items. I'm wondering if bots are buying things for which the owners are then canceling when it comes time to pay.

I've also had to return a few items for which eBay's AI-generated description was wrong in ways that couldn't be verified in the product's images. I can only imagine the increase in canceled/returned orders from all the different AI features and bots.

This was a common approach called a "honeypot". As I recall, bots eventually overcame this approach by evaluating visibility of elements and only filling out visible elements. We then started ensuring the element was technically visible (i.e. not `display: none` or `visibility: hidden`) and instead absolutely positioning elements to be off screen. Then the bots started evaluating for that as well. They also got better at reading the text for each input.

Interestingly, the award was specifically for the impact of AlphaFold2 that won CASP 14 in 2020 using their EvoFormer architecture evolved from the Transformer, and not for AlphaFold that won CASP 13 in 2018 with a collection of ML models each separately trained, and which despite winning, performed at a much lower level than AlphaFold2 would perform two years later.

I've gotten so much push-back over the years when I've asserted that startups are too quick to describe their products as AI in industries where the target customer considers it undesirable. I think it's done for ego and to impress investors. Examples I've seen include products in clinical diagnosis and financial accounting. Some needs require utmost predictability, observability, and ultimately, understandability.

Of course, there are some industries and markets that desire the capabilities only AI can provide. But that's the point. Analysis should precede the message. We should market the benefits. I've seen a few people at least claim AI isn't a benefit, it's a feature. I'd argue it's not even a feature, it's an implementation detail, like using an object-oriented programming language or a relational database; it has advantages and disadvantages.

Focus on the needs of the customer and industry. Describe the benefits. For customers and investors alike, remove the AI veil of opacity, by describing simply what the AI is doing and how/why.

It's interesting to see a study that seems to corroborate my anecdotal experiences. It's a marketing study though, so it shouldn't be overly generalized until more studies reproduce the results. Studies about human behavior tend to be difficult to reproduce and can yield conflicting conclusions. I wouldn't be surprised to see another study with slightly different questions or methods come to the opposite conclusion, especially if they don't control for consumer segments, industries, or types of products.

It's a little worse than that. If you need to upgrade at least once per billing cycle, the system keeps charging you more and more (since an upgrade will always have a higher total cost than the previous upgrade) each cycle while accruing credits that will never start paying down unless/until you have a billing cycle without an upgrade. That's not a great design for most systems, but especially bad for a scalable image service intended to facilitate the flexibility of growth.

What is reasoning if not the ability to assess "if this" and conclude "then that"?

A lot of things. There are entire fields of study which seek to define reasoning, breaking it down into areas that include logic and inference, problem solving, creative thinking, etc.

If you can do it with logic gates, who's to say you can't do it with transformers or one of the newer SSMs? And who's to say it can't be learned from data?

I'm not saying you can't do it with transformers. But what's the basis of the belief that it can be done with a single transformer model, and one trained on language specifically?

More specifically, the papers I've read so far that investigate the reasoning capabilities of neural network models (not just LLMs) seem to indicate that they're capable of emergent reasoning about the rules governing their input data. For example, being able to reverse-engineer equations (and not just approximations of them) from input/output pairs. Extending these studies would indicate that large language models are able to emergently learn the rules governing language, not necessarily much beyond that.

It makes me think of two anecdotes:

1. How many times have you heard someone say, "I'm a visual learner"? They've figured out for themselves that language isn't necessarily the best way for them to learn concepts to inform their reasoning. Indeed there are many concepts for which language is entirely inefficient, if not insufficient, to convey. The world's shortest published research paper is proof of this: https://paperpile.com/blog/shortest-papers/.

2. When I studied in school, I noticed that for many subjects and tests, sufficient rote memorization became indistinguishable from actual understanding. Conversely, better understanding of underlying principles often reduced the need for rote memorization. Taken to the extreme, there are many domains for which sufficient memorization makes actual understanding and reasoning unnecessary.

Perhaps the debate on whether LLMs can reason is a red herring, given that their ability to memorize surpasses any human by many orders of magnitude. Perhaps this is why they seem able to reason, especially given that our only indication so far is the language they output. The most useful use-cases are typically those which are used to trigger our own reasoning more efficiently, rather than relying on theirs (which may not exist).

I think the impressiveness of their capabilities is precisely what makes exaggeration unnecessary.

Saying LLMs develop emergent logic and reasoning, I think, is a stretch. Saying it's "within the realm of possibility that some limited form of reasoning might actually be part of the emergent behavior" sounds more realistic to me, though rightly less sensational.

EDIT:

I also think it's fair to say that the ELIZA program had the limited amount of reason that was programmed into it. However, the point of the ELIZA study was that it shows people's tendency to overestimate the amount of reasoning happening, based on their own inferences. This is significant, because this causes us to overestimate the generalizability of the program, which can lead to unintended consequences when reliance increases.

And it turns out that a lot of human reasoning is statistically predictable enough in writing that you can actually obtain reasoning-like behavior just by having a good auto-complete model.

I would disagree with this on a technicality that changes the conclusion. It's not that human reasoning is statistically predictable (though it may be), it's that all of the writing that has ever described human reasoning on an unimaginable number of topics is statistically summarizable, and therefore having a good auto-complete model does a good job of describing human reasoning that has been previously described at least combinatorially across various sources.

We don't have direct access to anyone else's reasoning. We infer their reasoning by seeing/hearing it described, then we fill in the blanks with our own reasoning-to-description experiences. When we see a model that's great at mimicking descriptions of reasoning, it triggers the same inferences, and we conclude similar reasoning must be going on under the hood. It's like the ELIZA Effect on steroids.

It might be the case that neural networks could theoretically, eventually reproduce the same kind of thinking we experience. But I think it's highly unlikely it'd be a single neural network trained on language, especially given the myriad studies showing the logic and reasoning capabilities of humans that are distinct from language. It'd probably be a large number of separate models trained on different domains that come together. At that point though, there are several domains that would be much more efficiently represented with something other than a neural network model, such as the modeling of physics and mathematics with equations (just because we're able to learn them with neurons in our brains doesn't mean that's the most efficient way to learn or remember them).

While a "sufficiently huge autocomplete model" is impressive and can do many things related to language, I think it's inaccurate to claim they develop reasoning capabilities. I think of transformer-based neural networks as giant compression algorithms. They're super lossy compression algorithms with super high compression ratios, which allows them to take in more information than any other models we've developed. They work well, because they have the unique ability to determine the least relevant information to lose. The auto-complete part is then using the compressed information in the form of the trained model to decompress prompts with astounding capability. We do similar things in our brains, but again, it's not entirely tied to language; that's just one of many tools we use.

Do you mean for AI to do the entire job of researching, creating, testing, manufacturing, and distributing a cure, or just for AI to be involved? And do you mean completely eradicating a disease, or just producing a cure for it? And do you mean an outright cure, or also a treatment or vaccine? If the latter in all cases, here's an example:

https://www.technologyreview.com/2022/08/26/1058743/i-was-th...

If you mean the former in any case, it'll probably be a while, if ever in our lifetimes.

First, I agree it's important to put Tesla recalls in context with the greater automotive industry, so I think what you posted is great info. The only thing I'd disagree with is that the numbers compared to Tesla, and relative media coverage, is surprising. It seems expected to me.

Tesla bills itself not as an automaker, but a tech company. So, it makes sense they'd have a larger media footprint, which covers not just the automotive industry, but the tech industry media as well. This isn't unfair, considering they get the benefit of a tech-based market cap to go with it [1].

They also put themselves in headlines more often than other car companies with outlandish claims such as Musk saying, "At this point, I think I know more about manufacturing than anyone currently alive on earth." [2] When Musk and the company put themselves in headlines so often, it makes sense that the media would cover them more. This is likely a direct result of their advertising strategy, to create buzz [3], so I think media coverage of failures is a direct result.

You could argue that's Musk, not the company, but they made the strategic decision that Musk is their PR function when they became the only car company to dissolve their PR department in 2020 [4].

One last thing I noticed was that the source of the recall data comes from the NHTSA [5], and they don't seem to distinguish recalls between different brands owned by the same company (for example, Ford's recalls seem like they would include both Ford and Lincoln, GM includes Chevrolet, GMC, etc.) Tesla's 20 recalls in 2022 cover I believe the four models they made in 2022, while Ford's 67 recalls are across the 39 models under the Ford brand and five models under Lincoln (I counted these by looking at the drop-down selectors on KBB's value estimator [6]).

In short, Tesla exploits the hype machine; is it surprising that their recalls are hyped as well?

[1] https://www.reuters.com/business/autos-transportation/tesla-...

[2] https://www.thestreet.com/markets/elon-musk-ted-talk

[3] https://fourweekmba.com/tesla-marketing-strategy/

[4] https://electrek.co/2020/10/06/tesla-dissolves-pr-department...

[5] https://datahub.transportation.gov/Automobiles/NHTSA-Recalls...

[6] https://www.kbb.com/car-prices/

I did something similar to this a while back with a one-liner aliased in my Bash includes, called gitsum (short for git summary).

    alias gitsum='git log --pretty=format:"* %s" --author `git config user.email`' #myself

    gitsum --since 1.day
    gitsum --since 1.week
    gitsum --since 8.hours

    gitsum --since 1.day | pbcopy

I think my favorite part of this prompt is that it starts with, "Please..."

With this new class of products based on crafting prompts that best exploit a GPT's algorithm and training data, are we going to start seeing pull requests that tweak individual parts or words of the prompt. I'm also curious how the test suite for projects like this would look for specific facts or phrases to be contained in the responses for specific inputs.

A lot of good feedback here. One additional thought I'd provide is that I wouldn't rule someone out based on a "Contact us for price" button. I'd click the button, let them know what you need, and then let them rule themselves out based on their responses. Sometimes they may give you a price directly in their response (or at least a ballpark), sometimes they may insist on a call. Take the call, but draw the line where you're comfortable (e.g. give me a ballpark after 30 minutes or I'm out - you can be more tactful in your wording).

Think about it from the company's perspective. Let's imagine as a company, you do an experiment where you have one landing page that shows the pricing, and another landing page that says "contact us". Let's also imagine that your product is enough of an enterprise solution that no one ever buys it without talking to someone at some point during the evaluation process (even if the pricing is up front, it requires enough of an investment that the customer wants to be absolutely certain it will satisfy their needs both now and as they grow). Finally, let's imagine that you have 3 full-time sales people to handle the incoming communications at whatever step of the evaluation process.

Now, if the outcome of this experiment is that the "contact us for pricing" results in 1/5th the incoming contacts, but those then convert twice the rate, you might choose the up-front pricing (2x conversion rate but on 20% of the leads means only 40% the sales compared to up-front pricing).

However, what if the 5x incoming contacts is too many for your sales team of 3 to respond, causing the up-front pricing to result in fewer sales with more work? Then you might choose the "contact us for pricing".

But what if it's so many more incoming contacts at a consistent enough conversion rate that you can justify adding a 4th and 5th sales person to realize those sales? Then you might choose the up-front pricing.

But what if the up-front pricing pigeon-holes you into your beachhead market and makes it more difficult to expand vertically or horizontally, which could lead to a trail-off in the incoming contacts and sales? Then you might choose the "contact us for pricing".

The point is, the one that makes the most sense for a company depends on a lot of variables, most of which would be opaque to an outside observer. Being able to tell the difference between a company that has "contact us for pricing" because it made sense for them, compared to one trying to exploit price discrimination (as described by some of the more cynical takes) is next to impossible without talking with them. Even in that scenario though, if they end up giving you a better product for a better price, then being willing to reach out to them could end up being a competitive advantage for your company over another which disqualified them on that basis.

Genomenon | Full-time | Remote (USA) | Senior Back-end and Full-stack Developers

Genomenon is an AI-driven genomics company. Our mission is to save and improve lives by making genomic information actionable. We organize the world’s genomic knowledge and put it at the fingertips of clinicians to help diagnose and treat patients suffering from rare diseases and cancer, and to help precision medicine companies create targeted therapies for diseases.

Genomenon was named 2020 Global Company of the Year in Clinical Genomics Interpretation by Frost & Sullivan.

We’re looking for Senior Full-stack (also Back-end) Software Developers who can take direction and run with it, building out new and existing products and features. We are especially looking for those with experience using Golang and ElasticSearch, as well as Python, Ruby (and Rails), and front-end JavaScript (Angular, React, or similar).

https://www.genomenon.com/blog/job/software-developer-2/

Genomenon | Full-time | Remote (USA) | Senior Back-end and Full-stack Developers

Genomenon is an AI-driven genomics company. Our mission is to save and improve lives by making genomic information actionable. We organize the world’s genomic knowledge and put it at the fingertips of clinicians to help diagnose and treat patients suffering from rare diseases and cancer, and to help precision medicine companies create targeted therapies for diseases.

Mastermind Genomic Landscapes inform pharmaceutical and bio-pharma companies on precision medicine development, deliver genomic biomarkers for clinical trial target selection, and support CDx regulatory submissions with empirical evidence.

Genomenon was named 2020 Global Company of the Year in Clinical Genomics Interpretation by Frost & Sullivan.

We’re looking for Senior Full-stack (also Back-end) Software Developers who can take direction and run with it, building out new and existing products and features. We are especially looking for those with experience using Golang and ElasticSearch, as well as Python, Ruby (and Rails), and front-end JavaScript (Angular, React, or similar).

https://www.genomenon.com/blog/job/software-developer-2/

Genomenon | Full-time | Remote (USA) | Senior Back-end and Full-stack Developers

Genomenon is an AI-driven genomics company. Our mission is to save and improve lives by making genomic information actionable. We organize the world’s genomic knowledge and put it at the fingertips of clinicians to help diagnose and treat patients suffering from rare diseases and cancer, and to help precision medicine companies create targeted therapies for diseases.

Mastermind Genomic Landscapes inform pharmaceutical and bio-pharma companies on precision medicine development, deliver genomic biomarkers for clinical trial target selection, and support CDx regulatory submissions with empirical evidence.

Genomenon was named 2020 Global Company of the Year in Clinical Genomics Interpretation by Frost & Sullivan.

We’re looking for Senior Full-stack (also Back-end) Software Developers who can take direction and run with it, building out new and existing products and features. We are especially looking for those with experience using Golang and ElasticSearch, as well as Python, Ruby (and Rails), and front-end JavaScript (Angular, React, or similar).

https://www.genomenon.com/blog/job/software-developer-2/

Genomenon | Full-time | Remote (USA) | Senior Back-end and Full-stack Developers

Genomenon is an AI-driven genomics company. Our mission is to save and improve lives by making genomic information actionable. We organize the world’s genomic knowledge and put it at the fingertips of clinicians to help diagnose and treat patients suffering from rare diseases and cancer, and to help precision medicine companies create targeted therapies for diseases.

Mastermind Genomic Landscapes inform pharmaceutical and bio-pharma companies on precision medicine development, deliver genomic biomarkers for clinical trial target selection, and support CDx regulatory submissions with empirical evidence.

Genomenon was named 2020 Global Company of the Year in Clinical Genomics Interpretation by Frost & Sullivan.

We’re looking for Senior Full-stack (also Back-end) Software Developers who can take direction and run with it, building out new and existing products and features. We are especially looking for those with experience using Golang and ElasticSearch, as well as Python, Ruby (and Rails), and front-end JavaScript (Angular, React, or similar).

https://www.genomenon.com/blog/job/software-developer-2/

I have what I call my 80/80 rule of early-stage startups, which is that for 80% of new startups, 80% of what they need is CRUD operations (create, read, update, delete). So basically, input data in this format, store it in this format, then display it in this format.

I think Rails is one of the most productive and capable frameworks for building and evolving CRUD applications, which is the reason it still remains my recommendation for new startups.

Are you sure? The entirety of the comic book genre is almost 100% based in social issues. Even going back to the first superhero, Superman, in the first issue of Action Comics #1, the protagonist rescues a woman about to be executed for a wrongful murder conviction (wrongful convictions and death penalty), rescues another woman being beaten by her husband (domestic abuse), rescues Lois Lane who was kidnapped by a gangster for rebuffing his advances (sexual harassment), and investigates a senator suspected of corruption (political corruption). It also acknowledges topics like media propaganda with the Daily Planet editor wanting to stir up the news. That's not even to speak of later super heroes like the X-Men whose entire existence was an exploration in marginalization of groups based on genetic differences. If those aren't main plot points based on social issues, I'm not sure what is.

Maybe it's just that as kids, we didn't recognize these things as being so politically charged as we do in our adulthood. Perhaps it's because as kids, it just seemed like common sense that these things shouldn't even be controversial so much as just being obviously bad things against which a hero would fight. It's hard to maintain a kid's perspective on things when we leave our kid years.

Edit: wording.

Genomenon | Full-time | Remote (USA) | Senior Back-end and Full-stack Developers

Genomenon is an AI-driven genomics company. Our mission is to save and improve lives by making genomic information actionable. We organize the world’s genomic knowledge and put it at the fingertips of clinicians to help diagnose and treat patients suffering from rare diseases and cancer, and to help precision medicine companies create targeted therapies for diseases.

Mastermind Genomic Landscapes inform pharmaceutical and bio-pharma companies on precision medicine development, deliver genomic biomarkers for clinical trial target selection, and support CDx regulatory submissions with empirical evidence.

Genomenon was named 2020 Global Company of the Year in Clinical Genomics Interpretation by Frost & Sullivan.

We’re looking for Senior Full-stack (also Back-end) Software Developers who can take direction and run with it, building out new and existing products and features. We are especially looking for those with experience using Golang and ElasticSearch, as well as Python, Ruby (and Rails), and front-end JavaScript (Angular, React, or similar).

https://www.genomenon.com/blog/job/software-developer-2/

This seems to be a prime example of the Pareto Principle, or the 80/20 rule. About 12 years ago, I had built something to do something similar and published how I did it [1]. It was a very simple process, which my younger self called "the meat algorithm", as in, how to get the meat of an article.

It was far less code and worked perfectly 95% of the time (though, the average web-page was also a little simpler 12 years ago). But that code would have quickly ballooned out if our use-case had called for addressing the other 5% of webpages, as the Firefox Reader View must do.

[1] https://www.alfajango.com/blog/create-a-printable-format-for...

Genomenon | FUll-time | Remote (USA) | Senior Back-end and Full-stack Developers

Genomenon is an AI-driven genomics company. Our mission is to save and improve lives by making genomic information actionable. We organize the world’s genomic knowledge and put it at the fingertips of clinicians to help diagnose and treat patients suffering from rare diseases and cancer, and to help precision medicine companies create targeted therapies for diseases.

Mastermind Genomic Landscapes inform pharmaceutical and bio-pharma companies on precision medicine development, deliver genomic biomarkers for clinical trial target selection, and support CDx regulatory submissions with empirical evidence.

Genomenon was named 2020 Global Company of the Year in Clinical Genomics Interpretation by Frost & Sullivan.

We’re looking for Senior Full-stack (also Back-end) Software Developers who can take direction and run with it, building out new and existing products and features. We are especially looking for those with experience using Golang and ElasticSearch, as well as Python, Ruby (and Rails), and front-end JavaScript (Angular, React, or similar).

https://www.genomenon.com/blog/job/software-developer-2/

Saying the effects of masking on younger age groups and with cloth masks in this study were statistically insignificant is also a misrepresentation of what the study found. The actual paper itself can be downloaded here:

https://www.poverty-action.org/publication/impact-community-...

From the paper:

    We found clear evidence that surgical masks are effective in reducing symptomatic seropreva-
    lence of SARS-CoV-2. While cloth masks clearly reduce symptoms, we find less clear evidence of
    their impact on symptomatic SARS-CoV-2 infections, with the statistical significance depending
    on whether we impute missing values for non-consenting adults. The number of cloth mask vil-
    lages (100) was half that for surgical masks (200), meaning that our results tend to be less precise.

Furthermore, they said that even without that consideration, the cloth masks still reduced symptoms of Covid, which is still a great win for masks, as it seems they reduce the viral load received by an individual enough to reduce the severity of their infection. So saying the overall advantage of masks was statistically insignificant isn't really true.

What the trial really found was indeed that masks seem to work, and surgical masks work better than cloth masks. This aligns with what many other studies have found as well. However, even though that's probably a valid inference to draw from the data collected in the trial, it's not really what the trial was designed to find, which was more specifically that if you actively distribute masks to people and encourage them to be worn, then more people will indeed wear them, and this in turn seems to have a positive effect on reducing both the transmission and severity of Covid.

There's another problem with the article's argument. It contends that the many studies on mask effectiveness have various shortcomings such as not being specific to kids, or having vaccination rates as a confounding variable. And therefore, we should not have mask mandates (despite the evidence that they work, however non-specific to school children they may be).

But its reason for concluding that children should not wear masks is because of all of these issues that themselves aren't well studied or understood by the article's own admission:

Despite how widespread all-day masking of children in school is, the short-term and long-term consequences of this practice are not well understood, in part because no one has successfully collected large-scale systematic data and few researchers have tried.

Why in this case should the serious lack of evidence for the risks of masking children outweigh the greater-but-imperfect evidence for the risks of increased transmission of Covid by not masking children? It seems like a double-standard allowing the author to arrive at their desired conclusion.

I also helped with a project once that used pgloader to migration an old MySQL db to Postgres, and I think this article may actually explain one of the issues we found. We had a UTF-8-configured table in MySQL (we also discovered in the project, that MySQL can have encodings set differently per-table, while Postgres sets the encoding for the entire database) with UTF-8 data, and when we migrated it into a UTF-8-configured Postgres database, some of the UTF-8 characters were silently corrupted. They were UTF-8 characters in both databases, so the corruption didn't raise any errors, but they were different characters, so when the data was read back out by the application, the text was different.

We only caught this, because thankfully, we had written a manual checksum script, which looped through every table, read out all values from each row into the application, and hashed the results, then compared between when the app was connected to the source MySQL database vs the destination Postgres database. We ended up having to massage and fix those silently-corrupted characters.

Genomenon | FUll-time | Remote (USA) | Senior Back-end and Full-stack Developers

Genomenon is an AI-driven genomics company. Our mission is to save and improve lives by making genomic information actionable. We organize the world’s genomic knowledge and put it at the fingertips of clinicians to help diagnose and treat patients suffering from rare diseases and cancer, and to help precision medicine companies create targeted therapies for diseases.

Mastermind Genomic Landscapes inform pharmaceutical and bio-pharma companies on precision medicine development, deliver genomic biomarkers for clinical trial target selection, and support CDx regulatory submissions with empirical evidence.

Genomenon was named 2020 Global Company of the Year in Clinical Genomics Interpretation by Frost & Sullivan.

We’re looking for Senior Full-stack (also Back-end) Software Developers who can take direction and run with it, building out new and existing products and features. We are especially looking for those with experience using Golang and ElasticSearch, as well as Python, Ruby (and Rails), and front-end JavaScript (Angular, React, or similar).

https://www.genomenon.com/blog/job/software-developer-2/

So they included the SSNs in the HTML source, and then said the reporter hacked and unencrypted the HTML by reading the non-displayed SSNs in the source. That's like taping the SSNs up to the inside of a tinted window and then saying the reporter committed breaking and entering by shining a flashlight on the window.

Nice. I've actually built a startup doing exactly this a few years ago, called PriceLocal [0]. Though, unfortunately, it never really got off the ground in terms of traction.

We ended up spending I think around two years building out the functionality. Most of that time was spent arranging agreements with brick and mortar businesses to share their inventory data with us, so that we could make it function on a per-product basis. You could shop normally on Amazon, and it would drop down a banner when your product was available from a local business at or below the Amazon price. We also had it working as extensions for Chrome, Firefox, and Safari. This article had a good screenshot showing how it actually worked [1].

That was definitely one of the favorite startups I've helped build to date. It was also cool to see it featured on The Today Show [2]. I'm super bummed it didn't end up working out. Good luck to you!

[0] https://www.producthunt.com/posts/pricelocal

[1] https://www.today.com/video/new-app-promises-to-find-amazon-...

[2] https://www.cnet.com/how-to/find-local-stores-that-will-matc...

EDIT: By the way, nice name! I can't believe we didn't think of that.

If I'm not mistaken, the studies indicated that in-person schooling could be done relatively safely with proper precautions, but in the US, most schools didn't/don't have the means to implement many of the precautions outlined. It's not really as simple as arguing that in-person schooling is safe, but democrat-controlled areas are closing down the schools anyway.

I'm also not sure what outcomes you're talking about with/without mask mandates. There are a lot of studies that show the effectiveness of mask mandates in slowing the spread of the disease. Do you have a study that shows outcomes are roughly similar in terms of spread and mortality between places with and without mandates? So far, I've only been able to find commentary that distorts the findings of the studies they purport to be citing.

Do you think racial cases of bias outweigh other demographic biases? And what's your opinion of the general level of intent (or lack of, to fix) relative to other issues?

When talking about systemic bias, I think it's less about intent, but that is one of the things that makes it so dangerous in terms of the level and longevity of impact; it's much more difficult to fix when there is no single bad actor at which to point the finger. This makes it easier for people to deny, either outright or at least in extent, and makes it harder for the systems to be fixed.

I don't know that I could place one type of demographic bias over another, but I think it's better for people to understand and acknowledge systemic bias of any form, if we have any hope to rectify any of them. I also think a lot of them are woefully entangled. For example, I've heard the argument that some bias isn't racial, it's economic, as in, a bias against poor people not people of color. But when there are already so many systemic economic biases against people of color, a lot of times it ends up being a distinction without a difference. I have plenty of examples of this, but don't want to dilute the impact of that last thought by going into detail.

That just shifts the definitional question to the scale of the institution, structure, or process.

I agree. Systemic bias is everywhere, for almost all scales of systems.

Is the definition, "a system is broad" or (I think you mean) "broad as in flexible", such that a system can be anything? Any micro or macro population?

Yes, you're right, I meant flexible. We can debate the severity of the impact of some form of systemic bias based on the size of the system (and the population or sub-population impacted), but to the individuals who are the victims of systemic bias, the result ends up being the same. At the end of the day, I don't think I'd fault any individual for taking up the cause of identifying and rectifying bias at any scale.

I guess where I was going with this is that the list seemed to be about some things that are very broadly impactful but not specifically racist, like facial-recognition not recognizing anyone but white men very well, but then a related issue of how this is encoded in the system. Is usage required, or was it random, etc.

Yeah, I didn't really provide much context for why I specifically keep a list. It's not intended to single out racism, and in fact, it is intended to focus on unintentional codification of bias, because I think there are more software developers out there who are at risk for unintentionally codifying bias into their algorithms than there are those who may intentionally encode it.

The reason I keep the list is as a reminder of the level of impact you can unintentionally have in the systems you build without extremely deep thought and broad context. It's a reminder that what we do can have real impact on real people in ways we never imagined.

This is especially recurring as a theme for machine learning and AI systems which are taught based on limited datasets which often systemically under-represent those to whom it will end up being applied.

Such that maybe top-to-bottom it'd be sorted by breadth. How many people does each impact, and how deeply-mandated or intertwined are the issues. And then different populations and problem to both give contrast, but also to indicate (when complete) the demographics of the problems.

I think this is a good idea.

So I was wondering if you'd focused on black issues to make the list.

I did not specifically focus on that. I started tracking the list I think 2 or 3 years ago, and I think issues of systemic racism have been especially visible in the mainstream in this time period for reasons I won't get into. So, it's largely a reflection of the systems I've been made aware of through research and publication.

So, statistically, how much subpopulation misrepresentation would you expect in various ways? And are you saying there's more in general, or more racial, than expected?

I have no idea. I just know that I've not yet been able to find any real systemic bias, at least in the US, against rich, caucasian males. If the question is, which slice of demographics is the most biased against, I think there are some studies which have looked into that, but I couldn't say myself.

I interpreted the one about kidneys as an error because of changing demographics and how people had caught it, made note that this sort of thing happens, and started checking other research for similar sampling bias. It read as a success story where the one about facial-recognition was more actively black mirror.

There is certainly a bright side in these things being identified as problems which need to be solved. However, even with the kidney example, I don't know that it's a success as much as further evidence of how difficult these things are to rectify at a systemic level, since the vast majority of doctors using the still-biased eGFR formula have no idea that it has this problem.

More that I'm wondering if it was an attempt to model, like for predictive policing, or a tool sold to simplify the sorting of people? Because models are good, even when they're wrong, but crappy predictive tools are worse than useless and - where I was ultimately going with this - perhaps fraudulent to sell.

I mostly agree with this sentiment. The biggest issue I see is that most of the models weren't built with the intention of being biased or badly predictive, and those selling them often don't know until after they've already been sold and broadly adopted. Fraud generally requires mal-intent, which most of these models and products didn't have, at least if we're being generous and optimistic. Negligence is probably closer to it, but I think difficult to prove in such a new frontier.

I just finished reading a book called, Weapons of Math Destruction, which talks about the damage models can do. The author posits a set of tests to tell whether the model is beneficial or destructive. One of the hallmarks, the author argues, of a good predictive modelling system is one which includes feedback into the system as a result of its predictions. Many examples, especially those which alter the outcome by making the prediction in the first place (e.g. criminal sentencing based on predicted criminal activity, excluding candidates from the hiring process, firing teachers based on opaque models), have no feedback mechanism for directly determining how successful their predictions were.

But yeah, to recommend sentences, total crap.

One of the other primary tests is dependent on the context of how the model is used, so it's not really reasonable to try to determine the benefits of a model in isolation with no consideration for the appropriateness of its implementation and utility. In this case, it's hard to defend a model with bad predictive data based on the presumption that it could have been more beneficial with another use, when the reality is that it was only sold with one use and that one use was harmful. Though, even this ignores the fact that it had no built-in feedback mechanism to determine how good a job it was even doing with its predictions in the first place.