fortytwo79

And could you imagine the bokeh on a portrait?!

Isn't this already the case with marijuana? Legal in one state, if you travel with it to another state, but make a stop in an illegal state, you can go to jail?

I don't know much about laws on that topic, but it seems to be a similar case to me.

Okay, now someone needs to do the same study with innocuous end game scenarios. (I know this study links to one other paper that makes this case, but I can't access it, and the abstract has a biased tone)

If you're going to explore worst-case, so you can think through preparedness, then you should also consider the trivial case to make sure we aren't over-responsive either.

The article also states this:

"With just over 19 weeks into the year, this averages out to about 10 such attacks a week," when referring to the Buffalo shooting.

Saying "10 such attacks" implies the attacks are all similar in characteristics beyond just the number of people involved. The article talks about mental health, it talks about a pre-planned desire to kill. But it doesn't talk about gang violence. These are very different sources of intent. If you want to use the gun violence archive data to make a point, it should be about gun violence in totality - not cherry picking their numbers to make the case that the US is full of hate-filled crazies who are randomly shooting places up 10x per week.

But Buffett's point is that each unit of investment he pursues has the potential to produce incrementally more value. You're saying the aggregate technology underlying bitcoin has value - which it does. But you don't get twice the tech value by doubling your amount of coin owned. Conversely, if you double your farm holdings, you have double the ability to produce food.

Also, Buffett's quote here isn't some significantly new perspective. Hasn't this been the primary argument against bitcoin since the beginning?

And yet, you shut down the possibility of religion without any degree of exploration.

I don't need you to be religious. I understand annoyance with the fact that the culture you're in is pushing something you don't agree with (I experience that as well on other topics). What I take issue with is the attitude that religion deserves condescension and vitriol, simply because you don't subscribe to it, or because you find it annoying. I see no recognition that faith and logic aren't mutually exclusive.

Don't have religion - that's fine. But don't pretend to understand (and then trivialize) any religious tenet, if you're unwilling to fairly explore it.

Science is exploration. Religion is not.

You're conflating religiousness with dogma. Dogma, by definition, is belief without exploration. Religion, however, invites endless exploration.

This kind of condescension towards religion is so close-minded, and it's incredibly tiring. I think you'd be surprised just how much the faithful appreciate and embrace science, and how much philosophical thinking about reality and existence we actually do. Adult religiousness is not the same as the cartoon version that children are introduced to, that so many atheists use to attack. The lazy, unintellectual thing to do is to base all your understanding of reality only on what science can describe, and to never explore beyond what can be proven as a "fact." Your statement about explaining away facts paints the faithful as simple-minded rubes, which is a cheap shot and is absent any understanding of how the religious actually view the world.

I appreciated the "retro encabulator"-esque reference in the industrial sample.

Agreed. Speaking as someone who has hired hundreds of software engineers, I can tell you that having a degree at all tells me a lot about your work ethic. Sure, I'd prefer a comp sci background, but your degree is nearly as good. It's an indicator of how you think.

You know what other degree I see a LOT of in SWE? Music. I've hired countless music majors.

People have always been that way. Before Rogan showed up, before the internet was even invented, there were "nutters" that had deeply misinformed perspectives which they followed, sometimes, to death. I'm sure someone will argue scale is the differentiator, but I'd argue that the proportions of people who ignore facts vs consider facts, is probably unchanged. (Although admittedly, I have no statistics)

Also I think a big part of the problem is the aggressive labeling of content as "misinformation" or "fake news." To me, misinformation implies propaganda issued and promoted by an enemy entity. But today, it is a term that is used to mean anything that has a fact (whether verifiably correct or incorrect) that implies a conclusion that is generally unacceptable.

For example, if the generally acceptable premise is: "everyone who is able should get a vaccination," then publicly talking to someone harmed by a vaccination (even if it's true) would be considered misinformation, because it potentially concludes something opposing the acceptable premise.

If we can't openly share ideas, good, bad, informed, misinformed, then the 99% (fake number) of us who aren't "nutters" that follow bad advice to extreme conclusions, will be denied the volume of data, perspectives, and opinions we need to make a truly informed decision.

I don't think that comment was implying those measures are to blame for rising deaths. I think it was to argue that they haven't had the impact they were expected to have.

Yeah, I need to fix that. You're seeing a performance bug. There's a cache in front of the API, and sometimes if you hit the API outside of the cache window, the lambda running in the background is a bit slow. Subsequent loads will hit the cache.

Although, I'm surprised you didn't see the page load UI. There should be a pulsating circle to indicate it's loading.

Thanks for the feedback!

You bet!

There's three major phases to the document matching: Corpus Management, Frequency Modeling, and Matching & Clustering.

1. Corpus Management

- First we pull the raw articles

- Then we build a bag-of-words document, based on our overall dictionary, so we end up with a bunch of numbers to represent all the words and their frequencies for each document.

- We store all the stories for that day as bag-of-word, in a Matrix Market format.

- The day's .mm file is updated every few minutes into an S3 bucket, along with the dictionary. This way I can easily compose a corpus of documents for a single day, multiple days, or all time

- I use the Gensim library for Python to do most of the above.

2. Frequency Modeling

- As stories come in throughout the day, I periodically refresh a TF-IDF model for the entire corpus of stories.

- TF-IDF just allows us to see which terms happens frequently in a story, but relatively infrequently across all stories. So a word like "Hacker" would be relevant if it appears frequently, but "the" would not since it occurs across all documents.

- TF-IDF modeling also uses the Gensim library.

- TF-IDF model is stored in another S3 bucket, to be pulled on update by the Matching & Clustering job.

3. Matching & Clustering

- First we fit a limited corpus (usually the last 10k or so stories) to the TF-IDF model, and keep that as a sparse matrix.

- This gives us the ability to quickly determine the per-word importance of a document, and to represent that as a vector.

- Next we do a simple cosine similarity of the 10k documents against themselves. This tells us how much of an angle there is between each document vector against each other document vector. In other words, it's a measure of similarity.

- We limit all of this to 10k documents, because it would be computationally prohibitive to compare ALL stories to ALL OTHER stories. Since most news stories are published relatively close to one another, we only have to compare recent stories. 10k seems to produce good results. We can further shard this data set by news category if we need to (i.e., compare only sports stories or political stories).

- Next we use SciPy, and create a Ward linkage matrix.

- Next, also using SciPy, we use fcluster to do agglomerative clustering. These last two steps produce a cluster tree that puts similar stories together.

- Finally, we slice apart the clustered data set at a certain similarity height. Kind of like cutting through a head of broccoli above the stalk. The clusters we're left with are the most similar articles of that batch of 10k.

After all that is done, we just store associations between articles that were determined to be similar to one another. Since we're constantly running this process, the 10k story window keeps sliding forward, so you're able to store similarities of stories that are similar to far older stories, provided there is another story in-between that both are similar to. For example, if story 12,000 is similar to story 9,000, and story 9,000 is similar to story 10, you'll end up storing that story 10 is similar to story 12,000, even though you didn't directly compare them.

I built Drewes.NEWS as a way to learn NLP and serverless architecture, and to find bias in news by reading the same story from multiple outlets. Now it's evolved into a useful, privacy-focused tool.

It's privacy-focused in that there are no cookies, no usage of Google or Facebook components (like Google Analytics or Ads). No data tracking on users whatsoever.

There are bugs I'm aware of, but am looking for feedback on if this format and function is useful.

For those interested in the NLP side of this or the serverless side, I'd be happy to answer questions about how it was put together. The short version is, I pull down RSS feeds from 33 news sites (approximately 1M stories in the database so far), store them, create a term frequency model, cluster the most recent 10k stories based on TF similarity vectors, then store story similarities.

In the future I'd like to add paging, searching, and more filtering. I'm also thinking about having a URL for each story, that would show all the similar articles. That way, if you don't want to link directly to a particular news source, you can link to the drewesnews aggregate URL, and the reader can pick whichever source he/she wants to read.

Any feedback would be much appreciated!

Ha! I literally LOL'ed this. Well played.

I think the full lyric here is actually "pandemic ain't real, they just planned it."

The meaning and intent of the lyrics are open to interpretation, but I think the latter part of the lyric implies political commentary rather than medical commentary.

Also, I think we need a tighter definition of "medical misinformation." Is it to promote anything that is medically harmful? Well, then why can musicians sing endlessly about drugs and unsafe sex? What about videos on non-fda regulated vitamins and minerals, or weight loss solutions? Surely most doctors wouldn't advise taking recreational drugs, or jumping on to the latest weight loss magic pill.

And yes, YouTube is a private enterprise. But we can still, as a society, openly dialog on what we think is wrong with their practices.

Watched it here: https://mobile.twitter.com/Saint_BTC/status/1451559663753859...

Mostly just the rappers waving their hands around and showing their guns.

I wish people would pay me to write headlines like this.

How about this idea for an article:

"People who find baseball stadium chairs uncomfortable risk developing diabetes.

Agree completely. Whenever I see the phrase, "let that sink in," that's a signal for me to stop reading. They're just going to tell me to be angry.

Trump was nominated because he brought more middle east leaders to the table for peace than any president in recent memory. Stalin and Hitler murdered gajillions. So, what's your point? If it's that Trump did things not considered peaceful, despite the peace inducing actions, then why not call out Obama for the drone bombing of Anwar Al-Awlaki, a US citizen? And Obama GOT a peace prize.

To be clear - my point is not to endorse the Trump peace prize nomination, or even to make this a Trump vs Obama debate. My only gripe here is this constant inclusion of Trump as a villain, of Hitler caliber. It's not a generally accepted premise, so let's stop using it as one.

The primary thing I disagree with here is the thought that we can squelch our way to a more informed public. You clearly believe you can discern truth from lies, as do I. So why should the solution to misinformation be to trust a small group (who we didn't elect) to parse truth from fiction? I'd rather have access to all the information, and be better at discernment. And I reject the idea that we represent a minority of the United States. I think most people are way better at detecting lies than we give them credit for. The problem is, both sides lie. So it's too easy for us to view those who hold opposing views as ignorant fools, because they are on the side of a lie, regardless of if they believe that falsehood or not.

I think he/she is saying labor rates are a supply/demand calculation, not a value one. The I-beam in your house might be the most valuable thing to the structural integrity, but it's probably not the most expensive thing in your home. Scarcity dictates that something else less valuable to your existence may cost more. You may not value that scarcity enough to pay that premium, but value alone does not set price. Air is the most valuable thing to my life, and I pay nothing for it.

So, a nurse or teacher provides a very valuable service, no doubt. Salaries won't naturally go up until availability goes down.

This, 100%. This is the point. Everyone is trying to assert the harmfulness of "misinformation," but is ignoring the harmfulness of censorship.

Also, has anyone considered that conspiracy theories are on the rise because they're entertaining rather than because they're believed?

Here's my problem with that. Twitter, Facebook, and Snopes likely don't have physicians, virologists, and epidemiologists on their staff. So who are they to determine truth or harmfulness of those doctors' positions? Snopes (and most of the media) largely attacked the unrelated views of one of the doctors. And yeah, she sounds like an odd duck. But that doesn't change the fact that FB and TWTR took it down for being "harmful," which they are not qualified to judge. Also, I find Snopes aggravating, because of stuff like this. They said they couldn't find evidence the doctor treated 200+ patients with COVID, thereby implying the doctor was lying. But lack of evidence isn't evidence of a lack of truth. Plus, that's clinical data. Wouldn't it be illegal for Snopes to have access to that without patient consent?

There are people of all demographics who've had a raw deal. In modern America, there are black people born into lives of "privilege," and there are white people born into adversity. So does flagging a business as minority-owned tell the full story of the owner's life circumstances? Certainly not.

On the other hand, veteran ownership is a common thing to see advertised. I guess I have no problem with businesses self-advertising their own values. I take more issue if Google has a short list of business attributes, and only causes they deem worthy make the list. I don't know if that's the case though.

You keep trying to pull prop 8 back into the conversation. This isn't a discussion on gay marriage.

While I agree that a CEO should be aware of their public image I think it's a sad commentary on the state of our society that public attacks and shaming have become the vehicle of protest now. This happens frequently with the constant push to de-fund voices by attacking their sponsors. Big corporations stand down, not because they agree ideologically with the vocal mob, but rather because they want to not become the next target.

How about instead of attacking the careers and livelihoods of those we disagree with, we instead support the ideas we do agree with?

But who is keeping score? You can't censure Alex Jones and QAnon, then give the MSM outlets that pushed the Steele dossier a free pass. They caused measurable harm to an entire nation. Most of QAnon is innocuous, even if conspiratorial. I'd argue Q is far less dangerous than a national media outlet that knowingly pushes dangerous narratives.

That assumes we should all seek to progress towards someone's ideal of human behavior and thought patterns. Why should she self-examine? To adhere to your standards for behavior? To a segment of society's standard? I argue that we need journalists and others who make us uncomfortable. That's how we learn to examine ideas for their own merit. If we cancel out people who espouse radical ideas, we cut off the pipeline of critical thinking, debate, and the surfacing of ideas. The only way to a better society is through more ideas. Not forced adherence to one that people are scared to challenge.

Her persona doesn't need to be fixed. She needs to be invited to more forums, so critically thinking adults can examine her ideas and either accept, reject, or refine those ideas.

Who is to say what the appropriate "normal" is? It seems to me that influencing a cultural norm is doing away with what the culture desires. You want to change the culture, not the norms. That's my big problem with all this PC wrongspeak policing. MY normal was a world where we can use terms like "blacklisted" and it not carry a racist connotation, because we were treated as intelligent enough to understand that it wasn't intended to be a racial superlative. This sort of action is driven by people's desire to NOT be labeled a racist. To signal that they agree with the ideal of racial equality. But it just further draws a boundary around segments of our population, and says you're either with US or with THEM. If you don't outwardly signal that you too aren't a racist, then you are THEM - a racist.

I for one will not change my language. My words mean what they are intended to mean. I may be black-balled for doing so, but at least I'm not actually a racist.

I had the same experience. It was a video of a riot line of police with shields walking down the street, and people shooting fireworks at them. I welcome any content that shows events that the media isn't providing. But if someone is going to curate a list like this, don't sell it as something it's not.