HN user

Kassius509

4 karma
Posts2
Comments23
View on HN

That is interesting. I"ll have to explore that correlation and make sure that we have a good baseline comparison for the v2.0 model.

Thanks!

Actually, username & story titles aren't included in the model. We may experiment with including them in the future but for now we left them out to avoid any undo bias.

Sweetest users is a good question. I'll have to build out that view some time.

For this version we used TextBlob out of the box with some custom weighting to avoid false positives.

We'll relaunch in about two weeks with an improved model that we're currently training on a much larger labeled dataset and SOTA BERT model. :)

Your account is pretty new. We intentionally remove any accounts with less then ? number of comments to discourage people from attempting to set a record. Lol

Sorry about that, and thank you for the feedback. I'll try to make sure it is more readable in the next version. Thanks for taking the time to check it out!

We used TextBlob for this one and penalized to help avoid false-positive classifications for saltiness (subjectivity + negativity).

You're spot on. This model is based off a dictionary with scored words!

For the curious you can see the dictionary here: https://github.com/sloria/TextBlob/blob/eb08c120d364e9086467...

The package used is a pretty popular one called TextBlob. It is nifty for working with unlabeled data like we have with the HackerNews dataset.

We really focused our definition of saltiness around being a combination of (subjective + negative) comments.

We reduced the impact of (objective + negative) as we feel that criticism, while at times painful, if presented objectively isn't necessarily salty.

We built this model fast (1 week) and have since iterated this week into developing a Fine Tuned BERT model that we are training over a much broader set of toxicity, demographic, and polarity features. The training set is much larger and higher quality so we are expecting a large jump in precision upon deployment.

I hope the app gave you some good chuckles as you went around though. It's hard to explain how excited I felt when I saw pg_is_a_butt at the top of my pandas data frame the first time I processed the data.

It's doing a little bit right. :)

I agree. Would have been much easier too. Unfortunately, HN doesn't have downvotes. The current model does incorporate upvotes, but we're seeing a lot more success training with like-kind labeled datasets + BERT fine tuning.

Thank you for trying the app and hopefully V2 will leave you feeling like the system is more precise.

Thanks for trying my app! And thank you for taking the time to respond!

It does have a LONG way to go. These are valid criticisms. And all areas we are working to improve on with our next ML model. (Sad vs Negative, rating numbers as "salty",

This model is pretty simple. It's using TextBlob and looking for a combination of negative sentiment (not necessarily condescending) and subjectivity. Essentially hand built heuristics derived by weighting each word in the sentence. Not a great way to make predictions.

The model is FAR from great. But great from afar. For high level (overall user saltiness) it performs better.

The unlabeled dataset of this size presents some unique challenges but in our testing of our new model (based on SOTA BERT fine-tuining & a large labeled training dataset) the results look promising. I'm really looking forward to getting it deployed.

I am encouraged by the words of @pg who said "you can and should give users an insanely great experience with an early, incomplete, buggy product, if you make up the difference with attentiveness.

Can, perhaps, but should? Yes. Over-engaging with early users is not just a permissible technique for getting growth rolling. For most successful startups it's a necessary part of the feedback loop that makes the product good. Making a better mousetrap is not an atomic operation. Even if you start the way most successful startups have, by building something you yourself need, the first thing you build is never quite right. And except in domains with big penalties for making mistakes, it's often better not to aim for perfection initially."