dfrankow

Please say more about how it could use people who aren't security researchers.

I totally agree but .. I feel companies evolve in an environment they do not control. If companies could just offer goods and services at a fair price without arsing around and be successful, I feel like we'd have a ton of such companies. I could be overly cynical.

This.

Thanks for your feedback.

We'll look into making the chat less obtrusive. I also don't love aggressive chat boxes, but didn't realize Intercom did that on mobile.

Wow, that's pretty harsh.

I don't accept the characterization as a "brogrammer without ethics."

I don't think anything I say would change your opinions, but I'm willing to answer questions if you have any you want answered.

Hi HN.

I'm Dan, CTO at Condorsay. We've built a tool that helps you make decisions, alone or with others.

* Summary

The short version of our tool:

- pick a goal (something to decide)

- pick factors important to the decision (helped by GPT-3)

- pick options (helped by GPT-3)

- use pairwise ranking to learn what you or a group think of the options

- see the results, including text notes (if you made the decision with others) and dissenters (people who disagree with the group)

NOTE: it does require a Google login to get past the "factors" screen. However, you get 5 decisions for free, so you can do everything for free. I read login makes HN cranky, but that is how our tool works. (A decision has to be owned by a user in the DB. Also, we are protecting against someone burning tons of money on GPT-3 calls.) Please don't be cranky.

* Motivation

We believe that decision-making could be improved by a tool that puts structure around it.

Some benefits:

- clarity: structure and record your decisions, alone or in a group

- focus: force hard choices with pairwise comparison

- revealed preferences: you actually don't always know what you think, but you learn through the simplest possible gut-level calls (pairwise choices). By the end, the results make sense, even if they aren't what you thought at first.

Further benefits if you’re making a decision in a group:

- alignment: get a group of people on the same page, by having the most important discussions quickly (i.e., where people disagree)

- independence: express your preferences before you see anyone else’s, to avoid information cascade

- asynchronicity: coordinate people in our increasingly remote-work world, mobile-friendly

Our scoring is straight-up Analytic Hierarchy Process (https://en.wikipedia.org/wiki/Analytic_hierarchy_process). For AHP in Python, we use the lovely https://github.com/PhilipGriffith/AHPy.

Happy to talk about paired-choice decision-making algorithms, AHP, GPT-3, or other TLAs (three-letter acronyms).

* Background

James, Andrew, and I worked on Barometer, an effort to use Facebook measurement to promote or demote ads to help defeat Trump in 2020. Even in that effort, we picked what to do next in various different ad-hoc ways. James thought, "Why isn't there a good tool for this?" and convinced us we should try to build it.

I look forward to your feedback and questions. Please don't be cranky.

Where was the UI? Help us!

This is prisonner's dilemma / tragedy of the commons. The best-performing ads made more money, anyone who didn't grab attention enough would be strongly pushed towards grabbing more attention by industry norms. It was rational for every individual to grab more attention, but it was bad for the group.

Prisoner's dilemma problems are hard to solve.

Police kill about 1000-1200 people per year in the U.S., depending on who's counting. See for example the Washington Post data. So it's not just tens over a decade.

https://twitter.com/UMNComputerSci/status/138496371833373082...

"The University of Minnesota Department of Computer Science & Engineering takes this situation extremely seriously. We have immediately suspended this line of research."

Can you provide detail or examples of why?

Now I'm confused.

I thought you meant split the infra pieces into different software components (process, machine, etc.), not split them by domain. Can you give an example, and why it's good to split by domain?

separate out your infrastructure from both your product and business

By infra, you mean for example a postgres DB? That makes sense. Don't want marketing to take down your website with a mail merge.

I think Nano just handed out a fixed amount of tokens through captchas. Now no more will ever be created, but they can still be transferred.

I am also rooting for currencies that don't use so much energy. I was intrigued by nano, but it looks like a pain to buy and sell right now.

Used to bother me too, as a mathematician. Then I learned about linguistics. If enough people use words a certain way, then it's vernacular. Sadly, I think this usage is long accepted.

ggplot?

If your benchmark is plotting 2 million points, it may also be slow. However, I find its ability to rapidly prototype tons of different visualizations useful. I try to avoid needing to plot 2 million points anyway.

Isn't ddg Bing results?

Firefox is way slower than chrome to just open a web page on my pixel 3a. Do others not have this experience?

Can you search protonmail email quickly? Search is pretty useful.

https://wiki.documentfoundation.org/Development/Easy_Hacks

A detailed post related to this: https://people.gnome.org/~michael/blog/2013-06-13-under-the-...

I wonder what the simplest changes to the codebase are that would make it much smaller. I wonder if there are tools that would make suggestions.

Hey all.

"Learn how looking for anomalies in your daily business metrics can help you more quickly find, learn about and respond to changes in your business."

Putting ("triggering") someone into an experiment group is most often some function calls: do they meet the criteria (e.g., country, ..), do they fall in the right bucket. The fact that they've been triggered is logged to Kafka, an open-source messaging system whose logs we push (eventually) into Hive.

1. Beware multiple tests: agreed. We could apply corrections (e.g., Bonferroni-like), but don't underestimate the logistical complication of this. Before a month starts, we don't even know how many experiments will be run (although we could try to predict). A different way to address this problem: use other information. Do the results make sense (good)? Is there corroboratory evidence (good)? Are there crazy outliers or things that smell funny (bad)? Of course the experimenter often has a bias (it worked!). Can they convince others? etc.

2. To compute statistical power, you need an estimate of effect size. For many experiments, we don't know; we could run pilots, but in the naive case that doubles the number of experiments to run, and time to wait. By default, we recommend a sample size that will detect a certain effect size (but not smaller ones). That is, we decide we are not interested in small effects, because it makes our life simpler.

Using Bayesian methods to learn about our experimental effects is very interesting to me. For example http://www.evanmiller.org/bayesian-ab-testing.html, which may be the type of bandit you refer to.

In the best case, it might be more robust to low-data outliers that cause random blips that look significant, as well as addressing the problem of looking at an experiment over and over (which frequentists are uncomfortable with: more chances to succeed).

However, it is non-trivial to understand how this works for us in practice. Examples: we want overall results as well as days-in results (to look for novelty effects); we would have to choose priors with consideration, because they have a huge impact on most results; etc.

This takes time and effort, and there are a lot of other things competing for those resources.

I work at Pinterest, and have some involvement in how we learn from experiments.

In each experiment, we assign users randomly based on a hash of experiment name and userid. So, we assume (and have tested) that the groups of one experiment have a fairly uniform mix of users from the groups of any other experiment. This means the primary effect of being in a particular experiment group G should be much larger than the effect of being in G and some other experiment's group. So, we usually analyze the effects of an experiment independent of any other experiments. Millions of users helps here.

I think this is how trials normally work: assume that random assignment into groups will spread all other factors equally across the groups.

There are experiments which depend on each other (change the same UI, one assumes a precondition of another), and that situation is complicated. We try to avoid it when we can, and negotiate carefully when we find ourselves in it.

You might also be referring to the fact that looking at hundreds of things will find "statistically significant" effects simply at random. As my advisor said, "95% confidence means wrong 1 in 20 times." That is a risk to be managed. We always have to ask things like: is this a reasonable outcome for this experiment? Do we have other corroborating evidence? Is the effect consistent? Do we have reason to disbelieve the result? Some fraction of our experiments are run incorrectly or have a broken implementation; we detect some, likely not all.

Also: is the risk higher to run an experiment that may be misinterpreted, or not to experiment at all and just release things? We like to learn from data if we think we can. There is art in the tradeoffs.

Aaaaand .. does it work?

I hate selling things on craigslist. When I tried to sell a comforter, I had to correspond via email with 9 different people before someone would actually come and pick up the damn thing. They kept dropping out silently. When I would ping them, they'd say, "Oh, I bought a different one." or "I decided to buy it from <blah website>."

The problem is there is no reputation on craigslist, so people can be as bad as they want.

I can't compare that to ebay, tho. Maybe ebay is worse for a seller.