HN user

sin7

310 karma
Posts0
Comments108
View on HN
No posts found.

Even worse, a good portion of the population believes that a good portion now believes that ... whatever you just said. And this good portion is annoying. Bring up nonsense like this in conversations where it does not fit, in the middle of dinner, in the middle of pumping gas. Like damn, I'm watching out for one of those kids with a gun just waiting to exhibit his mental health and you want me to worry about the guy in a dress?

    runs <- 100000
    x <- vector(mode = "numeric", length = runs)
    for (i in 1:runs){
      while (sum(sample(1:8, size = 3, replace = TRUE)) != 24){
        x[i] <- x[i] + 1
      }
    }

    summary(x)
     Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
      0.0   146.0   353.0   511.8   708.0  5112.0 

    quantile(x, c(0.5, 0.8, 0.9))
     50%  80%  90% 
     353  824 1187
Strangely enough the mean agrees. The other ntiles are off a bit, but that's randomness for you.
    runs <- 10000
    x <- vector(mode = "numeric", length = runs)
    for (i in 1:runs){
      while (sum(sample(1:6, size = 3, replace = TRUE)) != 18){
        x[i] <- x[i] + 1
      }
    }

    summary(x)
    quantile(x, c(0.5, 0.8, 0.9)) 

    > summary(x)
       Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
        0.0    62.0   149.0   216.2   300.0  1902.0
    > quantile(x, c(0.5, 0.8, 0.9))
    50% 80% 90%
    149 350 495
A simple simulation. Run 10K times. Count the number of times it takes for three dice to add up 18.

The numbers very much agree with you. The median is 149. The 90th is 495 in the simulation, which is close enough to 496. There is very much a long tail in the data. So, the median and the average will not be the same. Is it a coincidence that mean is a 216?

    runs <- 10000
    x <- vector(mode = "numeric", length = runs)
    for (i in 1:runs){
      while (sum(sample(1:6, size = 3, replace = TRUE)) != 18){
        x[i] <- x[i] + 1
      }
    }

    summary(x)
    quantile(x, c(0.5, 0.8, 0.9)) 

    > summary(x)
       Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
        0.0    62.0   149.0   216.2   300.0  1902.0
    > quantile(x, c(0.5, 0.8, 0.9))
    50% 80% 90%
    149 350 495
A simple simulation. Run 10K times. Count the number of times it takes for three dice to add up 18.

The numbers very much agree with you. The median is 149. The 90th is 495 in the simulation, which is close enough to 496. There is very much a long tail in the data. So, the median and the average will not be the same. Is it a coincidence that mean is a 216?

1*0 = 1 1*1 = 1 1*-1 = 1

2*0 = 1 2*1 = 2 2*-1 = 1/2

When 1 raised to any power equals 1, does the power matter at all? Even if it's unknown, the answer is 1.

Is it too deeply entrenched to change? The number of times I have had a data.frame grouped when it wasn't supposed to be, I can count on my fingers. But the hours that I spent trying to figure it out must amount to a paycheck or two.

PostgreSQL is not in the game. Completely different purposes. PostgreSQL is great for what it's meant to do. Analytics is not it. Reads are too slow.

The answer is yes. I used ClickHouse to calculate and forecast sales of products at a dozen or so stores. The compression was huge because it's essentially the same data every day except for changes to the inventory. At the time I checked vanilla PostgreSQL, TimeScaleDB and ClickHouse. It wasn't even close when it came to storage or performance. ClickHouse allowed me to work off of an old workstation where I installed Ubuntu.

In my case the data arrived in CSVs with around 20k skus. Had they arrived a couple at a time, I could have created a CSV and written to ClickHouse later or used any of the other storage methods available in ClickHouse.

I've been doing the data thing for a while. During one of my defenses of R, someone brought up that R was a black hole. That if you programmed in R, you were a user who just filled in the correct function arguments and it just spit out the answer. And that was when my thoughts on machine learning changed.

The vast majority of us are users. We massage the data to be in a certain shape, then feed it through a machine that someone else created. We can change the parameters. We can change the data. But few of us are going to look in to the code of a random forest function.

I've switched tracks and started doing web development. Playing with the hyper parameters in machine learning is no different than changing the feel of a drop down by changing the colors, fonts and other things to fit a certain aesthetic.

I could be wrong, but I have yet to meet anyone that has done anything besides use packages created by others to call themselves data scientists. I think that opens it up to becoming just another tool no different than Excel.

Some Russians too. Maybe all Russians.

Truth be told, I don't know and it matters very little to me. Yet I have read more than one article about Kenyans and hiding in the mountains from testers. Maybe it was propaganda, but I read it.

I have no dog in this fight. I'm sure Kenyans are great runners. Maybe the best.

Good data scientists also need to be great at a couple of things. They must be excellent presenters, and excellent diplomats. Getting the data is the hardest part. Data silos are create all over the place and getting access to that data requires speaking to a lot of higher ups.