HN user

reader5000

1,143 karma
Posts2
Comments534
View on HN

I think his argument assumes the existence of pseudorandom generators which map a small amount of "true" random bits to a large amount of bits that look random to any polytime observer. The "derandomization" is that we just have to check all possible states of the seed bits which hopefully will be logarithmic in the size of the problem so you can do exhaustive checking.

My intuition would be that both text and fingers are high frequency parts of the image and most image patches across most images don't have that amount of frequency. Text and fingers are relatively rare in images compared to e.g. clouds and textures on surfaces. Because of the rareness and difficulty of text image patches the models just don't dedicate that many parameters to it.

This could be entirely wrong however.

It would be interesting what would happen on a dataset with nothing but text.

Theres all sorts of statistics about the increasing amount of men that are virgins into their 20s and beyond, declining marriage rates, declining birth rates, "inceldom" etc. etc.

Its certainly plausible that at least for some subset of people becoming porn addicts when they enter puberty is disrupting their ability to form relationships as an adult.

Having said that its not really clear censorship is the answer. Probably over time phenotypes overly sensitive to porn addiction will just go extinct.

I think in the modern era a very good piece of advice, particularly for those of us without gorilla-like stamina to comb through a math text, is to go on your favorite video website and watch through multiple videos on the topic.

I think one missing step is to fuse training and inference regime into one, just as in animals

It's not clear it is one. Sleep is training (replay from hippocampus). Wake is inference.

I think 8 months is a little short for the utility of a new tech to be fully realized and utilized. I'm pretty sure there were still horses on the roads long after 8 months after the Model T first went on sale.

Recurrent neural nets is the general term for nets with memory as you describe. Indeed LSTMs, a type of recurrent net, used to be state of the art on language tasks until the GPT transformer models. I'm sure somebody somewhere is working to make a transformer with recurrency. The neural turing machine mentioned in another comment is such an example but it seems to have been abandoned.

The main problem with recurrent models is its hard to train them with backprop. For example the GPT-3 can handle sequences up to ~2000 tokens? I'm not sure what the largest sequence LSTMs could be trained on but it was probably less.

Yeah I thought the recursion+memo wasn't actually "dp" until I looked it up. Recursion without the memo is not DP however since you hit exp running time/memory and the whole point of DP is to avoid this.

In "Algorithms" [Dasgupta, Papamdimitriou, Vazirani] they state that the memoization (e.g. querying a hash table) can have significant overhead leading to a large constant factor in the big O analysis. On the other hand, the bottom up approach using a table solves all the possible subproblems including ones that are not needed and ones that the recursive approach would avoid. So from a big O perspective the recursive top-down and the table bottom-up approaches are the same but there can be significant constant factor differences.