HN user

highd

532 karma

http://www.highdimensionality.com/

contact@highdimensionality.com

Posts2
Comments164
View on HN

I'm fairly confident that the new reddit React implementation can be improved in performance by a factor of 3x to 10x. I would be interested to hear others who have good reason to explain why not. I can certainly imagine React-like systems that are capable of statically determining DOM influence sufficient to make comment-collapsing negligible.

You can do Q-Learning with a transformer. You simply define the state space as the observation sequence. This is in fact natural to do in partially observed settings. So your distinction does not make sense.

TFP cites decision transformers. Just using a transformer does not bypass the credit assignment problem. Transformers are an architecture for solving sequence modeling problems, e.g. the credit assignment problem as arises in RL. There have been many other such architectures.

The hardness of the credit assignment problem is a statement about data sparsity. Architecture choices do not "bypass" it.

Can you provide some technical details on what you do? Do you divide the number of channels on each real layered network by 4? I don't see anything describing this in the paper.

This model is showing predictions one day into the future. The "test set" plot is all predictions made with data from 1 day ago. The input sequences have size 1, so no recurrence is happening (see to_1dimension in https://github.com/miguelgfierro/sciblog_support/blob/master...). If you predict that tomorrow will have the same price as today you'll get better plots under the same operating conditions.

In[2]: TIME_AHEAD = 1

Train set has ~1e-6 MSE, Test set has ~0.8 (0.94^2).

EDIT: I should say this person is probably learning, and a lot of this is honest mistake.

Most of the research in compressed sensing type applications now is focused on more sophisticated prior distributions than sparse/Laplacian. These more general bayesian approaches provide significant performance improvements over LASSO etc.

Not to mention that it appears they're comparing against networks of the same architecture. If you build your quaternion components with with same number types as your reals you effectively have 4 times the number of parameters, which could be most of the benefit. They should also benchmark against similar architectures with equivalent parameter counts.

Regarding this and previous discussions on this topic on HN, it seems to me that one of the primary motivating factors when constructing a new ranking system should be the possibility of cyclical dominance a la rock/paper/scissors, which we should expect to see in many modern complex games. If one wanted to solve that problem, I think it would require some form of multidimensional ranking. Then one would have added flexibility so you could accurate predict winners from the scoring with some function f(x_1,x_2) even in cyclical cases. This would be less interpret-able, but it would have a fair shot at actually modelling/predicting/depicting the dominance relationships between players.

Investors that investigate companies, identify hidden problems and report on them to the public help enable more accurate price discovery. This wouldn't happen at all if they couldn't trade against it.

For instance, I recall a story where a hedge fund sent PIs to determine that a factory reported running was actually closed down. They can't just trade against that info - it also has to be made public for them to realize gains. I can't imagine how you'd consider activity like this "getting away with it", when the alternative is companies performing unscrupulous acts in secret.

Cash is cheaper for merchants, but people have only continued to use credit cards more. Further, if crypto becomes attractive from a cost perspective Visa will just be forced to bring down fees and stay competitive. It's pretty near impossible to imagine crypto with costs less than Visa's database.

This is secondary to there being no benefit to the average user for transacting in crypto vs Visa. What incentive is there for an end user to use even a hypothetically optimal crypto system? Then is that incentive sufficient to overcome network effects?

The appropriate and interesting applications of crypto seem to be totally leaving p2p cash behind - it just doesn't seem like a good fit.

Not to mention: > Data on blockchains is by default, unencrypted, especially data that needs to be validated by the nodes.

Databases are unsecured by default -> databases are overhyped!

It seems difficult to imagine any reasonable means to restrict communication like this, especially government imposed. Alternate possibilities:

1. People eventually tire of mass online communication and use it less. One can hope! 2. The continued abuse of public internet space by interested groups (political and commercial astroturfing, etc.) results in a drastic decline in the quality of public internet content (tragedy of the commons) and people migrate to more niche semi-private communities.

Modern discourse is clearly affected by the "leveling" of communication that a popular internet provides. People frequently interact with very large conglomerations of people, rather than primarily with their geographic neighbors. This analysis suggests to me an explanation for why massive internet inter-communication can be a bad thing for discourse.

And this matches intuition in the real world for different reasons - friend groups formed mostly by environmental and proximal circumstance result in more diverse yet still tightly-knitted communities than, say, a niche hobbyist subreddit might. So you have the opportunity to be exposed to new ideas by trusted members of your in-group.

You are of course free to try simple dimensionality reduction and nearest neighbors, and if that works on your problem that's fantastic. To the research community, though, problems where approaches like that work were considered "solved" decades ago. And of course, in industry, if there's a chance of that working it's tried. But no one's building self driving cars with PCA and LSH.