yazr

Paywall copy from reddit

https://old.reddit.com/r/MachineLearning/comments/khin4c/n_m...

Many had the same thought after AlphaZero i.e. some sort of expert-iteration for any (formal/combinatorial) domain what can be tree-searched

But it turns out, that interesting problems have existing refined algorithms that are difficult to beat

I agree that (DL + something) is the next step

Do you lean more towards symbolic search + neural heuristic or neural representation of symbolic domain ?

Should this be modifiable for higher-level byte-code?

My thinking is to provide hand-crafted primitives to transform byte code, and maybe use an existing optimizer to do the search.

Love to chat. Sent u an email (former HFT and kernel guy).

One topic which is feel is neglected is a good GCN (or any GNN) to operate on existing code trees. Most approaches seem to prefer seq or at most tree inputs.

Is this simply not finding yet a good network architecture, or is it a performance issue ?

What do you think is a more productive path leading to "AutoCode" ?!

A. Add external definitions or reward formalism to make the code-space easier to search?

OR

B. Keep adding code trees, execution traces, comments, memory dumps and learn from those?

My own instinct is that AlphaZero was a lot more convincing than AlphaStar, so lots of (A) is definitely needed

As a non-US person, is there anyway to bill using Stripe without incorporating? I dont mind withholding tax, etc, i just want to be able to bill and track the initial revenues.

AFAIK, Stripe is not available in all-EU countries. Paypal just works in this case.

My only nitpick is that AG (ML + search really) improved logarithmic-ally with exponential compute.

In addition, real world domains may not be some parallizable.

So yes, current AI is like a brilliant intern. It will eventually be your manager, but its gonna take a few more years ?

Is there some sort of group-defense, where several sue-ees group together to contest a ridiculous patent ?

I guess the main difficulty is not the legal framework, but the business framing

Great news.

I wonder if already there will be a selection bias to recession-friendly and revenue-producing startups

IANAVC (not a vc), but i think do this is doubling down on risk.

If you take seed from VC, then: 1. if you are mediocre then things become very difficult for you 2. if you are successful, the VC can discount the risk premium, and push u into an even higher valuation

A fully illustrated article [1]

And Lilian Weng blog on self-supervision [2]

.. CPC is .. translating a generative modeling problem to a classification problem... uses cross-entropy loss to measure how well the model can classify the “future” representation amongst a set of unrelated “negative” samples...

[1] https://ankeshanand.com/blog/2020/01/26/contrative-self-supe...

[2] https://lilianweng.github.io/lil-log/2019/11/10/self-supervi...

Over their lifetimes, red supergiants like Betelgeuse create and eject vast amounts of material

What % of material is then captured by other forming/formed stars? Doesnt it just end up in interstellar space ?

As someone patiently explained to me 2 yrs ago...

For the ATARI, the "real world" is the present frame, and a fixed set of 4 buttons and 4 directions. This of course is the game pre-programmed into the ALE ROM.

You can take any action, and get the next frame. but you cant "undo" an action, and you cant restart a game from a fixed state (see the Go-Explore controversy). And you cant explore 4 different actions in an interesting frame.

So now, if you learn a network which predicts the next frame, you can enter the world of model-based learning, where we do a simulated move tree roll-out (i.e. not calling the ATARI), try a gazillions moves, and only then select an action and get the next sample.

In a formally defined synthetic domain such as chess or logic programming, it is not clear whether this is helpful. We are simply trading one cpu time (calling the environment) for other cpu time (running our own learned im-precise model of the environment)

Of course DM has a chess function which does codes the rules of the next move. It can return a LOSS if you try an illegal move. But this function is NOT called for the tree roll out.

but I am confused by its claim ... without knowing the game rules.

probably some similar hard-coding of Atari actions

Nope, no hard coding.

Consider trying to MCTS on an Atari game. You have to "learn to predict" the <next frame, action> pairs. Initially this guess is very bad, but eventually your predictions are good enough that rolling out a tree of predictions improves your action selection

For Go, and chess, we twist our self into NOT using the game rules in the simulator e.g. for each move, just indicate if GAME LOSS WIN

Whether this paper worthy of a new Nature hype cycle is a separate debate

This is a recent presentation, apparently for Indonesia. I find it detailed and well presented.

I do not really hold much hope for new nuclear, as compared to renewables.

However, they do still have a case in less-sunny, dense urban areas (e.g. US North East, Japan).

The robo-taxi connotation is even worse.

Any robo-taxi can be programmed to drive like a grand-ma to improve battery life. The cost of a x2 battery size is easier to amortize. Its life time can be managed and optimized with ML.

And yes - it can get the greatest and latest battery chemistry.

At this stage - we just dont need a "battery miracle" or a Musk-moonshot (trade-mark?) for passenger EVs.

The GP is correct and not just being pedantic. This is just click bait.

A battery (life)range is degraded (mostly) by (dis)charging speed, average discharge level and ambient heat.

The bigger the battery, the less strain on each cell, and hence longer life time. I hope tesla is improving its battery, but this title is really bad.

Does anyone have a graph of recent $/transistor progress?

Its obvious that moore's law is stagnant, and clock speed is dead, etc, etc.

But are we at least getting the same core size at a lower cost ?

The main comment here seems to be sluggish performance.

Is it some sort O(n^2) rule handler deep inside the builder? Or is it just "python-slow" ? :/

Can someone give an example of a big open source project which is too slow to build?

Ireland has ~10GW generation capacity (mostly coal & extra dirty peat).

At around $1/W for new solar and wind generation, the Apple back taxes owned to Ireland can convert the entire Island to renewable energy. It will be an asset that generates income for 30 years.

(yes, ok, more storage needed, and interconnect to the UK, and keep some gas as backup ..)

DOes this 2500-cpu-hr cover the ENTIRE learning process?

Lets even your first run is crap, and you try again with RGB instead of YUV or whatever. So you do 4 runs.

So 10000-cpu-hours replace a week of work of a qualified ML engineer. This is pretty amazing. If i understand correctly.

Can u explain ?

Do u mean Euro Paypal accounts cant be just frozen unilaterally etc ?

There is an upcoming Manning ebook for graph-machine learning. Its a lot less technical than the papers. Fingers crossed for them.

https://www.manning.com/books/graph-powered-machine-learning

Why Rust? (In general..)

Are C memory-related bugs still such a real problem?

IIUC, Rust can prevent these kinds of memory vulnerabilities. But it cant really present the million of other types of vulnerabilities, such as mis-configuration, wrong logic in the code, races, etc

Am i understanding this correctly?

Tinyseed , Indie.vc, Lighter Capital, Earnest Capital, OnDeck, Kabbage, MainVest, Point9

I guess its not easy to qualify since they get a bazillion application.

Do you just apply on the web ?

Do u have good references ?

Is it "real revenue" or r u counting the nominal sale value ?

Also - if u r still on WP then 30% may not be too bad :/

This. ML is so vector-y. And code is so graph-y. Can you point out some SOTA on bridging this ?

Not for PhD, but for research purposes ;)

Can u point out any work on refactoring existing code for reduced code complexity?

Obtaining same (non)functional behavior using less code.

EDIT: i will read the CodeDeviant paper

If you need to set your mentor performance-metrics, u should probably find yourself someone else (I am not the down-voter). Also, 1% for later-stage? post-VC? sounds very high.

I like this article, which describes 3 different scenarios

https://www.inc.com/john-rampton/how-much-equity-should-you-...

Can you please explain why "new schedules do take some bedding" ?

Cant u sort of simulate the expected scenarios and get sort-of expected performance?

I realize there is unexpected events, but these will occur anyway in old and new schedules

Also - does real train software has nice graphics or do u get a boring excel ? ;(

I understand that a JIT engine cant do this, due to speed.

But would be surprised that a LLVM compiler with a custom-designed heuristic for specific optimizations cant manage this.

These problems may be polynomial etc etc, but MOST of the code generated should follow common patterns

I will have to give this some thought.

Thanks for the reply (also GP)