HN user

BiasRegularizer

200 karma

PhD student in Machine Learning, primarily focused on multimodal representation learning, contrastive learning, and robotic applications

Posts1
Comments60
View on HN

Distribution shift in the real world data will always be inherent to any data driven methods. Unless there are major advances in continual learning for DL models, they will always struggle with distribution shift degradation.

Similarly, humans are also prone to the distribution shift unless we get updated information on a specific topic. The key differences are that we are great at continual learning and we are much better at learning abstraction

A 17 million parameter model (~Resnet50) takes more than 50s proof time. Is this on top of the inference time?

I can see some niche applications for this system, but I am very skeptical it's ability to handle larger models (100M+) and the ability to and it's scalability when there are increased demand.

It's not impossible to see that LLMs can make finite step "reasoning" between input and output, as each block of the transformer can model probabilistic causality. Transformers are sometimes considered a fully connected graph neural network, which can be used for modeling causal graphs. One additional supporting evidence on the finite step reasoning hypothesis is that "train of thought" improves the performance of LLMs[1], meaning that letting the model explain itself explicitly reduces the amount of implicit "reasoning" steps that need to happen within the model.

Additionally, while we don't know how us humans functionally reason, it's believed that the predictive nature of our brain is central to our reasoning abilities. Maybe in some way, the autoregressive nature of LLMs is similar to our predictive brain.

For a collection of emergent properties in LLMs, I recommend this paper [2]

[1]https://arxiv.org/abs/2201.11903.pdf [2]https://arxiv.org/pdf/2206.07682.pdf

Apple Watch has so many health features, but the main hinderance for me is its relatively short battery life - I often forget to put it back on after charging it.

Has anyone tried having two apple watches so one can be worn while the other charges?

[edit] Charging it at night is kind of missing the point, as I would like to track both my daily activities and my sleep

VPN might have a part in this too. I remember when I was in China, I was connected to ExpressVPN 24/7 (whatsapp, gmail etc), and the battery drain was visibly higher than when I had the VPN off.

While this may be amongst the long term goals, I don't think it's useful for military applications yet until laser com is fully online. Currently there is no sat to sat com, so there's is always ground station within sight of a sat. Unless SpaceX places a ground station near a conflict zone, military can't utilize it much yet.

You didn't quite get the point. As individual currencies governments exert a lot of power over their relative value to other currencies. As a whole, worlds fiat currencies hold no intrinsic value. No government can dictate that a kilogram of gold to only cost $1.

I suggest you look into modern economic theories and why inflation is crucial to economic development. There are two problematic assumptions in your statement:

1. Nobody would store all their wealth in a deflationary currency - other forms of assets, stocks etc will still exist. They will need a high liquidity vehicle to trade a.k.a inflationary currency.

2. Deflationary currency doesn't necessarily mean it will gain value overtime. Unless it contains growing intrinsic value, the overall market cap will drop/plateau.

Think of inflation vs deflation as lubricant vs sandpaper, one encourages economic activities while the other discourages.

Yes, but one would prefer spending money using an inflationary currency since holding on to it devalues over time. As long as inflationary currencies exist, people will prefer to spending those first before deflationary currencies.

I believe this applies to all currencies - they don't have intrinsic values. The values are derived from the market. Be it USD, BTC, EUR, or the Zimbabwean dollar. However, the purpose of currency exists as a vehicle for wealth transfer.

IMO the biggest problem with most cryptocurrencies is their deflationary nature. Limited supply means they behave more like assets than currency. As currency nobody would want to spend it as they gain value over time. As an asset it lacks intrinsic value, and therefore overall market cap will tank slowly.

I believe he's referring to the absurdity of this statement, especially in relation to SpinningUp (it doesn't have any UI as it's a RL library, and the docs site are generated using standard Sphinx doc generator)

One can argue modern definition of racism has expanded much beyond race. Though I also didn't want to start a pedantic argument on the definition of racism with the original OP. Rather to point out how distastefully the Chinese Gov have acted in response to Trump and Pompeo.

Both are very classic traits of authoritarian gov: In times of crisis, it's far more efficient than democratic governments. However, in times of non-crisis, its centralized power and information control would cause a crisis that would've been otherwise avoided.

Mistakes can happen but wittingly suppressing news is not accidental nor a mistake.

I was born in Wuhan and half of my family are currently holed up there. A few days before the Wuhan lockdown was ordered, coronavirus already made it to the headline in CNN and NYT, but there was not any headline news about the coronavirus within China. Heck my dad wasn't even wearing a mask on his 1hr metro commute to work until the day before lockdown happened.

In early January, government officials and news outlets were reporting: "There are false rumors about Wuhan having a SARS like virus. This is an unknown virus that's _probably not_ transmittable from human to human. There are only 45 confirmed cases. There is no need to panic. "

One of the officials even organized a thousand people CNY dinner as part of the CNY celebration a few days before the lockdown.

They had a thousand opportunities to curb the disease and alert the people, but the Chinese bureaucracy and leadership's imcompetence directly caused the coronavirus disaster.

Edit: January 21 news article on the thousand people dinner from a Chinese source. About 40k people participated, this neighbourhood is about 10 minutes away from the epicenter: https://news.163.com/20/0121/15/F3E3UTKI0001899N.html