ergodic

If you are looking for a computational graph toolkit that is based on clean C++ go for Dynet.

It is very complete. Similar to pytorch (which borrowed some ideas from Dynet) and backed up by top academia (not big tech, hence its lower profile). It has a python wrapper as well, as everything nowadays.

If you are into deep learning for sequence to sequence (machine translation) Marian is also C++ centered. Fastest MT around. Complete (transformer and the like), extendable. Not that reading friendly from what I have heard.

Disclaimer: Python type of guy here.

IMO the economical argument is the most robust. I see it like this:

There is a big margin between

1) the amount of information that you are legally obligated to provide to public and private sectors

2) the amount information that can be mined about you through facebook and other services (+smart data-crunching)

Safeguarding that information that falls between 2 and 1 is not a matter of moral standpoints or having something to hide. It has only to do with a properly informed rational decision about your personal economy and your well being.

That information that you are willfully leaking will weaken your position in any future negotiation including, financial credit, insurance and employment among others. After Cambridge Analytica it is more clear than ever that it will also weaken your ability to protect yourself against well armed psychological manipulation.

and that is all assuming that public and private institutions do not make mistakes, that legal boundaries and procedures are flawless and that you leave in a well executed democracy.

I have not followed the news but was this really "considered the country's own answer to the iPhone"?. By whom exactly?

One thing is to say "A Spanish company's iPhone killer is a scam" an another "Spain's iPhone killer is a scam". I hope racism/cliches did not play a role here.

And I'm not sure that's a bad thing.

You surely can find other cases in which you end up paying more than the average. In the end, what matters is information asymmetry in any negotiation. In those terms, the consumer seems to be in an increasingly weaker position with the recent technological changes, social media, etc.

As a consumer I don't think that allowing market micro-segmentation is a sensible strategy. It's more something that I don't know very well how to stop, and it worries me.

Context: I lived in Lisbon four years and know one of the founders of Unbabel. I do not know you, neither your story.

Regarding your comments:

That's one thing I really hated about Portugal, they just don't offer any information in English

Thanks, but from the experience I had with both my employer and my landlord in Portugal I'd rather not do business with any Portuguese people ever again. Sorry.

it does seem like fucking people over is business as usual in Portugal

Have you had any experience with the Portuguese state or legal system? I'd probably still be standing in a queue somewhere.

Blaming the whole country for a 90 day experience does not seem very proportional to me.

Again, I do not know your story and can not speak for the veracity of your facts. What I do know is Lisbon (and I have lived many years in Spain and Germany to compare with). From that I know that you are just being, at best, exaggerated and I wonder if you have been exaggerating other stuff too.

nice!, does it have OCR?

Cool, you should try to sell this to community managers working on twitter.

no RSS?

It would be interesting to see if this is really so. The post just cites circumstantial evidence and does not reason why this would happen.

I personally hope the days of amateur, low frequency, niche blogging are not over. It seems to me as one of the fundamental goods that internet has brought and, as the post indicates, a solid reason to have RSS readers.

At the risk of being a bit annoying... regarding speech recognition, as said here

https://news.ycombinator.com/item?id=5376319

they use DNNs not DBNs (DBNs only used for pre-training, sometimes). Also if you read Microsoft's paper Table2, 7 hidden layer networks, which clearly qualifies as deep, work just fine with Back-propagation. Just a bigfat-MLP, no preprocessing! but 17.4 Word Error Rate (WER) vs 17.0 WER for DBN pre-training.

>they're not much like the old ANNs. DBNs are generative probabilistic graphical models using Bayesian inference.

MLPs (DNNs) can also be interpreted probabilistically. Just a directed model where inference is attained by marginalization of the hidden binary nodes in a layer-wise manner and by using a naive mean field approximation. All that to say the classic "forward-pass" ;).

Also could you indicate me a source confirming that Siri (Nuance) also switched to DNNs?. I am interested in that.

>Conceptually, going deeper (LOL) allows the networks to learn higher level concepts.

That is the really interesting part!. Now, I have not seen a proof for that. Wondering at individual neurons modeling individual features of e.g. a face or so is also a trend of the 90s and does not count as proof. I said this because it is what I usually hear.

Until now the justifications I saw for multiple layers of perceptrons being suitable for modeling arbitrary high level abstractions are reduced to

1) MLPs are universal approximators. This in my opinion is a superficial argument. GMMs also allow modeling "any" distribution and Taylor series any linear function, but in reality there are physical limitations to this argument. Maybe is true if you had a billion layer net, but will you get there?. If you had that computing power maybe a more realistic modeling of the brain might work better

2) They resemble how brain architecture works and similar arguments. Which I am fairly sure is not true. There are more human-brain based approaches to AI like e.g. cortical learning algorithms and those just seem to model that stuff to a certain extent.

I did not find that on the paper, are you referring to randomly switching off neurons?. I would be surprised if this would not be a technique of the original neural networks wave.

>DNNs can be thought of a stacked Restricted Boltzmann Machines

Agree, as explained in Hinton et al 2006.

http://www.cs.toronto.edu/~hinton/absps/ncfast.pdf

But this is just for pre-training, as I said. If you look at Seides paper, they pre-train treating the MLP as a DBN and then they train it as a classic MLP with BP. Also using layer-wise BP pre-training does bring performance close to DBN pre-training, with no use of DBNs paradigms at all.

>Their structure and training is very different to traditional MLPs

I insist if we are talking of the same DNNs explained in Microsofts paper, this is not true. If we were to be talking about different DNNs please elaborate I would love to hear about that (seriously, no irony here).

Again, the use of senones instead of monophones or diphones is just changing the output targets is not a novelty per sé.

rather than being so aggressive and waiting for downvotes you could choose to be constructive ;) and answer a concise technical question to a sub-thread that you yourself started

https://news.ycombinator.com/item?id=5377101

I read it in diagonal but the paper seems to use the same DNN architecture as before. They seem to tweak the pretraining with layer-wise back-propagation (instead of full MLP-as-DBN pre-training). This does not imply anything new with respect to what I commented and the cited paper.

The only reference to differences I found is about differences between a DNN and a MaxEnt models, which is again not an argument for differences between DNNs and MLPs.

Could you point me to a concrete paragraph?, I would be happy to be mistaken in this regard.

Again, please, have a look at Seide et al 2011 before commenting. Besides that I am not complaining, just saying, wait a little more before you claim the breakthrough of the decade.

> it might be great to have a human readable and tweakable solution (assuming you have the resources) but for something like recognising handwritten digits from images, not so much.

Agree, but with black-box I meant not something that is opaque to my grand-mother but partially opaque to engineers that implement MLP machine learning applications and the tech-lead that takes the decisions. The thing is that even research people (or maybe specially them) tend to positively bias things they do not completely understand (so I think, maybe its just me ;)). That is what I meant with black-box delusion. As you say only time will tell.

Regarding DBNs, again, the case of ASR uses DNNs which is to say big-fat MLPs. The model is handled as a DBN only for pre-training, and layer-wise pre-training does a similar job anyway.

If it was not clear enough, "fancy stuff" and "not making a big difference" refers to Seide et al 2011 mentioned in the same paragraph. Table 2 is particularly revealing to this regard.

http://research.microsoft.com/apps/pubs/default.aspx?id=1531...

As I said I can only speak with more or less certainty regarding ASR. I am fairly sure that the success in ASR (with Google and MS embracing DNNs for ASR) contribute significantly to the mainstream impact of deep learning.

Well, it is definitely something but it being the "Breakthrough of the Decade" seems pretty unlikely to me (given my available evidence).

I do not know well other examples beyond case of Automatic Speech Recognition, but since this case caused a lot of noise, I bet it is responsible for a reasonable chunk of the Deep learning "buzz". Here is my take about this.

If you look at papers from Microsoft like Seide et al 2011 and similar papers the reported improvement against state of the art (up to 30%) is really impressive and seems solid. Now, the technique is more or less using a very big multi-layer perceptron (MLP), a technique already established two decades ago (or more). There is some fancy stuff like the deep belief network based initialization, but it does not make big differences. The core of the recipe itself is not very new. What has changed is the scale of data we have available and the size of the models that we can handle.

With this I am not implying that this is not a very interesting discovery. But it is important to bear in mind that the change in the amount of data could also make other 20 year old techniques interesting again. On the other hand, neural networks had a bad name in the last years for understandable reasons. They are a blackbox, or at least less transparent than the statistical methods. This makes them prone to cause the "black box delusion" effect. You hear a new algorithm is in town, it has fancy stuff like remotely resembling human thinking architectures or cool math but you can not completely grasp it guts, then "voila!" suddenly you are overestimating its relevance and scope of applicably. MLPs were hailed as "the" tool for machine learning already once, I think for these same reasons. For me the right position here is a prudent skepticism.

On the other hand, this should also push people to try new/old radical stuff since the rules of the game seem to be changing, it is not a moment to be conservative in ML research :).

Well... Suzuki did change the name of their "Suzuki Wanker" (Suzuki Pajero) in Spain. It depends...