twiecki

There is a collection of curated resources here: https://www.pymc.io/projects/docs/en/stable/learn.html

If you are looking for an explanation of MCMC that focuses on intuitive understanding to complement more mathematical introductions, I wrote a blog post trying to explain things in simple terms here: https://twiecki.io/blog/2015/11/10/mcmc-sampling/

No, back-propagation would not give full Bayesian inference (although there are some tricks [0]). They instead use variational inference[1], which allows for fast inference of continuous PGMs.

[0] http://mlg.eng.cam.ac.uk/yarin/blog_2248.html

[1] https://arxiv.org/abs/1603.00788

Brian Moriarty gave a great talk in 2015 about his story of Loom with many interesting behind-the-scenes infos: https://www.youtube.com/watch?v=z1aVDael-KM Towards the end he also says he'd be excited to do a Loom sequel with the right studio.

There are packages that are just Python libraries, like PyMC3: http://pymc-devs.github.io/pymc3/ It allows you to specify a Probabilistic Program with a few lines of Python code (x ~ Normal(0, 1) becomes x = pymc3.Normal('x', 0, 1)) and then run automatic inference on whatever model you dream up, it's very powerful. I always found the approach of inventing a new language to specify Probabilistic Programs like Stan does a bit odd. There, you specify the model as a string and can't interact with it or debug if something goes wrong. Disclaimer: I'm a PyMC3 developer.

I don't think it has to be either or. For example, you can implement a Deep Net in a probabilistic programming framework (http://twiecki.github.io/blog/2016/07/05/bayesian-deep-learn...) like PyMC3. The inference here is not using sampling but rather ADVI (http://pymc-devs.github.io/pymc3/api.html#advi) which is almost as general but much much faster, and can be run on sub-samples of the data (similar to stochastic gradient descent used in deep learning). Once we bridge these two domains we can get the best of both worlds, like a deep net HMM.

PyMC3 (http://pymc-devs.github.io/pymc3/) has all the powerful samplers that Stan has (i.e. NUTS) as well as support for discrete priors. It allows you to specify your models in pure Python, supports matrix algebra, and also has variational inference which allows for large-ish scale machine learning (e.g. here is a blog post where I train a Bayesian Neural Net on MNIST http://twiecki.github.io/blog/2016/07/05/bayesian-deep-learn...). Under the hood, it uses theano for JIT compilation to C or GPU.

I'm one of the core devs.

Thanks for your comment! I did split the data in two, but it's easy to miss. X_test and X_train are the two sets. ann_input.set_value(X_test) then switches in the test values. That's identical to running make_moons() again.

If you're interested: http://pymc-devs.github.io/pymc3/

PyMC3 uses Theano to create a compute graph of the model which then gets compiled to C. Moreover, it gives us the gradient for free so that HMC and NUTS can be used which work models of high complexity.

I use it in production, despite it still being beta. We're close to the first stable release but there are still some small kinks to figure out.

Disclaimer: I'm a co-developer.

Anyone else notice that the code wasn't PEP8 compatible? ;)

I went ahead and fixed it: https://gist.github.com/twiecki/7361de09b8e4c0a6248e

The book is based on PyMC2, a Python module for Bayesian inference: http://pymc-devs.github.io/pymc/

PyMC3 is nearing the first beta release and is a complete rewrite that's using Theano to add just-in-time-compilation as well as support for next-generation sampling methods like Hamiltonian Monte Carlo (e.g. NUTS): http://pymc-devs.github.io/pymc3/

There is also the blue card (http://www.bluecard-eu.de/) which is pretty easy and quick to get if you have a university degree and a job-offer.

Yes, yahoo finance prices can be quite messy and sometimes even wrong for the reasons you mention. I wanted the code to be portable so that's why I used them.

We have the same strategy on Quantopian though (https://www.quantopian.com/posts/olmar-implementation-fixed-...) where we have 10 years of minutely data, adjusted for splits and dividends. It seems to do quite well.

Also, this is not your standard dual moving average cross-over strategy. Rather, it rebalances its portfolio (optimally) under the assumption that the stocks will revert to their moving average. The eps parameter, which I explored in this talk, influences how aggressively the algo rebalances its portfolio -- smaller values will result in a more stable, conservative strategy.

Certainly that's always a risk. Zipline does however support simulating transaction-costs and slippage so those will be accounted for (as best as possible).

As for market-paradigm shifts, I totally agree. One way to deal with that is to constantly re-optimize the parameters based on recent data. This is also known as walk-forward optimization on which there is a slide at the very end (see http://blog.quantopian.com/parameter-optimization/ for a more information).

Finally, as to running multiple strategies with different parameter settings. The OLMAR paper that describes the algorithm (http://arxiv.org/abs/1206.4626) has a variation of this where they use a range of different-length moving averages, rather than just one.

That's currently not implemented, but definitely something we would like include. What I'd really like to see is a client for a bitcoin exchange. See our contribution request page for more details: https://github.com/quantopian/zipline/wiki/Contribution-Requ...

Finally, live trading is something which we are working on to enable on Quantopian (which uses zipline for backtesting your algorithms). The syntax is very similar so you could just convert your zipline strategy to Quantopian (which is free to use).

Yeah, the code is cut-off unfortunately. It's the new IPython nbconverter that still has some kinks. The idea of the algo is expressed on the next slide.

You can, however, view the full IPython NB here: http://nbviewer.ipython.org/urls/raw.github.com/twiecki/zipl...