GChevalier

Another option is to create functions in the bashrc, they can contain logic like the suggested bin approach.

This is amazing. I want this.

I admit I chuckled at "You are not entitled to this explanation" while reading the article. How far can it go xD

Please someone do a PyCharm plugin for this!

You make me think of Azure ML studio.

"The relationship of spike-time-dependent plasticity to other models requires further clarifying work"

- I got something for that, in fact I think I discovered that backpropagation engenders STDP:

https://github.com/guillaume-chevalier/Spiking-Neural-Networ...

"The relationship of spike-time-dependent plasticity to other models requires further clarifying work"

- I got something for that, in fact I think I discovered that backpropagation engenders STDP:

https://github.com/guillaume-chevalier/Spiking-Neural-Networ...

Thanks for pointing out to "phase unwrapping", I didn't know it had a name! Interesting.

Also, I forgot: to improve convergence, I'd use an Hann-Poisson window such as here: https://en.wikipedia.org/wiki/Window_function#Hann%E2%80%93P...

I'd apply the window to randomly-sampled mini-batches of consecutive points instead of optimizing the neural network on just randomly-sampled batch points or on all the dataset at once. I guess that using an Hann-Poisson window will make the "gradient" valley easier to "ski down" with gradient descent which is a greedy algorithm. I guess that the spectral leakage caused by the Hann-Poisson window function will make the gradient landscape more monotonically decreasing in every point towards the global minima.

I would have instead done like this:

(tldr; use a sine and cosine function regression like a linear regression. Think like solving for a free angle and a free phase instead than for a free bias and weight).

1. Convert the hours to an angle in degrees or in radians (a simple linear transformation).

2. Take the cos and sin of the angle to get the x and y position in a plane, respectively.

3. Introduce a time axis such that the thing doesn't draw a circle but rather an helix (like DNA).

4. So we now have a ton of 3D data points: (time, x, y). Create a ML model to fit a sine and a cosine to those data points to match them perfectly. Your model has only 2 free parameters to optimize for: a shared phase offset and a shared frequency. The sine uses (time, y) and the cosine (time, x).

5. Initialize the model with a random phase offset and a frequency ideally already close to the one you think you have. Don't initialize with a too high frequency to avoid fitting just Nyquist-frequency-close-noise.

6. Optimize! (With the least squares.) I guess that you might congerge only to a local minima and need to try different randon starting frequencies if you fail to converge.

7. The answer to your problem is the now-optimized free parameter of the frequency. It won't sit between two bins of your fft anymore.

Related: https://stackoverflow.com/questions/16716302/how-do-i-fit-a-...

Note: This link contains images picturing the transformations I try to explain.

Disclaimer: I didn't do that yet, this is just off the top of my head. If I said something wrong, please comment. Mostly about a wrong convergence to Nyquist freq or something like that (?).

In the end, this way, you won't have discrete fft bins. You'll approach the problem orthogonally to that: you solve for finding the one best fft bin (frequency) directly.

In other words: solve for the content in the exponential of "e" as free parameters, and for one such frequency and phase offset instead of many bins.

Also, you should be able to use your favorite editor for the code outside notebooks (over time, more and more of the code will be outside of your notebook). You might often work in the editor, and at other times in the notebook depending on the nature of the work. As the project advances, notebooks will become less and less important, they only kickstart projects.

Never tried macOS, but matplotlib works fine under Windows and Linux. Maybe you could save plots to images on disks and prevent them to show? I once ran code that used PLT on a server and I needed to use something like `matplotlib.use('Agg')` to prevent the code from crashing because of lacking graphical output.

See this SO answer: https://stackoverflow.com/a/34583288/2476920

Personally, I love to have notebook cells to be able to code without re-running everything. Especially in the case of deep learning, training a model is long. Jupyter is very good for creating and debugging code that: A) needs a trained model loaded for it to work but you want to skip the part of saving/loading the model, or B) code that saves-then-load a model.

If the "mutable state with global variables" drives you crazy, you may want to avoid reusing the same variable names from one cell to another, and reset the notebook more often. Also, avoid side effects (such as writing/loading from disks) and try to have what's called pure functions (e.g.: avoid singletons or service locators, pass references instead). If your code is clean and does not do too much side effects, you should be able to work fine in notebooks without having headaches.

EDIT: typo.

Author here.

Whoa, thank you all for the nice comments, I didn't expect to make such a buzz here nor today. I'm glad to see the reactions - even the bad ones, it seems aligned with what I thought. Yes, notebooks are very useful for the faster coding cycle, but they become easily heavy (I'd love to see a better multiline edit and a better autocompletion in Jupiter).

Seems like I already posted my article 2 months ago, but renamed the GitHub repo since then, which may explain why someone else (jedwhite) could submit my article again: https://news.ycombinator.com/item?id=18339703

I didn't submit it twice to HN. Well, nice to see that in a parallel world my post did the 1st page on HN! :-)

But could this mean that my HN account is like "shadow banned" or something? Strange to see that all my own submissions on HN haven't got much attention for months. Or maybe it's just the random factor... Well, thanks!

https://news.ycombinator.com/item?id=13049143

How can Google possibly predict activity type between IN_TWO_WHEELER_VEHICLE vs IN_FOUR_WHEELER_VEHICLE ?!

Could they be using this from accelerometer data instead of GPS? https://github.com/guillaume-chevalier/LSTM-Human-Activity-R...

For maths and machine learning, Khan Academy and Coursera can get you somewhere.

Advertisement at it's finest: I was suggested to buy an oscilloscope on that page

Perhaps they simply used a U-Net?

I submitted it to HN too, why not! Here: https://news.ycombinator.com/item?id=18339703

I recently wrote on how to build/grow clean software out of Jupyter notebooks and on pitfalls to avoid when coding like that: https://github.com/guillaume-chevalier/How-to-Grow-Neat-Soft...

I see here that original poster (OP) of the post tried to use many-to-one LSTMs instead of many-to-many LSTMs. I tell that first by looking at the charts. Then I saw the method named "predict_point_by_point" with the comment "Predict each timestep given the last sequence of true data, in effect only predicting 1 step ahead each time" in his code here: https://github.com/jaungiers/LSTM-Neural-Network-for-Time-Se...

I strongly think the system would be better to perform many predictions at once instead, using seq2seq neural networks. The problem is properly explained here at the beginning of this other post: https://github.com/LukeTonin/keras-seq-2-seq-signal-predicti... This other post is, in turn, derived from my original project here doing seq2seq predictions with TensorFlow: https://github.com/guillaume-chevalier/seq2seq-signal-predic...

OP also forgot to cite the image I made: https://en.wikipedia.org/wiki/Long_short-term_memory#/media/...

Well, glad to see that some similar work as mine can get this much traction on HN. I would have loved to get this much traction when I did my post, too. Anyway, I would suggest OP to take a look at seq2seq, as it objectively performs better (and without the "laggy drift" visual effect observed as in OP's figure named "S&P500 multi-sequence prediction").

In other words, using many-to-one neural architectures creates some kind of feedback which doesn't happen with seq2seq which doesn't build on its own accumulated error. It has a decoder with different weights than the encoder, and can be deep (stacked).

So many ads and things that prompt for attention on this page! I needed to press "x" on 3 things to be able to read. And once read to the end, it refers to a "video above" that won't open under Chrome for mobile. Wired, calm down on crap!

No GitHub?

Seems promising and easy to use in practice.

I wonder if it can be applied a 3rd meta-time. E.g.: an RNN would edit the weights of another one which edits the weights of the main one. And maybe add the L2L meta optimizer in that?

I am co-writing a paper with a Ph.D. student and he is currently working on trying with other datasets. We are also trying different architectures, combining multiple LSTMs (stacked, residual connections + batch normalization, bidirectional LSTMs, and on)

I also tried my code without the gyro, and incredibly, the accuracy is still pretty high (86.90% rather than 90.77%).