HN user

data_scientist

247 karma

Mail : oao2005@gmail.com

I build machine learning stuff for fun & work. From Paris.

Posts7
Comments28
View on HN

It doesn't have a good accuracy. I have yet to see a real-life dataset where it's better than to just call LogisticRegressionCV from scikit-learn. For bigger datasets, you may use vowpal wabbit or Fasttext. It may be a little bit slower for the training (but not so much), and as fast as LR for the training. What is the purpose of using an algorithm when another one is just better?

Does anyone have a good book for advanced linear algebra? I learned the basics in school - eg. Matrix diagonalization, svd) but I need for ml to really understand things like angle between flats, tensor decomposition, etc

SEEKING WORK / REMOTE or Paris France

I'm a data scientist with experience in various types of machine learning projects. I have a statistical background, and strong SWE skills.

I have experience in structured data, NLP, images, deep learning, recommendation systems, predictive maintenance, etc.

Why should you hire me? - I can understand your problem, torture your data and find the right model - or I will try something new, out of the box, if you need it - I know how to work with best practices for the code and how to integrate into a workflow - I love to deliver quality work

Drop me a mail! matthieu@databiz.io

My main issue with R^2 is that it is an artificial indicator. It has no business meaning. The best alternative is to build a metric that is related to your business issue. How many $ do you loss if you're wrong by 1%? Or by 10 units? Do your business loss is linear in your error? Is it symmetrical?

These are the kind of questions you have to ask yourself. Defining a metric is hard, and there is no good shortcut.

Great idea! Some questions:

- Both R and Python support strings, factors, and complex objects in a dataframe. What is NOT supported by feather?

- Feather is "not for long term data storage". Will it be standardize in a distant future?

- Do you plan to integrate it into Pandas?

Genuine question, why do you find Pymc3 better than stan? Pymc3 is still in alpha, while Stan is stable since many years. They both implements the same algorithm (NUTS), so is it just the nicer syntax or is it anything else?

Please don't make it configurable, because it will mean there is no longer "a" YAPF standard. Also look at Pycharm reformat tool, it would be a shame if both couldn't be used in the same versionned project.

Minecraft online also have all the properties you cite, and there are even more options, collectiond, and human interactions. Think about it: the other players can be adversary or allies, and that can change whenever you want.

This is the bad part of HFT, the one that is theft and destroy value. It is paid by the big players, and "solved" by imperfect solution like dark pools. A better solution could be to add a hour component to the order, so all commands from an actor in all markets are executed at the exact same time.

This rotted apple should not hide the good part of HFT, which is to reduce spread and inconsistencies between markets and to generate profits from this (positive) action. HFT took the place of traders, who were paid a lot for doing that stupid task.

I tried to do a fair comparaison between comparaison between the main date implementations. The ciso8601 is really fast, 3.73 µs on my computer (MBA 2013). aniso8601, iso8601, isolate and arrow are all between 45 and 100µs. The dateutil parser is the slowest (150 µs).

  >>> ds = u'2014-01-09T21:48:00.921000+05:30'

  >>> %timeit ciso8601.parse_datetime(ds)
  100000 loops, best of 3: 3.73 µs per loop

  >>> %timeit dateutil.parser.parse(ds)
  10000 loops, best of 3: 157 µs per loop
A regex[1] can be fast, but the parsing is just a small part of the time spent.
  >>> %timeit regex_parse_datetime(ds)
  100000 loops, best of 3: 13 µs per loop

  >>> %timeit match = iso_regex.match(s)
  100000 loops, best of 3: 2.18 µs per loop
Pandas is also slow. However it is the fastest for a list of dates, just 0.43µs per date!!
  >>> %timeit pd.to_datetime(ds)
  10000 loops, best of 3: 47.9 µs per loop

  >>> l = [u'2014-01-09T21:{}:{}.921000+05:30'.format(
        ("0"+str(i%60))[-2:], ("0"+str(int(i/60)))[-2:]) 
     for i in xrange(1000)] #1000 differents dates
 
  >>> len(set(l)), len(l)
  (1000, 1000)

  >>> %timeit pd.to_datetime(l)
  1000 loops, best of 3: 437 µs per loop
NB: pandas is however very slow in ill-formed dates, like u'2014-01-09T21:00:0.921000+05:30' (just one figure for the second) (230 µs, no speedup by vectorization).

So if you care about speed and your dates are well formatted, make a vector of dates and use pandas. If you can't use it, go for ciso8601. For thomas-st: it may be possible to speed-up parsing of list of dates like Pandas do. Another nice feature would be caching.

[1] http://pastebin.com/ppJ4dzBP

[dead] 12 years ago

It is an interesting trick. Will I have to pay the school if I leave it early, and to reimburse any grants that I may have gotten? Do you know anyone who had done that?

[dead] 12 years ago

I am in contact with some of them. It is better than nothing but still very hard just to have a Skype interview.

[dead] 12 years ago

I don't want to go back to school now, I am tired of that. How about working for a US company outside of the USA, before to move within the company ?