Silicon Valley Hedge Fund Takes on Wall Street with AI Trader

https://www.bloomberg.com/news/articles/2017-02-06/silicon-valley-hedge-fund-takes-on-wall-street-with-ai-trader

by bunburying • 9 years ago

191 109 9 years ago

JTon• 9 years ago

This is the most interesting bit for me:

Sentient's system is inspired by evolution. According to patents, Sentient has thousands of machines running simultaneously around the world, algorithmically creating what are essentially trillions of virtual traders that it calls "genes." These genes are tested by giving them hypothetical sums of money to trade in simulated situations created from historical data. The genes that are unsuccessful die off, while those that make money are spliced together with others to create the next generation. Thanks to increases in computing power, Sentient can squeeze 1,800 simulated trading days into a few minutes

alva• 9 years ago

This is the most interesting bit for me :

It shares little about the data used for the AI's decision-making and isn't profitable

rayuela• 9 years ago

Yup. This is a bullshit PR piece.

rbinv• 9 years ago

So, genetic algorithms? Good luck with that.

sixtypoundhound• 9 years ago

I recall reading an early article about using genetic algorithms for stock trading in the early 90's. This isn't really that new....

vijucat• 9 years ago

Genetic Programming, rather. See the papers authored by their Co-founder and Chief Scientist:

https://scholar.google.com.hk/scholar?q=Babak+Hodjat+genetic...

jcfrei• 9 years ago

I believe anyone who has considered applying machine learning tools for identifying trading signals came across the idea of using some kind of optimization algorithm to find the best parameters. Evolutionary algorithms seem like a fair approach because the parameter space is (for lack of a better word) very sparse and anything but continuous (lots of parameter combinations don't make sense / don't work). Using a genetic algorithm - like for example the differential evolution algorithm by Price, Storn and Lampinen - is really the most naive approach. I assume they must have some pretty clever, human made models in the back - because in a very general model the parameter space would be so vast, they would probably never find good parameters. And even if they did, they would have to run thousands of tests to make sure they are not overfitting the data - making this a very slow optimization. Not saying this will or will not work - I'm just thinking out loud here.

uchish• 9 years ago

It's not slow if you have enough compute

scottlegrand2• 9 years ago

So basically genetic programming? They didn't invent that, John Koza did...

growt• 9 years ago

I just started reading 0 to 1 and I think according to Peter Thiel this news sums up everything that is wrong with the world in one sentence :)

fahadkhan• 9 years ago

care to elaborate?

pdog• 9 years ago

Thanks to increases in computing power, Sentient can squeeze 1,800 simulated trading days into a few minutes.

How is it possible to generate seven years of convincing sample data from historical trading data without overfitting your models?

problems• 9 years ago

Probably by testing against other data to make sure you're not overfit. Though it looks like they're not profitable currently, so... maybe they just don't.

lutusp2• 9 years ago

How is it possible to generate seven years of convincing sample data from historical trading data without overfitting your models?

Simple -- create a large set of models, each using different selected parameters, run them against the market, and pick the outcomes that make the scheme look good. It's called "data mining."

A famous "psychic," who shall go nameless, made a career of appearing with a sealed, dated, registered letter, opening it, and proving that she had correctly predicted the outcome of an election / horse race / other event in advance. She was always right, and the registered letters were real and were mailed before the event to be predicted. How did she do it? She mailed herself more than one registered letter for each event. People are sooo ... credulous.

BickNowstrom2• 9 years ago

How is it possible to generate seven years of convincing sample data

From the article:

These genes are tested by giving them hypothetical sums of money to trade in simulated situations created from historical data.

So they simply use historical data (going 7 years back). This is fed into a simulator (a trading day where you have access to the data of all days leading up to the simulated trading day).

without overfitting your models?

Traditionally backtesting is used: http://www.investopedia.com/terms/b/backtesting.asp

Care should be taken to run the tests over significant periods of time (to reflect changing market conditions) and to have a final out-of-time holdout set to lessen the effects of picking "winners" that were winning purely due to chance (introduced by cherry-picking winners from thousands of models).

antr• 9 years ago

PR bluff.

I find the Renaissance Technologies story more compelling and worthy than a press release drafted by the VCs funding/cheering Sentient (Kleiner Perkins, Tata, Horizon, etc.)

perlpimp• 9 years ago

fwiw http://www.forbes.com/sites/rickferri/2012/12/20/any-monkey-...

mjpuser• 9 years ago

I just checked out glassdoor, and it seems like they don't know what their product should be since top execs don't want to agree on anything.

We have too many executives for a company this size - most don't add much value except for fighting with each other and politicizing issues. The CEO lives in HongKong and remotely manages this crew of distrusting, non-supportive and close-minded execs. The HR function is practically a joke. You don't ask questions, challenge their decisions or speak up - they'll threaten to fire you if you did. They have not been able to productize their technology, their trading business has not picked up and their sales pipeline is pretty dry for the other businesses. They have laid off a large number of people recently and financial trouble seems to be brewing - lot of marketing smoke in here!
Focus on one product and give it more time.
Productize, productize, productize.

easytiger• 9 years ago

Sounds to me like a shopfront to build up an emotionally weighted patent Portfolio to then beat others around the head with.

Hope it fails

658272• 9 years ago

I mean sure, but take glassdoor with a huge grain of salt, people make up crap to post on there all the time.

drpgq• 9 years ago

The company I work for is around 100 people, but the few postings I have seen that are negative and descriptive are on the mark. Maybe Tolstoy was right about unhappy companies as well.

mifeng• 9 years ago

After a certain number of reviews, Glassdoor becomes a more honest reflection of a company's actual state than anything else.

It's certainly more truthful that marketing fluff pieces like this.

kybernetikos2• 9 years ago

I've also worked places that were pretty effective at getting glassdoor to take down the negative reviews. They could always find some sort of explanation for how the negative ones didn't meet the terms and conditions and glassdoor seemed relatively ok with taking them down.

reverend_gonzo2• 9 years ago

Which is really odd, since if they had any idea of what they were doing, they wouldn't need to sell. Trading is among one of the few industries out there where you don't need to have anything like a customer.

What you do is make money, and if you're making money, well, you don't need to ask anybody else for money. This tells me that its a bunch of executives who either don't know anything about trading and/or don't have any coherent strategy, and are really just trying to throw enough buzzwords out there to get investor capitol.

bogomipz3• 9 years ago

" Trading is among one of the few industries out there where you don't need to have anything like a customer."

Trading yes but a hedge fund or any fund really implies customers no?

edblarney3• 9 years ago

"Trading is among one of the few industries out there where you don't need to have anything like a customer."

Yes, but knowing how to trade is obviously very important, it's not just picking stocks, it's risk management, regulatory issues, and tons of industry specific knowledge.

That said - if they have 'tech that works' they should be able to partner with a few financial people and wrap a fund around it without too much trouble.

That so much negativity is on Glassdoor is really a bad sign.

lpage• 9 years ago

Peter Muller has the most succinct summary of this that I've seen in his (very worthwhile) opinion piece on proprietary trading [1]:

Sharpe Ratio, Marketing department

≤ 0: Runs the firm

0.25: Very important; involved in all investment decisions; major focus on asset gathering

0.5–1.0: Secondary

1.0–2.0: Almost superfluous

≥ 2.0: What marketing department?

[1] https://faculty.fuqua.duke.edu/~charvey/Teaching/BA453_2004/...

crdb• 9 years ago

if they had any idea of what they were doing, they wouldn't need to sell

Having spent some time in the industry, I disagree, you're always trying to maintain or increase AUM. Funds constantly sell to existing and future limited partners (investors). Internal prop trading desks sell to management in order to keep their allocated VaR and to risk for similar reasons. Traders themselves sell to their boss and peers. VCs sell their top quartile-ness. All of the above sell to their junior staff to keep them around.

The thing is you don't know if you are good or lucky until many years into your career and someone has to put up the capital until then. Even the top funds with great track records can one day stop producing results, at which point people abandon ship.

nerfhammer• 9 years ago

Trading is among one of the few industries out there where you don't need to have anything like a customer.

You can usually make a lot more money with Other People's Money than with in-house money.

robert_foss• 9 years ago

Taking on Wall Street by participating in Wall Street?

I somehow doubt how 'disruptive' this will be.

sunrisetofu• 9 years ago

There's this thing called data snooping bias, I hope these GA folks recognize that.

As trading goes, past performance does not equate future performance.

samfisher83• 9 years ago

Renaissance Technologies, DE Shaw, etc. wall street. Wall street is usually on the forefront on using math, technology, etc. to make trades. Sometimes the models end up causing calamity as well.

SEJeff9• 9 years ago

The clickbait headline is amusing, but "Wall Street" has been doing this > 10 years. I'm genuinely not sure if they're trying to "beat wall street" or simply be yet another quantitative hedge fund that happens to use machine learning and neural networks to power their strategies.

Most real electronic trading firms are putting those algorithms in hardware.

Source: Have worked in HFT the past 10 years.

frankc• 9 years ago

Longer than that. This article was passed around at my job to laugh at.

cr0sh• 9 years ago

Definitely much longer - I'm pretty sure I read about things like this in Omni Magazine back in the 1980s...

jredwards• 9 years ago

HFT is a different kind of 'beating the market', though, don't you think?

SEJeff9• 9 years ago

It isn't beating the market, it is literally "making the market":

https://en.wikipedia.org/wiki/Market_maker

dsacco• 9 years ago

It is "beating the market" if your strategy results in higher net fee-adjusted returns than the overall market (typically ~7-11%). But yes it's different because you're not deploying a trading strategy, you're being a market maker for other traders.

refurb• 9 years ago

Machine learning and neural networks?

SHUT UP AND TAKE MY MONEY!

6stringmerc• 9 years ago

I used to have a stock broker co-worker who had one rule for investing if day trading or even trying to employ a strategy: "You can't fight the Street."

His point was trying to bet against a momentum trade or some other scenario was foolish. He watched time and again. There were opportunities to get with the crowd, or be on the wrong side.

got2surf3• 9 years ago

Great point. And even if you are "with the crowd" more often than you are not, your timing/sizing on each trade you make as a day trader or mom-and-pop investor is very likely to be worse than an institutional trading firm. So you can be right on a "buy" vs "sell" decision 51% of the time, but an algo trading firm has probably executed with better timing/trade size and beaten you to a percentage of the profit.

A friend who works in quant finance said something along the lines of "if a trader can identify a mom-and-pop investor, it's like taking candy from a baby" in terms of making an easy profit.

jbmorgado4• 9 years ago

Source: Have worked in HFT the past 10 years.

This is not about High Frequency Trading, but about machine learning applied to the stock market.

SEJeff9• 9 years ago

Which every single HFT firm that isn't a pure market maker (such as Virtu Financial, my previous employer) does already and has done for years. That is my point. Here's a great example of Citadel talking about how they use Hashicorp nomad to spin up 100,000 cpu research jobs on AWS.

What do you think their 100k cpu jobs are doing? Machine learning and quantative analysis, what else?

https://www.youtube.com/watch?v=MRtRwhL5lwM

digler9994• 9 years ago

Source: Have worked in HFT the past 10 years.

OT question: how would I get into acquiring market data to try my own backtesting strategies ? Is there anyway to "get in" without putting up $5k + for the data ? I'm not green to this, I worked at a prop shop for a year writing market feed handlers. I know the dangers of overfitting data and how hard it is to make money (so I'm not deluded by a fantasy). I want to start out just for fun doing backtesting and see if I can scrape out a (paper) profit, but accessing historical data seems to be the hard part. I dont think online services that offer this would work for me because I want to use intraday data and replay many times with different parameters. I want access to "the whole firehose" if you will.

kybernetikos2• 9 years ago

Not a personal recommendation as I haven't used it, but you might find quantopian.com fun if you've got any python.

hendzen5• 9 years ago

You are not going to get usable intraday data for free. Best bet is to pony up for NYSE TAQ.

http://www.nyxdata.com/Data-Products/Daily-TAQ

pliny• 9 years ago

TickData[1] are cheap and good for US data, I have had bad experience with them for EU data. They have a minimum of (IIRC) $1K but you can get a ton for that.

Some exchanges offer free historical data, for instance BM&F BOVESPA[2], trading in some markets (such as Brasil and China) is often difficult if you are a foreigner.

[1] https://www.tickdata.com/

[2] http://www.bmfbovespa.com.br/en_us/services/market-data/hist...

jmalicki• 9 years ago

IQFeed has many many years of historical data at the minute level, and tick data for 6 months (for equities), at ~$80/month... they many not have the highest quality data from what I've heard, or the most tick data, but are so much cheaper than any of the more professional solutions a large fund would use that it's a pretty decent start for a hobbyist.

metakermit• 9 years ago

Not sure how usable it would be for your use-case, but I heard a talk explaining Quandl – they want to do to Bloomberg what Wikipedia did to Encyclopedia Britannica.

https://www.quandl.com

jliptzin• 9 years ago

Haven't wall street firms been doing this for decades now? I remember 10 years ago I created a trading algorithm that ran by itself without my intervention. I fail to see what is different about this company than some generic quant hedge fund.

solumos• 9 years ago

They use "the field of artificial intelligence known as machine learning" duh. "Big Data" and "Data Science" weren't around 10 years ago /s

CuriouslyC2• 9 years ago

The problem with algorithmic trading is that markets are complex systems. As an algorithm is adopted, it changes the dynamics of the market, and the result the assumptions it was based on tend to no longer hold. Note that humans trade via implicit algorithms, which is part of the reason why consistently outperforming the market is highly unlikely.

In my opinion, the whole idea of investing to try and maximize profits is myopic. The real reason to invest should be to gain a measure of control over the corporation being invested in. This suggests that the board should play a more active role in the governance of corporations.

People who just want to make a buck off a corporation should be limited to providing debt financing.

imaginenore• 9 years ago

Unless you're trading low volume penny stocks, or have hundreds of millions of dollars under your control, your trades will likely not affect anything. Considering that you need lots of data and high frequency, you will not be trading penny stocks with your [hypothetical] clever algorithm.

BickNowstrom2• 9 years ago

Let's assume your algorithm moves the market: 1) This is not a big problem: Lots of machine learning and control theory involves dealing with feedback loops. 2) If it moves the market in a predictable way, you can use this to make money.

The Western world is capitalist, not a Platonic ideal. Even if you strongly feel that people should have different reasons to invest, you will not be able to sway them (unless you can show that your alternative makes them even more money).

If you claim that long-term outperforming of the market is highly unlikely, you contribute outperforming to short-time flukes: The stock market is essentially unpredictable or in perfect equilibrium.

Unpredictability implies all these hedge funds would do better consulting random number generators, instead of well-paid quants.

Equilibrium implies the current market is operating at maximum efficiency, yet one currently makes money by exploiting non-equilibrium and erroneous evaluations.

CuriouslyC2• 9 years ago

Point #2 is what I'm talking about. Any algorithm that is going to make money long term on a large scale has to be inherently unpredictable, which is difficult if it is an algorithm that predicts the future state of the market from past data.

Sure, the world is less than perfect in a lot of ways. In most cases we try and fix it. Why is capitalism the one way where we say "oh well, that's just how things are" ??

Research has demonstrated that barring insider information, random stock picking often outperforms "experts".

I believe the market is fairly efficient, and people who make money are either lucky or are leveraging short term information differentials, which in today's hyper-connected society are going to become increasingly rare (barring insider information, again).

logicallee• 9 years ago

Can you imagine an investment AI that did not have human biases. Here are some my favorite investor biases:

SALIENCY bias. People remember memorable things. The guy who made $100 million by investing in a Romanian immigrant will invest in you if you're a Romanian immigrant, but not if you're one country over even if their education and politics are the same. Computers don't care.

AVAILABILITY bias. People analyze data that is available. If you have worldwide sales figures your market seems huge. If you no data investors will be strongly prejudiced against it.

CONFIRMATION bias. Investors who have a bubble mentality will see the positive and reinforce their theory, however non-rooted in facts (for example the theory that silicon valley teams will succeed and, for example, foreign teams will fail)

There are a bunch more, too.

http://rationalwiki.org/wiki/List_of_cognitive_biases

https://en.wikipedia.org/wiki/List_of_cognitive_biases

An AI would not suffer from any of these. However, due to the skills required to evaluate a pitch, the AI would have to be much, much smarter than any expert system today. Today you can't even tell a robot how to boil a pot of water (no matter how explicit your verbal description is) or anything else, and have it even come close to succeeding.

We're far away from robots (AI) evaluating pitches and business plans. But how cool would that be!

user5994461• 9 years ago

Can you imagine an investment AI that did not have human biases. Here are some my favorite investor biases:

It's actually a trivial problem. Just need to ask a few questions to determine the investor risk profile and goals, then pick the appropriate Vanguard fund.

bfrog• 9 years ago

Huh, hasn't wallstreet being doing this, and even putting these things into FPGA's for nearly a decade now? Hell I wouldn't be surprised if they're printing ASICs with their algos.

alfalfasprout• 9 years ago

Other than trading on news, you really don't need that kind of speed for using ML powered algos. Generally these rely on a ton of data so GPUs still reign king.

But yes, hedge funds and prop shops have been doing this for over a decade.

edblarney3• 9 years ago

Surely - but 'AI' is new. Surely 'Deep Learning' is something at least they are not doing widely, because it's difficult to pull off.

That said, anyone actually doing it would be smart enough to shut the hell up about it.

Finally - one might argue that the big money is all made in 'soft insider' information anyhow, and that with so much tech, analysts already there ... there's just no way to win without clear leverage i.e. relationships, servers on premises of the trading facility, or some other non-market advantage.

brilliantcode3• 9 years ago

This is just an anecdotal experience but according to a former hedge fund manager, insider trading is an open secret in the industry. You really can't expect to gamble away billions of investor's money without an edge. Information is the only possible edge that is closely guarded in the markets.

I don't know how true that is, he's a washed up piece of shit that wall street chewed out but has the eyes and ears of gordon gekko wanna be finance grads.

but I'm writing this because this is like the 3rd time I've heard this. Just throwing it up there if "soft insider" information is what they were referring to.

cityhall• 9 years ago

Not all algorithmic trading is high-frequency where you need to shave microseconds to beat someone else's order. If you're predicting actual price movements instead of order book fluctuations there's no reason to use hardware.

kiernanmcgowan• 9 years ago

I'm having flash backs to xkcd:

https://xkcd.com/1570/

brilliantcode3• 9 years ago

Sentient Technologies won't disclose its performance

"We are doing some shit but we have nothing great to show but you should probably read the rest of our PR article."

The CEO lives in HongKong and remotely manages this crew of distrusting, non-supportive and close-minded execs.

"We are a highly disruptive people working on disruptive technology in a disruptive way."

raw232• 9 years ago

Frustrating that so many great minds are swooped into the finance industry.

The lure of money is too strong.

hendzen5• 9 years ago

Frustrating that so many great minds are swooped into the online advertising industry.

The lure of money is too strong.

lutusp2• 9 years ago

"Silicon Valley Hedge Fund Takes on Wall Street with AI Trader"

Another generic, contentless article using the standard outline: (name) takes on Wall Street with (scheme).

Percentage of traders who beat the market average: 50%.

Traders to the left of the average who assign the outcome of bad luck: 100%.

Traders to the right of the average who assign the outcome to a secret method and/or genius: 100%.

An unscupulous broker can "prove" to you that he is a stock picking genius, by mailing you correct predictions of the market in advance of the outcomes for, say, six months, then ask you to assign your assets over to him -- but it's a scam, a trick. The explanation: http://arachnoid.com/equities_myths/#Miracle_Man

99_00• 9 years ago

Hedge funds need other people's money to make money. So I assume everything they make public is PR and marketing.

Silicon Valley Hedge Fund Takes on Wall Street with AI Trader

Related Stories