Algorithms take control of Wall Street

http://arstechnica.com/tech-policy/news/2011/01/algorithms-take-control-of-wall-street.ars

by shawndumas • 16 years ago

53 37 16 years ago

chrisaycock4• 16 years ago

This article is very poorly researched. For example, it refers to high-frequency traders as "flash traders". I have never heard that term before. The cited Wikipedia article discusses flash orders, which was a quoting mechanism that DirectEdge experimented with in 2009. Flash orders are not some most automated market markers even care about.

Also, the article mentions Harold Bradley as "among the first traders to explore the power of algorithms in the late '90s", completely omitting much earlier pioneers such as Peter Muller and David E. Shaw.

And of course it goes into depth about the Flash Crash without once mentioning Waddell & Reed, a discretionary (non-quant) fund whose errant trade started the crash to begin with.

Reading this was like watching the 11-o'clock news discuss a medical discovery.

w1ntermute2• 16 years ago

Nothing unusual then. Pretty much every tech article in the WSJ is of this caliber.

atuladhar• 16 years ago

Except that I see no indication this is a Wall Street Journal article. The site is Ars Technica; the by-line says "By Felix Salmon (wired.com), Jon Stokes" and the end of the article says "Felix Salmon (felix@felix salmon.com) is a blogger for Reuters."

Dn_Ab5• 16 years ago

>Also, the article mentions Harold Bradley as "among the first traders to explore the power of algorithms in the late '90s", completely omitting much earlier pioneers such as Peter Muller and David E. Shaw

Don't forget the super pioneering work that Claude Shannon and Ed Thorp were doing way back in the 60's and 70's too!

jfager• 16 years ago

which was a quoting mechanism that DirectEdge experimented with in 2009

Could you go into more detail about what you mean by that? Unless you're being pedantic about something specific, I don't get how you could say that. Flash trading (in several variations) is basically the only thing Direct Edge offers, and they currently carry roughly 12% of the market's daily trading volume.

beagle3• 16 years ago

Everything else you said was spot on, but this is wrong:

> And of course it goes into depth about the Flash Crash without once mentioning Waddell & Reed, a discretionary (non-quant) fund whose errant trade started the crash to begin with.

Waddell & Reed had nothing to do with the big flash crash, nor with the 10 slightly smaller crashes that has happened since -- at least, not any other non-HFT trader had that day. If you want something which isn't some ridiculous CYA, read Nanex's analysis, not the SEC.

ct• 16 years ago

The neural network that Bradley was working on sounds interesting.

aothman3• 16 years ago

NB: This article (and several more like it) was in January's wired.

As an AI researcher, I get suspicious when I see anyone talking about Genetic Algorithms and Neural Nets. These are techniques that current researchers simply do not use (Neural Nets are used very sparingly, GAs should never be used at all). They make up for their technical failings by being approachable, particularly for journalists. In short, these methods intuitively sound like they should work much better than they actually do.

istjohn• 16 years ago

What techniques are now used in place of GAs?

chrisaycock4• 16 years ago

And in trading applications (as with many other domains), GAs can lead to fitting if the quant isn't careful. What looks like a great strategy in a back-test suddenly blows-up in production. I worked at one hedge fund whose safe-guard against GAs was to make traders submit their algos a month before production so they could be tested again out-of-sample.

Dn_Ab5• 16 years ago

I am not sure I agree with your take on neural networks. Neural network is a vague terminology. But most people mean multilayer perceptrons which I agree are not very useful. But logistic regression is a very widely used 'single layer neural network'. Even the original single layer perceptron has its place in this big data world (with appropriate modifications: e.g. adapting ensemble and on-line techniques , using the kernel trick etc.).

Then there are auto-encoders, Restricted Boltzmann machines and Deep Belief Nets that are certainly getting a lot of attention and are a type of neural network.

Eliezer• 16 years ago

I was wondering if anyone else had that reaction. I'm looking at a story about how Bradley was using elaborate GAs to choose the weightings on a small handful of variables that, from the description, sounds like they should've yielded to a regularized linear regression producing a fast cheap optimal answer; and I'm thinking to myself, "Either the journalist completely failed to understand and/or convey the problem that was being solved, or nobody designing this system understood anything about statistical learning and were just throwing cool-sounding tools at the problem, or they were just having fun at the reporter's expense."

For those unlearned in AI, an analogy: the original article reads like someone talking about how they used Haskell monads to add 2 + 2.

leogau• 16 years ago

"Even if each individual algorithm makes perfect sense, collectively they obey an emergent logic—artificial intelligence, but not artificial human intelligence. It is, simply, alien, operating at the natural scale of silicon, not neurons and synapses. We may be able to slow it down, but we can never contain, control, or comprehend it"

I can see algorithms being used to automate decision making in almost every other industry, not just Wall St. It's both scary and exciting to see what happens with the resulting "emergent intelligence".

jacques_chester2• 16 years ago

The systems dynamics folks point out that systems with more rapid feedback often have more variability.

The classic example is a car dealership deciding to order cars from the factory. When should they be ordered, and how many should be ordered?

Suppose they decide that every day they place orders based on the 7 day moving average. Then they get a busy day and over-order for a week. That's an expensive mistake.

Next they decide to shrink the window to 3 days (more rapid feedback). Now after another busy day they find stocks dropping precipitously and order big to make up, follow by drops, followed by big orders. In an attempt to stabilise the system, they've made it worse.

The modern stock market has extremely short feedback times, on the order of microseconds. When a feedback loop forms on the market it can spiral out of control within seconds and minutes -- hence the 'flash crash'.

Two ways to deal with this might be to develop some kind of balancing feedback loop (for example, higher prices for more trades-per-second, or a progressive price for trade based on delta with the last trade) or to reduce the feedback rate. An economics blogger I host, Nicholas Gruen (you may know of him as Australia's Gov 2.0 inquiry leader), has suggested just that:

http://clubtroppo.com.au/2010/08/07/a-modest-proposal-to-rem...

http://clubtroppo.com.au/2010/10/08/a-self-denying-ordinance...

joshu• 16 years ago

Or to do more orderly crossings at regular intervals.

The market makers (and shadow market makers) on the ECNs get paid for providing liquidity, rather than having to pay.

This kind of stuff isn't supposed to happen on the NYSE, as all the trading goes through a single specialist.

yummyfajitas• 16 years ago

The systems dynamics folks point out that systems with more rapid feedback often have more variability.

You've got systems dynamics folks completely backwards. Smaller, more frequent controls tend to stabilized systems, not make them unstable. A simple example we've all seen: take a stable ODE. Now try to discretize it - if you are unlucky or uncareful, your discrete approximation can easily blow up exponentially.

The example you provide is different - you are describing two different control strategies, one of which fails to correct for noise (and note: any HFT who makes this mistake loses money FAST). If you made orders every day based on the 7-day moving average, it would be better than making orders once per week based on the 7 day moving average.

This is exactly what we saw with the flash crash - there was a large exogenous shock and the system self-corrected within minutes.

Sniffnoy• 16 years ago

> "Automated trading systems will follow their coded logic regardless of outcome,"

Of course, so will humans. They won't make the absurd trades referred to, but a system of trading humans can still, say, get stuck in a positive feedback loop. Of course, they aren't pure trading algorithm, so they may notice the problem, but actually putting a stop to it requires coordination and is hard. Mostly the loop just occurs more slowly.

chrisaycock4• 16 years ago

Yep, which is precisely why economic bubbles happen. Algo traders weren't responsible for housing prices in the last decade.

extension2• 16 years ago

Comforting to know that the economy is being gambled on a roshambo programming competition.

http://webdocs.cs.ualberta.ca/~darse/rsbpc.html

jrockway2• 16 years ago

Why do you think this is true? For every trade, there is someone on each side. The market as a whole does not care if you make bad trading decisions. Someone else makes up for it.

extension2• 16 years ago

I think our money needs to be working towards things that we actually care about: advancing science and technology, creating art, socializing, and so on. Putting it all into a big game of craps is not going to improve the human condition in the long run. I mean, it's an interesting game, but not interesting enough to justify the entire economy revolving around it. And people don't play because they find it interesting.

Algorithms take control of Wall Street

Related Stories