Mysterious Algorithm Was 4% of Trading Activity Last Week

http://www.cnbc.com/id/49333454

by mikecane • 14 years ago

114 126 14 years ago

ahi• 14 years ago

"Translation: the ultimate goal of many of these programs is to gum up the system so it slows down the quote feed to others and allows the computer traders (with their co-located servers at the exchanges) to gain a money-making arbitrage opportunity." I thought the justification for HFT was liquidity. The market is clearly broken.

jacobwil• 14 years ago

I believe that they were talking specifically about the programs which create orders which are never fulfulled. That way their programs creating real trades have an advantage (since they are running on servers located closer to the exchanges).

skorgu2• 14 years ago

Bear in mind the exchanges aren't stupid, Nasdaq at least has instituted a penalty system for sending a bajillion orders and not actually trading [0]. One may certainly quibble with the exact rates and penalties but this is a thing that exists.

[0] http://www.nasdaqtrader.com/TraderNews.aspx?id=ETA2012-13

davidtyleryork• 14 years ago

This makes Mark Cuban's article on High Frequency Trading (HFT) even more relevant than it was before (link: http://blogmaverick.com/2012/09/21/what-business-is-wall-str...). WHY is it not regulated or at the very least taxed, when it creates such outsized risk for the public and doesn't accomplish anything for the market besides increased volatility

twoodfin• 14 years ago

If you're so worried about it, just don't invest in markets that allow it. The "public" is only capable of risking their own money.

marshallp23• 14 years ago

Most people can't do that because their money will be put there by their 401K (which if they don't put money in willl lose out as well).

Steko• 14 years ago

This sounds like "don't go outside if you're worried about being mugged". Sorry, no we almost all pay taxes used for bailouts and have our retirements invested there by our employers. We get to criticize and advocate for change.

confluence10• 14 years ago

What people don't seem to get about HFTs is that retail investors cannot and should not attempt to compete with them or trade actively at all - HFT or prop trading is a big no-no for the unsophisticated.

Historically buying into an index fund during a crisis is a perfectly reasonable way to achieve relatively easy alpha above the mutual fund/bond rates (~12% per year).

This type of investing (aka diversified value) is completely separate from HFT. As far as many retail investors should be concerned HFTs can duke it all out between themselves as much as they want - it really doesn't matter and is irrelevant to your long term investment decision.

Indeed HFTs provide the benefit that during times of crisis they provide plenty of buy side liquidity for you to get in at a lower spread and a lower spot price relatively easily.

smoyer• 14 years ago

I agree completely ... let the HFTs eek out a few pennies per trade (or even fractions of cents). But I also wanted to see how much bandwidth my 4G phone could really use. Sorry for the perturbations I might have caused and I promise I'm only doing what's good for man-kind. Now if I could just get my phone to make a decent call!

confluence10• 14 years ago

I don't know what you mean. HFTs don't do what's good for mankind - they do what's good for themselves. Just like everyone else. No one does what's good for mankind - they do what's best for themselves.

We allow those who don't harm others to continue doing what benefits them - until such a time as it is shown to be harmful - then we stop it. HFTs have not been shown to be harmful.

dfc3• 14 years ago

Are you saying that HFTs have no impact on Alice and Bob regardless of how prudent they are about investing? In addition to ETFs Alice and Bob also have market exposure through a company 401k. Are the 401k managers and ETF managers immune to the effects of the HFTs?

On the page[1] first documenting this algo NANEX says the following:

"We believe that this algo will continue to grow and if left unchecked, could very well contribute to the next flash crash because it removes precious network capacity and provides zero economic value such as price discovery."

Do you disagree?

[1] http://www.nanex.net/aqck2/3610.html

confluence10• 14 years ago

It has no effect on Alice and Bob if they don't sell. It is favourable to them if they buy.

The stock price is just the opinion of 0.2% of people at any one time - and hence means essentially nothing to an investor. Does the spot price opinion/first impression of a person matter - or does the long term attributes of their character matter? Just because there are more opinions - being given at a faster rate - doesn't mean that a) they're right or b) they should be acted upon as fact and c) that they shouldn't be exploited for those of a more stable nature.

For example: I bought a huge amount of TSLA stock when it fell 12% in one day a week or so ago for no particular reason. I subsequently realised a 6% gain. I'm happy - thank you HFTs and short term traders - your vol makes my alpha.

mcphilip2• 14 years ago

>Indeed HFTs provide the benefit that during times of crisis they provide plenty of sell side liquidity...

Any proof to that claim? Liquidity evaporated during the flash crash. The SLPs have no obligation to provide liquidity and can simply pull the plug when the market is in crisis. [1][2]

[1]http://finance.yahoo.com/mbview/threadview/?m=tm&bn=228&... [2]http://compoundingmyinterests.com/compounding-the-blog/2012/...

confluence10• 14 years ago

Buy side liquidity means quick access to sellers at little cost and on favourable terms to you. This was always the case - HFTs just make it faster and easier.

So if you are a buyer in a crisis - you get a better deal with HFTs quickly matching supply/demand of large stock orders. If you are a seller - you're screwed either way - HFT or not (see history).

trotsky• 14 years ago

Prop trading (proprietary trading) just means a firm is using their own money as opposed to trading on a customer's behalf. So the retail investor is by definition almost always engaging in the practice.

marshallp23• 14 years ago

Another reason to limit trading to daily or weekly or montly. The trading arms race is focusing on speed when it should be focusing on predicting longer term outcomes. The easiest way to do so would be to limit all trades to once per day in an auction system. Then traders would pour their resources into building sophisticated systems that make long range predictions and so also identify new innovations/companies that should exist in the market. This is in literally almost everyone's best interest - investors, public corp's, governments, consumer - only a small group of high-speed traders would disagree. Why this hasn't occurred yet is mind-boggling.

dclusin• 14 years ago

Trading has always been about speed. This dates back to the creation of the earliest financial markets (http://en.wikipedia.org/wiki/Rothschild_family).

marshallp23• 14 years ago

The evil Rothschild's are still costing today. A redistribution of wealth that was accumulated prior to 1900 should be instituted (or maybe for even before 1980).

darknoon• 14 years ago

Even if trades were only executed daily, wouldn't a futures market develop that was traded at the millisecond level anyway?

marshallp23• 14 years ago

Reducing the trading increment is what a futures market does anyway (you can buy contracts to remove the volatility of second/minute/daily activity).

nancyhua• 14 years ago

High frequency traders talk about this and I heard about one being developed. Regulators and basically everyone you listed don't know what's in their best interest so it's up to the traders who decide they want to stop the arms race. It's hard to switch over bc traders go where the liquidity is so we'll have to see if they can overcome the chicken and egg problem.

Evbn9• 14 years ago

24 hours is your definition of long-term investment?

marshallp23• 14 years ago

I didn't say long-term investment. I said limit trading to once per day or week. Then resources will go towards creating predictive models in the longer time range than milliseconds/seconds etc. Even limiting to monthly would be good. Most real world projects (that directly benefit soceity) occur on days/weeks+ timescales. There's no reason for stock trading to occur faster than that since it's ultimate purpose is to finance those projects.

ArchD3• 14 years ago

Suppose the NYSE has a new rule where trades only happen once a day, in a morning auction. Imagine that you just bought some stock at the auction in a certain company. Later on in the day, the company announces some catastrophic news, but you have to hold on to your stocks until the next day's auction, only to suffer great loss when you sell.

Meanwhile, on a options exchange, a different exchange, where trading happens throughout the day because there is no such regulation, other investors in the same stock get to hedge their position by selling call options. They get to act ahead of you because their exchange has no such regulation. Unfortunately for whatever reason, you have no access to this venue. Should regulators regulate these exchanges as well? If they don't, someone will complain that it's unfair to people who have no access to such exotic venues. So, let's say they regulate all exchanges in the country.

Then, people who really want to trade in the middle of the day have to do it the 'good' old-fashion way, away from the exchange, making phone calls to each other, through brokers, where there's less transparency and more chance for corruption and unfair deals because the fair price of an instrument is unclear due to the absence of exchange activity. This would be a step backward.

Additionally, in terms of raising and reallocating capital, companies whose stocks are thus regulated will be at a disadvantage relative to their international counterparts whose stocks are more liquid in the absence of similar regulation, as capital tends to flow where trading is more convenient. So, this effect would impact the country's competitiveness as well.

marshallp23• 14 years ago

If that's the case, things are so efficient that things will spill over in those ways, why not open the exchanges to everyone. Anyone should be able to list a company for zero cost, and anyone can purchase stock in anything for zero cost and no net worth requirements. The competitivness of the market will sort it all out.

vl• 14 years ago

Bonkers, currently exchanges are only open from 9:30am to 4pm EST Mon-Fri. So your "doomsday" argument is equally applicable - most companies make announcements outside trading hours and most unhedged investor loose money once marker opens if announcement is negative.

yk• 14 years ago

Translation of "Translation: the ultimate goal of many of these programs is to gum up the system so it slows down the quote feed to others and allows the computer traders (with their co-located servers at the exchanges) to gain a money-making arbitrage opportunity." Annomyous is (D)Dosing the market. :D

lojack• 14 years ago

So, the video made it sound like someone was holding a large amount of stocks and very quickly sold all of them (like the flash crash). It didn't lead me to believe that they were placing orders and pulling them before they could be executed. Was this type of order discovered after the video recording, or am I misunderstanding what they said?

Also, in the video they say:

"What you're saying is the individual investor has no shot. Mary and Joe sitting out there have no quants on their staff"

My assumption was always that the average trader holds their shares over a longer period of time, and thus aren't really affected by these HFT algorithms. Is that assumption incorrect?

tptacek6• 14 years ago

No, it is not incorrect. The impact of HFT on buy & hold investors is probably a wash. Automated trading has cut the bid/ask spread but may contribute to volatility (it is tougher to say than it seems).

Either way, Mary and Joe were never competing with market makers and active trading strategies, and still aren't now.

marshallp23• 14 years ago

That's not entirely true. I'm an amateur and I want to invest actively using statistics. But getting tick data (covering years/decades) is costly and dealing with it as well. I could handle it if it were daily (if all trades were limited to 9am auction every day. This would also allow room for much more extensive modeling of other data sources and longer term predictions).

greeneggs• 14 years ago

1. From what I've heard, all large organizations need to trade with HFT strategies in mind. (Is this wrong?) So if Mary and Joe own index funds or mutual funds, then they are competing with active trading strategies. It is basically a tax.

2. If HFTraders are collectively making money, and presumably they must be, that money must be coming from somewhere. If, for example, the stock market rises 50%, via some sequence of gyrations, then there is only a fixed gain to be divided up. Maybe HFT increases the size of the pie, but that feels like mostly a higher-order effect.

Am I wrong?

_zeo8• 14 years ago

HFT is an exploitation technique to extract value from the market. It drives away capital formation and investor participation, and is bad for everybody except the HFT traders themselves. All non-HFT traders are affected negatively.

Evbn9• 14 years ago

Citation needed.

mey• 14 years ago

How does the exchange charge for access? Per trade, per bandwidth, per colo distance?

It would be simple for the exchanges to reign in trading by adjusting their pricing model. Make it more expensive to make each type of request. Adding a few cents to the request would squash HFT while not drastically effecting other trading systems/people.

cube13• 14 years ago

All of the above, actually.

The exchanges have a per-trade charge, and for the data, they charge fees on a per data stream basis, and charge for colocation.

And they already do make each tier more expensive than the previous.

skorgu2• 14 years ago

Here's Nasdaq's price list: http://www.nasdaqtrader.com/trader.aspx?id=pricelisttrading2

johnmacintyre• 14 years ago

"abruptly ended at about 10:30 a.m. Friday" ... am I the only one who sees this as a probable programming bug? Mysterious algorithm? I suspect

if(condition = true){...} // notice the single '='

:-D

dfc3• 14 years ago

The NANEX research page[1] has a lot more information about the different MOs that they spot. It looks like NANEX first spotted it on October 1st:

"On October 1, 2012, we detected a new form of quote stuffing that tries to hide under the radar. It involves hundreds of stocks and millions of bogus quotes during the trading day. The algo that generates them is very careful to spread out the stocks targeted and limits each test to a blast of exactly 200 quotes in 25 milliseconds or less. The quotes all come from Nasdaq, and sometimes affect the Best Bid or Offer: the only change appears to be a fluttering of the bid or offer size."[2]

Doing my best tptacek impression "the nut graf is":

They posted an update on the 5th with a neat visualization of the trading activity[3]. If you just want to see a neat animated gif the direct link is:

http://www.nxcoreapi.com/aqck2/Algo200.gif

They have a list of the activity from October 1st[4], but it is not clear to me what the second and third fields are:

Example:

    09:01:03|P|9|UMDD
    09:01:03|P|6|MVV
    09:01:03|P|6|MIDU
    09:01:03|P|4|IVOO
    09:26:20|P|9|UMDD
    09:28:44|P|7|SGOL

The second and third fields have the following possible values:

    13 4 N P Q
    0 1 2 3 4 5 6 7 8 9 a A b B C D E F

I was looking at the nxcore api docs [5] and I could not tell what these fields could be. The field values do not seem to match up to the listed values for exchanges or order types.

[1] http://www.nanex.net/aqck/aqckIndex.html

[2] New Quote Stuffing Algo http://www.nanex.net/aqck2/3610.html

[3] Quote Spammer Spotted http://www.nanex.net/aqck2/3614.html

[4] http://www.nanex.net/aqck2/3610%5C20121001.200.txt

[5] http://www.nanex.net/apidocs.html

EDIT:

I am delighted and rather surprised to say that someone from NANEX responded to my inquiry about the second and third fields.

"P = ARCA (we've since determined that the algo only ran on NYSE and Nasdaq listed issues)

N = NYSE

Q = Nasdaq

The 3rd column is the SIP multicast line that would carry the quote. For Nasdaq listed, those are carried on UQDF which uses 6 lines and we label as A, B, C, D, E, F. For NYSE/ARCA listed those are carried on CQS which uses 12 lines, we label as 1-9,a,b."

mcphilip2• 14 years ago

Excellent write up, I was hoping Nanex had some info on this. Tangentially related, The Rise of the HFT Machines is another great Nanex article that argues how it's not so much HF trading that threatens market stability, it's the HF quoting[1].

There are some great charts illustrating how every time exchanges increase quote processing capacity the HF algos gobble up the new bandwidth, but ultimately the number of executed trades has remained relatively consistent over the past five years.

[1]http://www.nanex.net/aqck/2804.HTML

delinka2• 14 years ago

"Mysterious algorithm..."

I suppose that's technically correct-- we don't know how the algorithm works. But they make it sound like they have no idea to whom the algorithm belongs. If your logging can't get you this information (the bot's owner), I have no hope for the security of the exchanges.

If this kind of thing is a problem, then read your logs, visit the 'offending' party, and Do Something About It.

dfc3• 14 years ago

Read your logs? The people that identified the algo are just looking at quotes coming in from different exchanges. What logs do you think they can look at?

Jupe• 14 years ago

So, requests come into the trading system and there's no tracability? No credentials? No security? I'm not very religious, but heaven help us if this is the state of the Wall Street automation!

rorrr• 14 years ago

Trades are pretty much anonymous, only the exchange itself knows who the participants are.

delinka2• 14 years ago

I guess I missed that the exchanges aren't commenting on this story. Now I suspect they haven't commented for many (if any) of these kinds of stories.

podperson• 14 years ago

For many years George Soros has been suggesting that a tiny tax on transactions would eliminate a huge amount of high frequency bullshit (he was suggesting it for currency markets, but the same applies here). I think he proposed that the revenue generated could fund the UN... It's about the same Bill Gates's suggestion of charging a penny to send an email -- no impact on legitimate use and a serious impediment to spammers.

xenophanes2• 14 years ago

How does charging me somewhere vaguely in the ballpark of $36.50 per year qualify as "no impact"?

And I'm just an individual. What about when I send to an email list and it goes to 100 people? Should Google Groups pay a dollar to send out those 100 emails? What about MailChimp, do they have pay $200 when they send out a newsletter to 20,000 people?

I'm also concerned about how this money will be collected and how, from a technical perspective, all emails will be monitored. Every SMTP server will now have to be registered with the central oversight organization? VPNs and rogue email installations will be the enemy or blocked from communicating with anyone "on the grid"?

jon6• 14 years ago

I suppose bitcoin could be used as the technical solution. Maybe receivers of money can white-list certain senders so they don't have to pay anything to send to them. Then only spammers and unknown people would have to pay.

Evbn9• 14 years ago

You may not be aware now, but postal mail is on fact charged per unit, no central registry needed, just a stamp.

dredmorbius2• 14 years ago

"A penny an email" is a figure that's tossed out, but it could be far lower. And let's ignore the technical aspects of how you'd go about collecting it (SMTP really isn't designed to make such payments easy, though we could bolt, staple, glue, and wire on yet another patch to the mess).

The point is that abusive email -- brain-dead bulk spam, and even annoying recruiter pitches and "real" business adverts -- are sent in far higher volume than even the all but the very largest of mailing lists. The standard figure bandied about for most of the past decade is that some high-90% of all email is spam. Let's say 97% (as Microsoft reports: http://news.bbc.co.uk/2/hi/technology/7988579.stm).

Spammers are generating returns based on a small fraction of a percent response rate on an overwhelming volume of mail, which is close to free. Sender costs are estimated at $0.00001 (http://www.clickz.com/clickz/column/2138759/make-spammers-pa...). That's 1/1000th of a peny per mail, or a cent per mil (thousand mails).

I suspect that raising that cost by even just a few cents per mil would be sufficient to put the brakes on most spam. For personal email, that would literally be a few pennies a year for even a high-volume correspondent. Protocols for mailing lists and other legitimate noncommercial use would probably rely on offsetting costs to users.

In reality, proposals such as hash-cash, greymilter, targrubing, and the like extract computational work from an untrusted SMTP sender as a requirement for accepting mail. It's a pretty effective way to put the brakes on high-volume delivery. Computational power is a pretty reasonable proxy for coin of the realm.

And if that's not enough ... there's plenty that's going on now with DKIM and other header validation that would serve to identify a sender. Extend that such that you don't accept non-signed mail (except at a very, very, very slow rate) and that you can associate a designated signer with a verified escrow account, and you've got much of the nuts and bolts of putting a system into place. Not that I see this as entirely desirable, but we may have to go there.

Permit8• 14 years ago

Many exchanges want high frequency traders as market makers and pay them rebates for completed trades. High frequency trading is not inherently "bad". It's encouraged and desired by many exchanges.

cma2• 14 years ago

Exchanges are paid by the trade, why wouldn't they want a high frequency of trades to occur?

dredmorbius2• 14 years ago

Sounds as if the HFTs are initiating trades but not completing them.

In network terms, it's a SYN attack -- you're opening a port but not connecting to it, or dropping it.

A tax on uncompleted transactions might also help here. I'd say Soros's suggestion also has merits, though perhaps the incomplete transaction tax could be set somewhat higher.

Neither would have to be large to make HFT grossly unprofitable.

tantalor• 14 years ago

Misleading headline; trades are not the same as quotes.

mratzloff• 14 years ago

Someone help me out. Why can't they just throttle connections?

Mysterious Algorithm Was 4% of Trading Activity Last Week

Related Stories