Bootstrapping an Ultra Low Latency Trading Firm, Part 2

http://veyronb.wordpress.com/2011/08/01/bootstrapping-an-ultra-low-latency-trading-firm-part-2/

by veyron • 15 years ago

119 47 15 years ago

veyronb.wordpress.com

getsat2• 15 years ago

These are great! Keep posting 'em! :)

I'm more interested in the tech than the economics, but it's still a good read.

veyron9• 15 years ago

someone left a comment in the first post saying: "... most of the crowd here probably comes from HN and as techies we don’t have much knowledge of the trading part of HFT."

I assure you, it's in the pipeline.

getsat2• 15 years ago

Understood. Thanks again for publishing these!

gaika2• 15 years ago

Don't ask who is losing on the other side of your trades but think instead who is your customer. What is the service that you provide and how it helps them.

If you're making a profit it means you bought inventory when the price was below fair value (your customers didn't need it and wanted to sell as fast as possible) and you sold it when the price was above (your customers really needed the shares right now). The net benefit to everybody is that the volatility is lower, as you moved the price down when it was too high and moved it up when it was too low.

The market efficiency is higher too: a lot less capital is required to establish fair prices as market reacts immediately to any imbalance.

It also makes the spread lower and makes buying and selling stock cheaper for your customers. Only a few years ago market makers and specialists would chicken out at the first sign of trouble and would widen the spread between bid and ask prices. Crossing the spread is a huge part of your overall expense of trading. Unfortunately very few investors understand full impact of it on their returns and don't appreciate your contribution.

Execution time is better now. Even during flash crash it was possible to buy and sell with retail brokers, where's I still remember times in 2001 when retail broker market orders sometimes took minutes to fill.

veyron9• 15 years ago

"If you're making a profit it means you bought inventory when the price was below fair value (your customers didn't need it and wanted to sell as fast as possible), and you sold it when the price was high (your customers really needed the shares right now)."

It's a bit more complicated than that, because it's fully possible that the seller in a trade is selling above his fair value and the buyer in that trade is buying below her fair value. Supply/demand certainly plays into it (a trader with a very large position is definitely willing to take some price hit if they are able to quicky their position), which will invariably lead to a discussion of utility functions and slowly bore everyone :P

gaika2• 15 years ago

> invariably lead to a discussion of utility functions and slowly bore everyone

But that's the crux of the matter - both sides of the trade are getting some utility gain (otherwise the trade would not happen) and thus it is not a zero sum game.

It is still zero sum in short term dollars, which just obscures the subject for the people who equate utility with dollars.

Nrsolis• 15 years ago

Dont forget that a lot of trades didn't clear.

I read somewhere that something like 30% of trades used to fall through because of paperwork mixups and other problems before electronic exchanges.

How would you like to think you sold at a profit only to find out that the trade didn't clear and now you're in the hole because the stock bottomed out?

veyron9• 15 years ago

The worst part was May 6 2010 (the flash crash). The exchanges and SEC arbitrarily threw out a number 60%, and people who managed to buy at 59% discount got to keep their (very) profitable trade and people who managed to buy at 61% discount lost only one leg of the trade. The process was not transparent, at all :/

I know some people who made a boatload (all trades cleared), and others who lost a boatload (one part of their trade was cleared, but the other part was broken, and they had to go back and buy back at a very disadvantageous price)

epaulson• 15 years ago

This is fun reading. It reminded me of another blog, "WK's High Frequency Trading Blog" (howtohft.wordpress.com) which got about 9 posts in with some great details, and then went silent.

I found it in my Google Reader feedlist, and the last post was from March, but Wordpress now says the blog has been deleted. Anyone know what happened it?

Veyron, I'm glad you're writing these posts. It's cool to have HFT demystified.

vitobcn• 15 years ago

No idea what happened to it, but I found this backup if anyone is interested:

http://www.opensourcetrader.com/author/wk-selph/

veyron9• 15 years ago

I personally think he was writing those so that someone would stumble across them and hire him. The new employer must have asked him to take them down.

snikolic6• 15 years ago

What bothers me is how simple this is. This is simple arbitrage that anyone with access to wikipedia could figure out in a few days.

What makes these strategies profitable is exclusive access to information - not actual investing prowess. It's about who can afford to buy the fastest hardware on the fastest pipes in the closest proximity to the exchanges and top tier e-brokers. It's frequently even about who you know.

If your only competitive advantage is systemically restricting the public's access to information, you're doing it wrong.

(but my inner nerd thinks this is super cool, and my inner Gordon Gekko wants to get started today)

sliverstorm2• 15 years ago

They are probably pretty convinced that if your only competitive advantage is restricting access to information, you are doing it right.

I mean, isn't capitalism basically wrapped around the idea that without competition, businesses will act in their own best interest, which is diametrically opposed to the interests of consumers? That is to say, given the choice any "smart" business will leap at the chance to lock out competition, because that is in the best interests of the business (if it can get away with it)?

Anyway, what this is really about is getting the information faster. I don't really see anything wrong with that; some information is just time sensitive, and a good informant will practically be in the business of getting you information faster than his competitors... I see this as much the same thing.

dereg• 15 years ago

No, capitalism is not a zero sum game, if that is what you're implying.

nl• 15 years ago

It's not so much restricting access to information as getting and acting on it quicker.

In "the olden days" there were very similar trades done by people who (for example) could get information about tea clippers from China arriving in England. Now, it's automated, but the idea is the same.

veyron9• 15 years ago

There is a signficant cost to obtain data and most of the seemingly simple trades don't make much money anymore. Purchasing realtime market data is very expensive, and you generally need to make 1000$ per trading day to break even. It's not easy in this economic climate, especially if you are trading your own money.

Amongst other things, dark pools and alternative venues have sucked a lot of liquidity that would traditionally hit the market.

The reason the information is available on Wikipedia is because people like me contributed to many of the wiki articles on the topic :)

The profitable opportunities themselves are fleeting and tiny, which means only the first few traders actually get the trades they want. This, coupled with the sheer number of firms (in the hundreds) forces traders to do what it takes to minimize their delays, including obtaining lower latency market data, high performance servers and proximity. And yes, some exchanges (like NYSE) definitely play favorites.

snikolic6• 15 years ago

What happens to the industry when market access and data is openly available at an affordable price and a speed/scale comparable to what professional traders currently have access to? And why hasn't this been done yet? As an outsider to the financial industry, it looks like this information/access is slowly becoming commoditized (IB, Lime, etc.)... is there going to be chaos when any HN reader can begin low latency trading (or mining market data) as easily as they could get started on a service like Twilio?

Why are so many exchanges still private? Why do you have to pay exorbitant amounts of money for an exchange seat or high-quality exchange access? Why do some exchanges have designated market makers? Are these systems actually effective, or is this just nepotism among bankers?

Disclaimer: I have no idea what I'm talking about; I'm legitimately curious about this stuff. Hope I'm not coming across as too brash/angry. :-)

snikolic6• 15 years ago

In the heat of criticizing the financial industry, I forgot to say that I'm enjoying your posts, and I think what you're working on is really cool. :-)

sjaakkkkk• 15 years ago

You might want to check out this paper by Alexander D. Wissner-Gross and Freer: http://www.alexwg.org/publications/PhysRevE_82-056104.pdf

snikolic6• 15 years ago

Neat!

gaius2• 15 years ago

It's about who can afford to buy the fastest hardware on the fastest pipes

I think you underestimate the skill required to do that. I mean, Google is not king of search just because they can afford bigger datacentres than Bing. Search is not just "looking stuff up" that anyone with access to Wikipaedia could figure out in a few days, even tho' there is an article on MapReduce!

LiveTheDream• 15 years ago

From part 1:

    Question: Why don’t more people do it?

    Answer: People do try, but its not as easy as it looks.

ajays• 15 years ago

A long time ago, there was a startup called Digital Island. They were a small(er) exchange (or is the term "market maker"?), but they handled about 20% of NASDAQ's daily volume.

The good thing was, they would let you buy their data post-facto (not realtime). At the end of the day, you could download their entire set of completed orders for the day. The cost was minimal ($30/month, IIRC).

But they were bought out by NASDAQ, and NASDAQ shut down this data-sharing program. Which, to me, reinforces the point that the powers-that-be don't want to lose control of the information; it's a pay-to-play system, and information is one of the privileges to paying the big bucks.

One would assume that being a heavily regulated system, it would be expected that all orders would be open to the public (at least the historical ones), to ensure transparency. But one would be mistaken.....

snikolic6• 15 years ago

> "powers-that-be don't want to lose control of the information"

This. What happens when they inevitably do?

euroclydon• 15 years ago

I'm not sure "the public" wants access to microsecond trade order data. I mean, I could send it to my mom's laptop, but she'd probably be pissed.

It's the market makers who are fulfilling these orders and the strike prices are already set once the HFT algos get in on the action (right?). I'm assuming that if the MMs had a problem with it, they'd have the clout to do something.

maximilianburke• 15 years ago

What services would you use for executing the trades?

allenbrunson• 15 years ago

115 days ago, you posted that you had quit your job and struck out on your own. Is this the project you've been working on since then?

veyron9• 15 years ago

A friend and I built an iPhone app in pursuit of a new concept, and we did a few field tests, but Apple did not approve the app. At that point, although we could have continued pushing, we decided it was easier to stop the project and part ways. I decided I didn't want to go back and work for someone else (blame HN for being bitten by the startup bug :)

joshu• 15 years ago

The first example is called "pairs trading". Try extending out the math to multiple sets.

I once wrote an ADR arb detector, which even took into account the forward contract for the currency trade.

helwr• 15 years ago

sthatipamala2• 15 years ago

Is it feasible to apply these basic HFT principles to the Bitcoin market? I assume it would be easier than the "real" markets because you aren't competing against the big players.

sthatipamala2• 15 years ago

Wow, downvotes for asking a question?

Sukotto• 15 years ago

Nice post veyron. Please keep it up.

For those interested, here's some other good reading on this topic while you wait for veyron's next post :)

- http://www.technologyreview.com/blog/mimssbits/25308/ Using textual analysis

- http://www.institutionalinvestor.com/Article/2593339/Markets... Article on HFT

- http://news.ycombinator.com/item?id=1517339 Ask HN: Is it feasible to do high-frequency trading as an individual?

- http://www.reddit.com/r/IAmA/comments/9s9d7/iama_100_automat... I bootstrapped an automated trading firm ama

- http://quant.stackexchange.com/

- http://bitstat.wordpress.com/

known• 15 years ago

You can buy off the shelf HFT solution from http://www.stoneridgetechnology.com/products/pci-e-developme...

veyron9• 15 years ago

There are all kinds of companies selling technology solutions for everything (as targeted as FPGA-driven feed data processors; as all-encompassing as the solution you presented). Most of them are crap and oftentimes have no paying customers. Hopefully, I will be discussing the state of the financial tech industry in a later post.

orenmazor• 15 years ago

I don't know why I wrote this field off as boring several years ago. This is absolutely fascinating (from a technical perspective, I mean)

Bootstrapping an Ultra Low Latency Trading Firm, Part 2

Related Stories