High Frequency Trading on the Coinbase Exchange

https://medium.com/@abarisser/high-frequency-trading-on-the-coinbase-exchange-f804c80f507b

by modeless • 11 years ago

111 51 11 years ago

jklein11• 11 years ago

Just curious... would your market making strategy be susceptible to the spoofing strategy that allegedly caused the 2010 flash crash.

kasey_junk6• 11 years ago

My bot even has additional logic to prevent it from being tricked by fake volume walls from other bots.

Not the author, but this sentence I believe speaks to counter measures he is using to prevent spoofing.

But like any counter measures it is an arms race. As the spoofers get more sophisticated he has to be as well.

tptacek2• 11 years ago

So, what parts of coding a simple conservative MM were tricky for you? What surprised you? How math-y did the code get?

SniperOwl• 11 years ago

I'm curious to know what the AUM of the account is, is this a one man shop trading over regular internet lines and how much is paid in commissions a day on average?

kasey_junk6• 11 years ago

There’s no premium access, no expensive trading floor credentials. It’s totally open. I love that.

I completely understand this sentiment and agree for the most part. That said, for the specific case of co-location it would be better for the exchange to offer paid access.

The reason is that if the exchange doesn't offer it as a standardized option (and the exchange gets successful enough) being close to the exchange will become it's own pet industry. Only those "in the know" or with the ability to pay them will know the right places to host their bots and in the extreme cases only very rich players will have access as they buy up all the premium real estate.

This happened with the other exchanges, so we can only assume at some level of success it will happen with bitcoin exchanges.

hexedpackets• 11 years ago

Coinbase is hosted on AWS in the us-east-1 region; this makes co-location incredibly cheap and accessible. The specific availability zone isn't publicized, but it would be trivial to do latency comparisons between the zones and figure it out - since Coinbase is a fairly new company, there are only 3 zones they could be in.

DrStalker• 11 years ago

Availability zones are randomized per accounts; my us-east-1a might be the same as your us-east-1b.

This is presumably done to stop everyone putting things into the same AZ by just choosing the first one in the list.

There is some more detail here: http://alestic.com/2009/07/ec2-availability-zones

kasey_junk6• 11 years ago

The CME & ICE were hosted at 350 Cermak in Chicago. One of the largest data centers in the world. Getting machines in that building was never hard or expensive.

But there was a whole cottage industry built up of consultants which promised to tell you which specific room or cabinet in a room had shorter cable runs to the exchange. There were companies that would idle those cabinets/rooms just so no one else could use them.

AWS hosting is more opaque and thus more likely to be gamed in this way. I wouldn't expect anyone to pay for physical presence information (the latency requirements aren't there yet). But I can totally see large players buying massive amounts of nodes in order to monitor/cycle through the best performing ones or in an attempt to impact the exchange operations. You would also expect to see the pay for Amazon employees with non-public details about node creation/communications to all of a sudden become much more valuable to large banks/trading firms.

Is this something that Coinbase needs to deal with right now? No, it isn't big enough or valuable enough for it. But long term the idea of "open" access falls apart under these conditions, and the more fair approach is actually the paid fee one. As counter-intuitive as that seems.

nmjohn• 11 years ago

    ➜ curl -i https://api.exchange.coinbase.com
        HTTP/1.1 404 Not Found
        Server: cloudflare-nginx
        Set-Cookie: AWSELB=7541......;PATH=/

It looks like they are hosted on AWS and only provide access via cloudflare.

Given that, I don't think offering paid access to be closer is really relevant.

The closest you can get physically would be to find the availability zone - however your traffic is still going to be routed through cloudflare, so you might be better being closer to the cloudflare servers handling your requests.

jsmthrowaway• 11 years ago

Cloudflare will respond to a ping, but a request will pass through. Clever folks will realize that finding which AWS region(s) host(s) the ELB is trivial with curl --trace-time and ping, since you can deduce the RTT from Cloudflare's proxy through to the ELB. You also know geographical Route 53 and other tricks aren't in use, because you automatically lose that information through Cloudflare.

I'm fairly confident it's in us-east-1 based on a quick glance.

erichurkman• 11 years ago

There's nothing preventing them from also opening up direct traffic to those paying for 'close' access that does not flow through CloudFlare or setting up for-pay servers in in their same availability zone(s), maybe through a direct VPN for paying customers, bypassing as much of the public internet as possible.

waterlesscloud• 11 years ago

I've hear OKCoin has 'priorty access' available for their api, paid access that lets you have more frequent queries. Though I don't know for sure.

imaginenore• 11 years ago

1) How do you deal with wash sales tax rules?

http://www.investopedia.com/university/definitive-bitcoin-ta...

2) How much % do you make a month from it?

harryh2• 11 years ago

I'm not the OP but it doesn't sound like he's holding bitcoins for long periods of time so wash sale rules wouldn't apply.

toast0• 11 years ago

Not the OP, but the best way to deal with wash sale rules is to never sell at a loss. :) Other options would be

1) close out all positions around december and wait 31 days (and until the next year) to open positions again.

2) There's a mark-to-market election you could use to be not subject to wash sale rules, this seems like a complicated topic, but is probably worth investigating.

3) In line with always sell for a gain, you may be able to structure bitcoin lots like you would stock lots, and sell specific lots to avoid many wash sales.

Legogris• 11 years ago

Not the OP, but those specific rules are US-based and OP didn't state their nationality.

scintill76• 11 years ago

Your link says "Bitcoin traders may be off the hook for now". Do you have a solid source saying that wash sale rules apply to Bitcoin?

greenleafjacob• 11 years ago

Setting it loose for the first time, knowing that any bug could literally throw away cash, was terrifying.

I am interested in projects that deal with real cash values as well and the unique concerns that come in. One design pattern I've read of is circuit breakers, like those used on the official exchanges [1] are typically implemented in HFT's; just stop trading if your total net loss that day is some order of magnitude higher than your typical expected losses.

I expected the market for bitcoins was saturated with market makers. Interesting that author was able to set up shop and make money.

[1] https://www.nyse.com/markets/nyse/market-model

lucaspiller• 11 years ago

OP didn't say how much he was making though, I expect it's more a free trip to Starbucks every week rather than sipping mai tais on a beach.

Rainymood• 11 years ago

And it's also how much risk he's taking. Which is unkown to him I figure? If you risk 10k for a chance at free starbucks every week ... not sure if I am willing to take that risk.

cheatdeath• 11 years ago

A max loss is probably the first thing you should code!

It's not so easy though. What if you have a bug in your pnl calculator? Also, do you just block all new orders when you hit the max loss or do you liquidate your positions when you hit the max loss - because that's more code you could have bugs in. Maybe you cancel initiating orders and go to market on liquidating orders? Maybe you have code to limit the number of orders/size you can be working simultaneously - might have a bug in there too!

What if someone clears out the book i.e. a huge sell orders clears out all the buy limit orders and your pnl prints down massively for a millisecond? Should you also sell when this happens - probably getting a terrible exit price or do you wait and see if the book repopulates?

sputknick• 11 years ago

I love this story and I love the idea of it. This sounds like a fun way to feel like a big baller. But I'm dying to know how much he's making. How could you write an article about how to make money doing something, then not at least give us an idea of how much money you made!?

j_lev2• 11 years ago

Market-making also delivers real social utility. The deeper the liquidity provided by market makers, the more difficult it is to cause erratic spikes in price. Market makers also reduce the bid-ask spread, a concept most people aren’t even aware of: a testament to successful practitioners on Wall Street.

Until they don't. The moment people need liquidity the most is the moment the market makers have pulled all their orders and the market is in free-fall.

tptacek2• 11 years ago

This statement doesn't make sense. People don't need liquidity at a moment. Liquidity isn't binary. Less liquidity, higher spreads, higher cost of trades. Those costs are a tax paid by long-term investors into the pockets of middlemen.

hawkice• 11 years ago

I'm not parent, but here's an idea that's floating around, maybe it can be used as a reasonable guess to as how to make sense of it:

There is a concern that high frequency trading is technology that, while creating liquidity, increases the tax paid by long term investors. Instead of having a meaningful spread, long term investors often have trouble filling an order before the entire market jumps, settling back down less than a tenth of a second after the investor's trade clears. The tax is paid a different way, and often at much higher amounts, given the limited data coming from trading arms of various banks.

So there is more liquidity, which is good (although, as a civilian, I still need 3 days for _my_ trades to clear on the stock market, so I guess if it was such a huge public good for trades to clear faster the rules would be different). But the market is so jumpy, and so few of the orders placed reflections of actual investor intent, that markets can enter free-fall more easily (because it's happened for no reason). So there is higher liquidity, but it doesn't come with two of the upsides of a free-flowing market: stability and lower transaction costs.

harmegido• 11 years ago

Are you trying to imply that this didn't happen when computers weren't involved? Like the traders in the pits were all keen to make markets when volatility was spiking and prices were dropping?

j_lev2• 11 years ago

I am saying that if you're a self-styled market maker but only bidding for the 99.9% of the time when the market isn't in free-fall then the "social utility" claim is dubious.

People who place stop-loss orders would not do so if they were faced with an order book with no bids. However this is effectively what happens when the market starts to drop and the bids are all pulled.

sneak2• 11 years ago

What you are describing is not open; you went through several legal hoops to be able to participate in these markets. Not just anyone can do what you did.

Bootvis• 11 years ago

Which legal hoops do you mean?

sneak2• 11 years ago

These days, participating in most western bitcoin exchanges requires going through many government requirements such as providing identity documents and going through various arbitrary screening and watch-list checks.

Of course, if any of these public or private databases matches you, it's not really in the exchange's interest to go to bat for you. There are other customers and they do business based on volume.

The basic premise that bitcoin has ushered in this new system of open money is false. Everything he describes is happening on existing, closed businesses.

The fact that the asset he's trading happens to be bitcoin is somewhat immaterial.

Animats• 11 years ago

This "high frequency trading" isn't making liquidity. It's faking liquidity. Buy and sell orders with specific price triggers can be put up and taken down rapidly without ever being executed. Most of this type of trading does just that. The Bitcoin world calls this "fake walls".

If an order couldn't be taken down until N minutes had elapsed, orders would represent real liquidity. You could look at the market and know you really could buy or sell a large volume at some amount. What actually happens is that there are orders that run away if they're close to being executed. That's fake liquidity.

KMag• 11 years ago

A less risky solution would be to have a blackout period equal to something like the same account being unable to place new orders for 30 seconds (minus the amount of time the order was in the market) after cancelling an order.

People make mistakes and you need to let people get out of the market quickly if they do. (People mistakenly transpose the price and quantity more often than you'd expect, etc.) Having a forced cool down period after placing a short-lived order reduces the impact of these sorts of orders without increasing the risk of mistaken price/quantity/side.

I'm not sure if coinbase allows amendments of quantity and price on existing orders, but they should have similar time restrictions on amending up the quantity or amending the price to be more aggressive following a quantity amend down or price amend more passive.

derriz• 11 years ago

There's nothing fake about it. If the liquidity were truly "fake", by definition his orders would never trade which defeats the purpose. The more his orders get hit, the more liquidity his bot provides which also means the more potential profit for his bot. I don't understand why so many love being outraged by even the idea of algorithmic trading without knowing the simplest principles of market making.

IkmoIkmo2• 11 years ago

I think what he's saying is that, say the price is $100 and the daily volume is a few million dollars. And someone places a huge, say $50 million buy order at $98.

This implies to the market that there is a ton of demand not far from the price. It shows support for the price level, it shows demand, confidence in the product for sale, and it shows that the price is not going to drop anytime soon even if many people sell, as there's a big-shot with $50m ready to soak up any sell orders for weeks (as normally, daily volume is only a few million).

That's a signal to the market: there is demand, there is confidence, and there is very little downside risk. If I buy, it's likely the price goes up. If many people sell, a large buyer will soak up the price drop and so my risk is small.

When in fact, the guy who put a $50m buy order at $98 will cancel it as soon as the price drops from $100 to $99. He never intends to let that order execute. It's 'fake demand'. He's got some of the product and he's hoping that his $50m buy order is the gesture that fuels more demand, raising the price to say $105, at which point he sells his product, cancels his order, watches the price drop and buys some more product at cheaper rates.

This doesn't have much to do with algorithmic trading, mostly it's a manual set up and the only thing you program is for a bot to cancel an order you set up once it's close to being executed.

His solution is to say you can't just cancel an order right after you created it over and over again. In other words, he's suggesting that instead of an order book, trades ought to only be allowed to buy or sell options. e.g. if you want to buy $50m at $95, you can sell the option for traders to sell you coins at $95 for some timeframe. As any contract, it can't simply be cancelled, and remains valid for some limited time, which means you can't fake demand as once you express demand, you can contractually be locked in to exercising demand. It's not a great solution, don't get me wrong, but I can see where he's coming from.

zhte415• 11 years ago

If an order couldn't be taken down until N minutes had elapsed, orders would represent real liquidity.

I have nothing to say about HFT, but this is terrible. For meny reasons. Thia is requesting every buyer/seller to be in effect be on a laggy connection. It would not be possible to look at the market and 'decide' as every buyer/seller is on the same lag.

lordnacho• 11 years ago

No, if you never get filled, you'll never make or lose any money.

Consider what would happen if orders had to stay fixed for a while: Any new information would cause the price to change beyond the fixed orders. People with the information would trade the orders, and the MM would lose. So what would the MM do? Naturally, they will keep things wider to minimise the loss. Which is bad for everyone when there's no new information.

Also, consider what you're saying about large orders. The MMs have said "well, given what the market looks like now, we're willing to offer 100 at this price". Anything up to 100 is fine; you will get filled on all of it so long if you send in a buy of 100. If you want more, they have to be able to reassess. If you're just dipping your toes with 10 at a time, they are entitled to guess that you're willing to pay more. If there's 100 shares offered, and someone wants to buy 1000, they are eating all the liquidity and more. This is information, and needs to be reacted to.

It's no different from any other market. If you're at the supermarket and you buy all the milk, the supermarket is more than entitled to say "hey, I think people are willing to pay more for milk" and restock milk at whatever price they think is sensible.

functional_test2• 11 years ago

This is blatant misinformation. Please stop spreading it.

Canceling orders is something _all_ participants do. It's a vital part of risk management and a healthy market. Some markets do have minimum quote lives, but they typically fail as market makers cannot manage risk there, so trading moves elsewhere. The only time I've seen a market place survive adding MQL's is when they were on the order of milliseconds, not minutes.

There are certainly strategies that are manipulative. For example, having a hidden order on the ask and posting an inflated bid then pulling it once your ask gets filled. Since you posted it with no intention to get filled and with the intention of manipulating the price, that's spoofing, and illegal.

But the fact that someone placed an order and then canceled it when the price got close means exactly nothing -- everyone does it as a part of normal business.

JoeAltmaier• 11 years ago

Confused: you describe the practice as both 'spoofing, and illegal" and "means exactly nothing". Which is it? How do you tell the difference from innocent cancellation and malfeasance? Maybe that's what the Op was getting at.

pontifier2• 11 years ago

I built a bot to trade on MtGox just as it was going down. It was exhilarating. I have it trading on BTC-e right now and it's open source. I don't make much from it, but I don't have much invested. It's more of a test.

Unlike the original poster, I'm not afraid of anyone gaming my bot. So far, the bot hasn't made ANY bad trades. It's also not a High frequency trader. Basically it decides what trades would be good for it, and makes a buy and sell limit order, then waits. If one happens, then it cancels the other, and recalculates what to do. If both happen, all the better.

It's based on rebalancing a portfolio. And just the idea that it actually works might be more valuable than anything the bot has made so far.

If people are interested it's at Github.com/pontifier/rebalancer.git

alexchamberlain• 11 years ago

How does your bot determine the ratio, or more precisely the price it uses to determine the ratio?

pontifier2• 11 years ago

It looks at what your holdings are, and calculates what the asset price would be if you were at the ratio you set. It then makes trades on either side of this value. It doesn't even look at a ticker. The only input is whether a trade is made or not, and the balances you have of each currency.

uknj• 11 years ago

I highly recommend the book the author mentioned: Flash Boys, I found it a really interesting read.

kasey_junk6• 11 years ago

I conversely drastically don't. Further, I've yet to meet anyone who is technically knowledgable of HFT systems who does.

See http://kc.my-junk.info/hft-books/ for what I do recommend.

High Frequency Trading on the Coinbase Exchange

Related Stories