Ask HN: How to launch an algo trading side project?

https://news.ycombinator.com/item?id=16376015

by meifun • 8 years ago

58 88 8 years ago

news.ycombinator.com

I've been writing my own implementations and using 20 year historical stock data. Have been test trading to see how my logic is working.

However, I have an issue. When I think I am ready to use real money how do I do this? Do I need to get ETrade or Robinhood to actually start automated trading?

arthurcolle• 8 years ago

have you looked into quantopian?

jason_slack22• 8 years ago

Thanks. I just looked at them. I like their idea of letting you test, but all of my code is in c++. I have 6 apps that need to continually run to fetch new data, import it, create models, make predictions and execute them. I'm not ready to share code, but I could always share how my architecture is works.

elecengin• 8 years ago

Then probably something like IB is your best bet. Do you have margin requirements?

Apologies if you already considered this, but make sure you consider execution costs. Depending on your type of strategy and selection of stocks, it may be a major determinant of the success or failure of the strategy.

rb8082• 8 years ago

http://quantconnect.com is a better alternative for you, take a look.

1001101• 8 years ago

TradeStation. I wouldn't bank on any HFT, unless you're doing FPGAs, and you've got the 'good' port on the switch.

jason_slack22• 8 years ago

I have always wondered if FPGA's are really needed. Can you share more info about this?

jetti• 8 years ago

Are you doing HFT trading? If so, then the FPGA could help. Are you just doing plain ol' algo trading then you wouldn't really need it.

lordvon3• 8 years ago

Do you have any experience with InteractiveBrokers?

godelmachine3• 8 years ago

May I ask what do you mean by good port on the switch?

jason_slack22• 8 years ago

I think he/she is referring to co-locating servers at exchanges or using companies that are already hosting their servers there.

1024core• 8 years ago

1. Only use money that you can 100% afford to lose.

2. Once (if..) you've made some profit, take out your initial investment and now you're gambling with free money.

jason_slack22• 8 years ago

Exactly. Thanks for mentioning this. It is important to get the initial investment back into my bank.

dkyc• 8 years ago

"Once (if..) you've made some profit, take out your initial investment and now you're gambling with free money."

I never understood this sentiment, and I'm surprised to see it so often on Hacker News. It's irrelevant whether the money came from profits or initial investment - money is money? There is no need to distinguish the both, it's purely psychological.

lev99• 8 years ago

It's a classic example of loss aversion[0]. In my mind the real loss risk of this venture is the opportunity cost spent developing the algorithm. Many people that fail to build trading algorithms would have been very successful if they invested their time/expertise doing something else. The person stands to lose not only his cash investment, but all of the cash he could have received if they focused their efforts on a less risky venture[1].

[0] https://en.wikipedia.org/wiki/Loss_aversion

[1] I'm all for risk, but Vegas has better odds.

csomar2• 8 years ago

It makes no sense. That's why I suggest you look for advice from seasoned guys. The average guy is very emotional to trade regardless of their technical and financial background.

There is zero reasons to lose money or carry on running a losing strategy. If you are trading professionally and quantitatively, you are runnings lots of positions. Unless you have a technical issue, you should not worry about losses on one or a few positions since you only care about what your average is doing.

That assuming your models work but you should not start trading with real money before you are certain of that.

linsomniac• 8 years ago

The easiest way to make a small fortune in trading is to start with a large one. :-)

HFT is like picking up pennies in front of a steamroller.

jason_slack22• 8 years ago

HFT is like picking up pennies in front of a steamroller.

here's to hoping I can stay one step ahead, lol. It's risky.

cdnsteve• 8 years ago

Check out https://www.quantopian.com/

arca_vorago• 8 years ago

I strongly second this. I've had some other projects get in the way, but I signed up for quantopian and have been very impressed with the things I have gleaned from reading others algos and postings. If your algo is really that good you get a cut.

That said, I do think ultimately you'd want to find something with a fast api for yourself lately.

Oh, and for entertainment purposes read /r/wallstreetbets

LiamPa• 8 years ago

+1 Read the zipline source, I learnt a lot in how to structure trading code / solving common problems.

patatino3• 8 years ago

Check out this podcast. Dis guy had a algo trading as a sideproject (now fulltime): https://www.youtube.com/watch?v=Co_GYku903k

jason_slack22• 8 years ago

Awesome, I'll watch it later today.

SirLJ5• 8 years ago

Looks like this guy is looking to create a Quantopian clone, so in the end it feels like the whole interview was a bit of advert for the signal service to come...

In fact all podcasts of this guy are like than, a lot of charlatans and scammers are appearing on the podcast all the time, probably 99% of the guests...

patatino3• 8 years ago

I listened to ~10 podcasts and most people are very humble and talking about a lot of mistakes they made and I learned a lot. I cannot share that 99% of the guets should be charlatans or scammers. Did you listened to most of the episodes?

wpasc2• 8 years ago

1. Read Flashboys

2. a) Realize how CRAZILY far many HFT firms have gone to inch out literally every single microsecond and achieve economies of scale with billions in assets

2. b) individual user-scale tools like ETrade won't get your trades in fast enough/any cost per transaction fees will erase any gains

3. Invest said capital in something like Betterment

If your model has been truly beating the market for a sustainable amount of time, I apologize for my comment and go all out :)

jason_slack22• 8 years ago

I've been ahead for 2 months in just testing my models.

Edit: granted not FY ahead.

wpasc2• 8 years ago

Cool! Pretty cool to see some alpha generated by a single IC. I still recommend Flash Boys as a book though about HFT, very interesting and a good read

topbanana• 8 years ago

HFT is only one type of algo trading.

You can make money with simple trend following algos that execute perhaps only a few times a week.

To the OP, if you aren't sure about market execution, it makes me think you haven't modeled for slippage. Make sure you do, as that nudges many profitable algos into the red.

cjalmeida• 8 years ago

And maybe trading costs.

mseebach• 8 years ago

HFT is a subset of algorithmic trading, but algorithmic trading isn't all HFT.

For an example, a value based investment strategy can be implemented as an algorithm.

rco8786• 8 years ago

To other folks in this thread - your warnings are justified but please stop conflating “algo” and “HFT”.

iothetiger• 8 years ago

I went with Interactive Brokers and their python API.

paulgb• 8 years ago

A friend of mine did this and had the algorithm output the trades and then he made them manually. If it trades too quickly for that to work, trading costs will likely be an issue for you anyway.

Remember hindsight bias, and as others have said, don't invest more than you can lose.

mring33621• 8 years ago

This is how I did it. My problem was that I listened to my algos for entry points but let my emotions take over for exits, so I often left money on the table.

jason_slack22• 8 years ago

What you did is a natural instinct/emotion! I can see this happening to me also.

adjkant2• 8 years ago

Consider cryptocurrency over stocks, as it comes with exchange API's and low/no fees.

toomanybeersies• 8 years ago

Crypto doesn't have 20 years of historical price data though.

adjkant2• 8 years ago

True, but there are many other market conditions in play there to be exploited, and a year or more of data is still useful.

wruza• 8 years ago

Isn’t history-based trading a myth? I mean, technical analysis of markets in general.

kirubakaran• 8 years ago

I agree. There seem to be several good leading indicators for cryptocurrency prices, especially for new altcoins. I'm building https://coinparse.com/ exactly for this. I wrote a post on this recently: https://coinparse.com/articles/omg-anatomy-of-price/ as an introduction.

lordvon3• 8 years ago

I have only tried one algo trading platform, Interactive Brokers. API documentation: http://interactivebrokers.github.io/tws-api/index.html#gsc.t...

Their live market data is gated at 250 ms, so definitely not for HFT. Market data subscriptions are only $5 / month (US Equities, and others I don't remember), and commissions are low.

You can program in C#, Java, VB, C++, or Python. I use their C++ platform. It is not particularly pretty, but it seems well-tested and works. Perhaps most importantly, they have an awesome support phone number. They are responsive and very helpful.

usgroup• 8 years ago

Depends...

What instruments are you trading?

What frequency?

What size?

Why does it need to be automated?

a_c• 8 years ago

Mind sharing where you get your market data? Are you using a paper account that you can simulate the trades? what markets are you participated in?

Apologies for the burst of questions. Am interested in testing out some ideas too

jason_slack22• 8 years ago

Historically I have scraped data, but now: http://eoddata.com/default.aspx

turc16562• 8 years ago

Be careful. EODdata has a lot of data issues. Seriously...a lot. I would recommend checking out Alpha Vantage. It's free and I didn't see any of the issues with their data that I saw with EODdata.

boulos2• 8 years ago

Interactive Brokers (IB) is your best bet. However, I’d like to reiterate as others have stated that you need to be prepared to lose it all (and moreover that you likely need much more capital than you realize to stay solvent on down swings).

Don’t take this the wrong way, but it seems like you haven’t done a lot of research into this: Interactive Brokers is one of the first things that comes up when you search on Google (etrade, Robin Hood or any other retail trading just isn’t credible).

Finally, is your 20 years of historical data for just open/close? Is that your execution plan? (Otherwise you can get really tripped up as you move to intraday trading)

jason_slack22• 8 years ago

I know about IB but initially I thought that I couldn't use their API unless I also ran on their platform. Meaning I can't use my own VPS, etc.

I have been using 1 minute bar data.

lordvon3• 8 years ago

I don't know about setting up VPS, but the standard way to use IB is to run their main client (either the GUI-based "TWS" or the GUI-less "IB Gateway"), then connect to the main client. Your binary is totally separate from the main client, which you do not modify.

Have you looked at Lightspeed Trading?

gilj2• 8 years ago

When using IB, you can trade on your own machine/vps/whatever. I suggest you do this, and set it up such that you trade on an IB paper account, make sure everything is running correctly and in line with your backtests, and then start trading with some real money. Good luck!

Sukotto• 8 years ago

You can use their API from your own system.

https://www.interactivebrokers.com/en/index.php?f=5041

I strongly urge you to paper-trade the live market before risking real money. https://www.interactivebrokers.com/en/software/am/am/managea...

kerberos84• 8 years ago

If you are not planning to do HFT then let your system create signals and send them to your e-mail. Then buy/sell manually.

crimsonalucard• 8 years ago

What if I want to do fundamental analysis? Anyone know of any cheap or free data sources that have parseable financial statements?

someguy_lvl90012• 8 years ago

I am working on the same thing. I'm still in the model building phase and I still need to complete my backtesting framework. You seem to be ahead but I've already looked into the trading API process.

Go with whatever broker offers a trading API obviously, and has a fee structure that matches your trading strategy. If you can afford to lose a bit on the trade through commissions and your strategy is swing trading then you can pick pretty much any broker. If you need specific fill times and commission percentages before your strategy pays off then you are stuck with Interactive Brokers ($10k minimum[1], low comms, fast exec) or Robinhood (low minimum, free comms, slow exec). And technically Robinhood doesn't have a public API[2] so you would have to do a little reversing of their app's messaging. And then there's terms of service... yeah. Probably not a big deal for a small fry like yourself.

Next would be testing. You've got isolated tests with historic data but try get a broker that has a paper money version. TDAmeritrade's paper & api is good (but they are expensive). To be clear you would be using your algo to trade in a paper money account. That lets you get more realistic trade tests and if you blow something up you are still ok!

Keep in mind Robinhood is free because they are paid to route your orders to hedge funds and they are re-ordered in the book. You will get ok price execution through them but never the best. One thing you should include in your backtests/simulations is order execution delay and fill price variance. If you place market orders they won't always fill at the quote price. If you place limit orders they may take a while to fill (or not at all) unless the price is within the other side's order prices.

Last is the pattern day trading rule. Make sure you have a strategy that won't make more than 3 day trades in a 5 day window, otherwise your account might get suspended for a few months. It's a pile of horses but you can get around that by simply having more money in your account. > $25k will buy you unlimited day trading privileges.

Just curious what your strategy is based on if you don't mind giving a few details.

Good luck!

[1]https://www.interactivebrokers.com/en/index.php?f=4969 [2]https://support.robinhood.com/hc/en-us/articles/210216823-Ro...

testeroniblla• 8 years ago

One suggestion to avoid the PDT rule (and in general be more efficient with your capital) is to look at futures. For instance rather than trading SPY trade ES. ~8k per contract minimum to control ~140k (50*Index Points) of the SP500 and no PDT.

someguy_lvl90012• 8 years ago

You say more efficient, I say leveraged. Potato Potato. :) Yea futures are much more interesting in my opinion and no PDT is way better!

jason_slack22• 8 years ago

Wow, thanks for this advice. I think I have a few questions. Let me read more carefully to be sure later.

kolbe• 8 years ago

How often do you trade and how important is fast executions? I've noticed many programmers try to shoehorn in execution algorithms into a strategy that doesn't need it. If you can run an analytics program that just sends you a trade alert from which you manually execute, that will save you a lot in systems risk.

Also, the fact that you're asking this question tells me this is very very unlikely to be a good idea. Be careful.

jason_slack22• 8 years ago

My thought is to trade as infrequently as possible, sort of batching trades to do all at one. Nothing is automated for me yet. I am just manually reviewing my output.

SirLJ5• 8 years ago

I am glad you ran with my idea and found an edge pretty quick!

What is your average yearly return on the 20 years back testing?

jason_slack22• 8 years ago

Your advice was absolutely amazing and cemented my desire to continue! Now I need to figure out how to actually execute. If you have any advice! I keep staring at a list of possible transactions and manually checking how my trade might have turned out and keeping a gain/loss figure.

I was hoping you would chime in on this thread. I was not so comfortable with the work I was doing and felt like my direction was wrong and I was questioning why I was even going to try this. Your advice helped me decide to continue.

SirLJ5• 8 years ago

Hey Jason, I am glad I could help!

Before you start with the execution, make sure you did use the 20 years of data, go and back test all the years, they encompass few market cycles, so you will know how your strategy will perform in bull and bear markets.

For the execution part, when I started, I had an ssh session open to my server with the output of my bot running on the screen and had it configured to chime with something like” EXECUTE buy 2000 HYZ at $65 stop 56. I did it manually for a few weeks until I made sure everything worked properly and then did the order execution with the API. Interactive Brothers are a good choice with a good API.

I would stay away from Quantopian and other platforms that are holding your code, because this is the prefect target for a hacker and the perfect crime – nothing to steal or sell, they will just copy your winning strategy…

thisisit• 8 years ago

If I may, what is the timeframe for your algo trades?

If it's short one like say in minutes, then you might want to test it with small amount of real money before going all in. The reason is orders fills are imperfect. So, cost price you had during backtests vs realtime might differ a lot.

jason_slack22• 8 years ago

This is the hard part. How much money am I willing to part with :-) Some of these places I think need $25k in an account.

boulos2• 8 years ago

Actually, that threshold is an SEC one for being a “frequent trader”. So anyone who doesn’t is just not telling you that you’ll need at least 25k (and IIRC, at all times) to make the volume of trades you want.

tty72• 8 years ago

Robinhood have an unplublished api. There are hard limits on the number of trades you can do. But i would use that

jason_slack22• 8 years ago

Can you share more about this? IIRC web isn't ready yet so mobile only.

tty72• 8 years ago

https://github.com/Jamonek/Robinhood I used that a few months ago, you might need to sniff the traffic if there has been changes. it wasnt too difficult

1e-9• 8 years ago

The Interactive Brokers (IB) API would probably be your best option for trading on the side.

Here are a few words of advice regarding IB:

- Thoroughly read and understand the IB documentation. - Be sure that you never get close to any of IB’s risk limits that can result in the automatic liquidation of some, or all, of your positions. - If you don’t want to put up enough cash in your IB account for day trading, be sure you don’t exceed the trading limits that will get you classified as a day trader and thus prevent you from further trading until you add more cash. - Be aware that the IB market data can have significant inaccuracies and/or delays.

Here are a few words of advice on automated trading:

- Thoroughly understand the exchanges that handle the instruments you want to trade. Each exchange has online documentation explaining details of traded instruments, matching algorithms, pricing, and market data. Even though you won’t be accessing the exchanges directly, these details have an effect. - Thoroughly understand IB’s order routing and how they construct their market data (if you use it). - Just because your historical data says an instrument closed at a particular price, that does not mean you could have actually bought or sold at that price with IB or even with direct market access. The same is true of tick data constructed at 1 minute, 1 second, or even millisecond resolution. Trading simulations can be terribly inaccurate if you don’t have a reasonable approximation of the exchange order books, the exchange latencies, IB’s latencies, and your own trading system latency. To counter this, you need to be extremely conservative in your assumptions about getting an order filled at a particular price. - Even with accurate market data and exact latency measurements, predicting future profits by backtesting over historical data is perilous. I advise having a strong background in the statistical/machine learning methods for constructing and analyzing appropriate training, testing, and validation sets. - Don’t make a trade just because some analysis suggests the market will go in that direction. Doing this will probably get you hurt and it harms markets by pushing them to overshoot or undershoot the correct prices. The trend is your friend until he turns around and stabs you in the back. Make a trade because that’s where the market should go and probably will go eventually.

My best advice for trading profitably is to do something to improve a market. Focus on areas that can benefit from your strongest personal advantages (mathematical, technical, domain knowledge, etc.). Gain an understanding of the exchanges and instruments that allows you to recognize ways to make prices more accurate. This improvement could be in an absolute sense for individual prices or it could be in a relative sense in terms of how instruments should be priced with respect to one another. It could also be in terms of adding liquidity (offering to buy or sell) where needed and justified. Improving markets improves our economy by helping it to better prioritize resources and limit risks. That is a valuable service and the market will pay you for it.

lee101• 8 years ago

As others have said checkout cryptocurrency, im founder of https://bitbank.nz providing ai forecasts, charting and bulk data for backtesting bots. Checkout our blog at http://blog.bitbank.nz for info on how to easily develop profitable trading algorithms from some of our open cryptocurrency data. Also checkout our GitHub

godelmachine3• 8 years ago

Since this topic is trending, I would like to grab this opportunity to ask if anyone's knows any material on how FPGAs are used in HFT/ Quantitative trading? Any books/ web sites / blogs?

Any pointers would be highly appreciated?

julox• 8 years ago

for development in Python you can use https://www.backtrader.com with simulations and live trading

csomar2• 8 years ago

You said you have been test trading. What does that mean? Where are you testing? Which platform are you using?

Ask HN: How to launch an algo trading side project?

Related Stories