Designing and Building Stockfighter, Our Programming Game

http://www.kalzumeus.com/2015/08/20/designing-and-building-stockfighter-our-programming-game/

by jsnell • 11 years ago

365 155 11 years ago

jsingleton2• 11 years ago

This sounds really fun but that is one long post/email. Almost 8000 words! Will read it later though.

Looks like the schedule has already slipped. The email said "2~3 weeks of today" but the blog post says "3 weeks of today"! :P

tptacek21• 11 years ago

Imagine that! An ambitious development project with a constantly slipping schedule! Maybe we're the Duke Nuke'Em Forever of CTFs! :)

jsingleton2• 11 years ago

I know how it is. At least you didn't compare it to Daikatana. :)

BillyParadise2• 11 years ago

I have ADD and was able to read it.

lifeisstillgood5• 11 years ago

Well apart from "there goes October, dang I had real work planned", I am really welcoming this game. I would normally have jumped all over their compiler "tree", but I have been in my first Finance contract this year and I have not yet understood HFT.

Edit: this post got longer than planned - apologies - but this HFT / algo thing has been bugging at me for ages and this seems a likely thread for knowledgeable folks.

The references to Flash Boys worry me slightly - Lewis sold a false scenario there - it's pretty clear that no fool is going to see Bob buying a million Apple shares in London, then zip over their speed of light towers and front run Bob in Frankfurt for another million Apple shares. Who knows if Bob wants a million? Flash boys it seemed relied too much on a protected type of trader (market maker) getting roasted by the emergence of real competition - the "algo" traders.

It seems incontrovertible- equity market spreads have fallen by about 5/6ths in the past decade (no wonder Lewis' hero trader thought someone was stealing from him) but ... How and Why?

what algo traders actually do is a bit of a mystery to me.

The basics I kind of understand :

There are correlations between shares - spot those and you can make money. Let's say oil goes up, car manufacturers will go down by some amount. Short them and make money. The first time people did this they probably thought they had found a money printer. But then someone else starts playing the same algorithm - so you have to get your orders in taste than that guy - pretty soon you are worrying about the speed of light over glass.

But other correlations exist and get spotted and tried out and ... Well it seems a bit of a weak business model to me to live in P of .95.

So the business model of HFT is a bit weird, and the way their actions feed into reduced spread / liquidity is a bit unclear - but anyway, I look forward to totally screwing up my social life in a few weeks.

Anyone who can enlighten me, please shout.

tptacek21• 11 years ago

Trust me, Patrick agrees with you about Flash Boys. If you haven't yet, read the followup/critique book he cited, even if you haven't read Flash Boys. It's fantastic and chock full of interesting technical details. It's way nerdier than Lewis' book.

lifeisstillgood5• 11 years ago

That's the "Not so Fast" book? I actually read that before flash boys. Somehow the narrative from flash boys had penetrated the NYT/Economist meme-spreading network and I had accepted it at face value.

However the microstructure book looks ... Deep - but I guess I need to decide if I am in or out :-)

jndsn402• 11 years ago

I have read Flash Boys but not the books written in response.

How do the other viewpoints explain why the IEX exchange was created and designed specifically to stop HFT? That is the central narrative of the book, and in Lewis's telling it makes complete sense as a way to save the common man from the evil HFT traders. But if Lewis got it all wrong and HFT is perfectly fine, why would a completely new exchange be set up to stop it?

kbenson3• 11 years ago

A few years back there was a HFT guy that posted a few blog entries describing (and defending through it's liquidity creating properties, IIRC) HFT setups. I don't have the article, but I imagine both tptacek and patio11 (and a slew of others) have a good idea of what I'm talking about, and may be able to dig up a link. You might find it interesting (if it's not already entirely covered by Flash Boys, which I haven't read).

patio1111• 11 years ago

Was that HN's own yummyfajitas?

https://www.chrisstucchio.com/blog/2012/hft_apology.html https://www.chrisstucchio.com/blog/2012/hft_apology2.html

ctz• 11 years ago

I do wonder about the logic of collecting the email addresses of interested parties, then sending them 2000 and 8000 word (respectively) essays. Do people actually read and value such long emails? Do you not lose a bunch of on-the-fence users each time you send them an email which isn't immediately useful or actionable?

Then again, I know very little about running a startup; patio11 has a track record. Maybe this works as a strategy.

kbenson3• 11 years ago

It may work to radicalize a certain segment. I've noticed that some segment of those that are eagerly anticipating something find that the more information they get about it, the more they anticipate it, and the more eagerly they evangelize it to others.

I hadn't heard of this, or microcorruption, before the prior blog post regarding writing the emulator. Then I started microcorruption, and it clicked, and then I spent the next few days talking about both microcorruption and starfighter to anyone I thought might understand my eager rambling.

Let's just say I'm looking forward to reading this...

patio1111• 11 years ago

Do you not lose a bunch of on-the-fence users each time you send them an email which isn't immediately useful or actionable?

That's a number which is trivially calculable. I don't know what it will be yet, but if you asked me to pick the over/under, I'm in for 80 out of 16k people on our list at present. An unsubscribe is not "I hate you guys and wish you would die in a fire"; it's the email equivalent of a browser's back button. "Not for me, not right now." Many startups let each unsubscribe punch them in the gut. Don't do that. The only way to avoid unsubscribes is to avoid email. You profoundly don't want to avoid email.

We've sent ~2 emails since... March? If I were a consulting client of myself, I'd have written somewhere between one and two dozen.

Is this email going to produce business results for the company? Hard to say. Wild guess? I get a contact in the next hour from one of the companies who has a stalled LOI saying "Oh great you're shipping let's get that signed." Depending on which company that is, the economics are worth hundreds of thousands of dollars.

Many professional email marketers strongly disagree with writing long emails. I will chalk that up to a respectable professional disagreement. They work very well for me, by any trackable metric and by most of the intangibles. I've never felt like I created much value with the little 125-word-and-a-picture squibs that the industry blasts out by the hundred million; I know (because people tell me) that some people like my writing rather a lot and are disappointed that I don't hit them with 8k words every Friday. As it happens, a portion of Starfighter's list is folks who are in exactly that bucket.

Does that answer the question? Not trying to be very defensive about this, just trying to help other startups out. The modal startup sucks at email marketing. It is, far and away, the highest ROI channel you have and probably the easiest to use well. You should send more email. You should send better email.

lifeisstillgood5• 11 years ago

Agreed - it just feels nicer to read fewer longer emails - sort of New Yorker vs CNN tickertron.

Of course writing them is a pain so it's an investment. Considering I just gave a friend the "you must write long articles and build an audience" advice, it's amazing how badly I take my own advice...

jc4p• 11 years ago

Just anecdotal evidence: The two e-mails y'all have sent since I signed up have caused a pavlovian response, I literally left my office and sat down on a couch to read your last e-mail the second I saw it in my inbox.

grayclhn3• 11 years ago

One more anecdote: I read the email immediately, shared it with anyone I thought would be interested, and started to count down how long it would be before I could play the game with my 6yo daughter. So, um, it's working for me so far.

thenomad2• 11 years ago

As a data point - I'm one of the people Patrick mentions. I like his writing a lot and would be very happy to see an 8k word essay from him every Friday.

He's on my very short list of 3 or so people who do email marketing particularly well.

One of the other people on that list, incidentally, writes 200-ish word emails every day. I read everything she writes too. There's not a single optimal solution here.

(I'm also very much looking forward to playing the game after reading that post. Given I'm not a professional programmer, I have very little desire to become one, I'm very happy in my current jobs, and I suspect Stockfighter is going to kick my ass until I can wear it like a hat, that's of dubious business value to Starfighter, but I guess I'll probably also recommend it to some folk who may be of more value...)

juletide3• 11 years ago

Counter-anecdote: I got the email, was excited to see that Starfighter was launching soon... but was annoyed that I couldn't quickly skim the rest of the email to get the "tl;dr" version (the basics of what the challenge will involve, or if there's anything else I need to know about the launch, etc). I've been cutting down on my RSS reader lately because there are a lot of things I want to read and there's not time to read them all; I wish the emails were more respectful of my time. Just including a clearly-marked summary, followed by the long version, would make me far happier.

fmstephe• 11 years ago

Keep sending your long emails. I have butterflies in my stomach. Very excited.

mark212• 11 years ago

great response, Patrick. And count me in with the folks who love the longer emails. Your writing is invariably thoughtful and thorough.

I think the long email is a great example of how it's better to have 1,000 people who love you rather than 100,000 people who like you.

amalcon2• 11 years ago

For what it's worth, I just signed up for the mailing list. I did this because I want to read articles like this that are mostly high-level technical stuff with a dash of business details. That sort of article is the reason I pay attention to HN.

I'd personally be fine with first finding out about Starfighter itself well after launch. I'll probably check it out, but it's not like I anticipate spending hours on it every week. On the other hand, your blog takes only a few minutes to read, and has great signal:noise.

wglb• 11 years ago

Many professional email marketers strongly disagree with writing long emails.

I might venture an opinion that this is likely because they don't write as well or as interestingly as you do. By a long shot.

yellowapple• 11 years ago

An unsubscribe is not "I hate you guys and wish you would die in a fire"

That's not universally true ;)

JacobAldridge• 11 years ago

80 out of 16K is bang on the half-of-one-percent benchmark I've always used for Unsubscribes. If it gets too close to 1%, you'd have cause for concern, and if it gets too close to 0% then you have a spam filter issue that stops a lot o delivery.

svec• 11 years ago

I read it, and look forward to more!

I signed up for a tech-heavy game-ish thing, so I expect (and welcome!) tech-heavy content.

thebrettd• 11 years ago

I find the emails verbose, and don't really care for the for the prose. That being said, at least in my head, when its public i'll say 'here goes'.

Sukotto• 11 years ago

To be one data point.

I read everything Patrick (and Amy Hoy and Brennan Dunn and Ramit Sethi and a few others) emails me. All tend to be long form (typically x,000 words) essays and/or sales letters.

My ROI on time spent reading has been totally positive and I have no plans to stop.

radmuzom2• 11 years ago

Interesting sentence in the article - "Trading systems touch low-level coding, networking (all seven layers of hello OSI Model), APIs, end-user UI, databases, embedded systems, enterprise web frameworks, cutting-edge programming language research, Big Data, state management, etc etc.".

However, whenever the topics of programming in finance come up in HN, I see a general tone of disdain somewhere along the lines of "why do you want to waste your life in finance when you could be starting a start-up in Silicon Valley?". Of course I am generalizing a bit here, but I have always felt that finance provides incredible challenge to technical people - especially those with a background in mathematics or physics and interested in programming.

radmuzom2• 11 years ago

Oops commented too fast without reading the entire article. It says later - "I hope people will, in the course of robbing them blind, actually learn what major Wall Street institutions do for a living and come to a more nuanced understanding of their position in society."

rm_-rf_slash• 11 years ago

When I read the title "stockfighter," I thought it would be a game about joining a tech company 10 years past the IPO and you have to work your ass off for .000001% in options that'll take a year to afford once you factor in rent.

lifeisstillgood5• 11 years ago

I have only a years perspective on the sector now, but I think finance has started down a road all sectors must tread over the next decade or two. We will not escape software eating our world, and while most banks have had too much money and so not confronted many of these upcoming realities, they have had twenty years of faster and faster software-isation- and I can reasonably state that finance are mostly software houses that think they are banks.

If you can't code, your future there is bleak. If you can its not that much better. And this is where we are all heading.

Yes there are challenges - but it's not often technical but organisational, project, people and process challenges that are the main issues. Those interesting challenges for good STEM people will be in transport, farming and pharmacy too by 2020.

Wages might be lower :-)

davidw2• 11 years ago

I'll admit to not finishing that - it was cool to see Patrick geeking out about the impressive tech he's been working on, rather than marketing, though.

Good luck, guys!

lmorris84• 11 years ago

We solved this in a roughly similar way to a real stock exchange: the order book is persisted entirely in memory, in Go.

What happens if your process crashes? how do you rebuild the order book?

Great work btw guys, can't wait to get started!

otoburb• 11 years ago

Maybe that's one of the attack vectors we are supposed to discover ourselves; trigger a kernel panic?

patio1111• 11 years ago

Please don't trigger a kernel panic in one of the stock exchanges; you're on a server with many other actual humans even if they're not hooked up to your stock exchange. More guidance on "bugs you should not attempt to exploit" versus "Things which are totally in scope" at a later date -- Thomas is writing it.

bhousel• 11 years ago

Dumb idea. If you can trigger a kernel panic, you can do things much more interesting than triggering a kernel panic.

patio1111• 11 years ago

At that moment, we can't recover from that and players connected to one of the crashed exchanges (1 server process: N exchanges) are told to restart levels, which is not an ideal outcome but it's essentially free. I hope to eventually be able to do something like use our persisted-out-of-memory (e.g. to NSQ) tapes of the orders, bring up the crashed exchanges, and fire every order back into it in order, which (since it's a stock exchange) should result in the same orderbook it crashed with minus whatever we lost during the crash.

After we can do that reliably, then I will strongly consider making an advanced mode level where a crash / market reset / rollback of trades is a planned feature of the level (they are absolutely a thing which happens in real life) and dealing with it from the player's perspective is the main task. (e.g. What would you do if you got a message from the stock exchange saying, basically, "We lost data. How much? Umm, not sure, the last orderId we have is X. Orders after X are no longer on our books but, um, I guess there exists the possibility that they will nonetheless settle successfully if they matched before we went down. ... Good luck to you!")

DavidPP• 11 years ago

For other readers who aren't aware of that pattern, it's called Event Sourcing. (currently implementing it using Firebase as a combined message bus + storage)

Ref: http://ookami86.github.io/event-sourcing-in-practice/#title....

inconshreveable• 11 years ago

I imagine you've read it, but if you haven't, you may want to read about the LMAX architecture (from which they created the well-known Disruptor) because they dealt with this exact same problem: http://martinfowler.com/articles/lmax.html

mwcampbell• 11 years ago

Looking forward to figuring out the REST API for the GM server, then writing a command-line client for that and the exchange itself. Then, presumably, we can play the trading tree of Stockfighter mostly or entirely from the terminal.

Chirael• 11 years ago

Maybe it's from the length of the message or the general impressiveness of the project, but the first thought that came to mind was, "Wow, I'd have to be unemployed to have enough time to work on this - there's no way I could tackle these challenges in the real world otherwise."

Worse, if this is something that's designed to showcase one's technical skills to potential employers, I don't imagine there's a way for the system to distinguish between "took a long time solving challenges because just not smart enough" vs. "took a long time solving challenges because have a day job and a life outside of work/tech".

Since this is designed, essentially, to showcase how smart someone is to potential employers, assuming solving time is a factor, it's almost better not to compete until you know you'll definitely have lots of time to devote to it, because otherwise there is the risk of taking too long and looking not-good.

I'll add it to the long list of "things I really, really wish I had time for" - or I'll assign it to clone #1023 once that becomes viable :-/

tptacek21• 11 years ago

If we get this right, it will hopefully be the case that we eventually have so much content that most people won't ever get through all of it.

I don't think you have much to worry about in chapter 1, though.

patio1111• 11 years ago

We want it to be playable by people who have demanding commitments like families, jobs, etc. This affects our level design (the first six will actually time out after 30 minutes; they're beatable with the right script in seconds or minutes after you get it working or by hand in ~10 with play at least as good as mine) and how we go about executing on the business.

Ultimately the mechanics of our matchmaking come down to "Starfighter spits a heck of a lot of signal about you at Thomas/Erin/Patrick; we try to decide if it makes sense to introduce to to clients and, if so, to which." All of us are working parents. We will certainly not penalize people for prioritizing piano lessons over being one of the first 5 people to crack a new level.

I won't commit to "I'll never look at time between a level is first opened and when it is solved" but, as somebody who has fed his family with code for as long as I've had a family, that strikes me as being among the least interesting things I could possibly learn about an engineer's ability.

thenomad2• 11 years ago

Presumably you're allowed multiple tries, on the challenges that time out?

jscottmiller• 11 years ago

After para 1, I was going to ask "what's to stop someone from tossing a solution in a pastebin and sharing it around?" Related, "what's to stop someone from sharing the questions ahead of time?"

From the second, it sounds like you guys will seek out the top performers and weed out the bozos. I like it! Are you looking at a traditional recruiter fee model or something else? Apologies if this is posted somewhere else.

trentmb• 11 years ago

I'm dumb and slow- can I still submit a solution to know if I'm correct, even if I get a dingus label?

chc• 11 years ago

I think you're looking at Starfighter as a much more adversarial thing than it's designed to be. From what they've said, it's my impression that it's supposed to be a fun challenge that demonstrates marketable skills, not a pop quiz. I don't think they intend to grade you on whether you spend all your time on it or do it casually or whatever.

Swizec• 11 years ago

"took a long time solving challenges because have a day job and a life outside of work/tech"

What's the problem with just saying that? "I worked on this during my commute to work" or "I did this cool thing on my weekends" or "I tinkered with this in the evenings"

There is a lot more you can do with 1h per day than you think. If you're consistent.

Hell, I am currently making decent money on the side with book sales by working on writing and marketing for about an hour a day. My biggest problem has gone from "Fuck I have no time" to "Fuck, I've run out of ideas to try".

cpach• 11 years ago

”There is a lot more you can do with 1h per day than you think. If you're consistent.”

I’ll second that. This fact is a great opportunity e.g. for people who get bored on their train commute or similar.

grayclhn3• 11 years ago

There is a lot more you can do with 1h per day than you think. If you're consistent.

Or, for some of us, there's a lot more you can do with 1h per day than you think, if you're willing to let that one hour consume your thoughts for the rest of the time you're awake. That's much more my concern than anything else --- when I'm at work, I need to think about work.

dre85• 11 years ago

I felt the exact same way when reading the email. To me it sounded like each level is a sizeable/realistic project for which most companies would probably pay people to develop.

tptacek21• 11 years ago

You found us out! This is actually an evil scheme we've cooked up to provide YC companies with free trading platform implementations crowdsourced from unwitting Starfighter participants!

netcan• 11 years ago

Hiring is an incredibly difficult problem to solve at any scale, though it's different on different scales. As times goes on my impression of how hard the task is keeps shifting upwards.

It's a huge deal for both parties. I think that if there is a way to spend money and/or time to solve it, both sides can come out on top with room to spare.

There's plenty of evidence that both sides are willing to invest a lot in it. For employers that's easily measurable in fees they pay for recruiting services (which usually suck), for referrals and such. For Applicants, it's harder to say. But, people do whole degrees and certifications in order to appeal to employees. Often that stuff is at a total tangent to acquiring useful skills or pursuing interest. It's basically time an money invested in being (more easily) hirable.

Basically, anything that can improve hiring meaningfully may be worth this kind of big investment for an employee as well as an employer. One way it could pay off is higher salary. A higher quality, de-risked hire is worth more.

I think one of the things that rely needs to happen though is that applicants need to be able to evaluate their chances. Spending 3 days interviewing may be no big deal in the context of a job that will last 3 years, but it is in the context of some unknowable probability of getting said job. A third party would probably be helpful here.

soham• 11 years ago

Not exactly what you are looking for, but maybe close; check http://InterviewKickstart.com.

I believe that most engineers fail in interviews not because they are incapable, but because they underestimate the prep required. And prep is difficult for experienced engineers because the employers expect a lot of academic stuff which they have forgotten and need to refresh in a short time.

We hence designed a solid program, which takes you through interview important CS in less than two months. We are having fantastic results with that. Good Engineers who would normally not crack into stellar companies, are now cracking in.

This way, candidates don't need additional expensive degrees and employers waste less time hiring . At scale, this is one way to fix the hiring problem.

minimax2• 11 years ago

My order came back saying that I had bought 0 shares. And the quote from the exchange now read “500 shares available at $120.”

Maybe I don't get the full picture, but it sounds like you have to poll via REST API to query the book? Modern exchanges just give you a feed of updates (adds, deletes, trades), so you don't run into situations like this. You can just check the feed to see what happened.

patio1111• 11 years ago

The feed is hooked up via a websocket to the UI, but not persisted anywhere on the client side. From my eye's perspective, the cancel never registered. If I had had wireshark open I would have seen it. (I encourage folks who understand why this is not desirable to build their own trading UIs using our provided e.g. websockets.)

pjungwir• 11 years ago

Patrick, you should just admit it that the reason you are starting off with a financial theme is so you can include a level about breaking into a Bitcoin exchange. :-)

More seriously, to somewhat address the "8000 words?" folks, the runup to Starfighter reminds me a lot of StackOverflow. Both seem like projects that would have flopped if not for the creators' existing audience and great care in cultivating excitement by explaining in depth the need, the goals, and the philosophy of the project. It'd be interesting to revisit the early StackOverflow marketing and compare the two more closely.

listic• 11 years ago

There's a lot of things going on here. But what is the point of Arduino?

tptacek21• 11 years ago

AVR. Not Arduino.

Starfighter is the successor to Microcorruption, a microcontroller-based memory corruption CTF.

We wrote Microcorruption to provide hands-on learning for memory corruption exploits.

We used microcontrollers instead of full-featured CPUs so we could (a) lose most of the irrelevant details of operating systems and (b) fit the whole memories of thousands of emulated CPUs into a single server process. We also liked the idea of leveling the playing field, so that pros with large libraries of X86 tools didn't have too much of an advantage.

Microcorruption used MSP430. We picked MSP430 because it's the simplest, most elegant MCU instruction set for which GCC support was trivial.

Starfighter uses AVR because we didn't want to use MSP430 twice, and AVR is the most popular MCU instruction set among hobbyists. It turns out AVR does other interesting things for a security CTF, because unlike MSP430, it's a Harvard architecture.

The AVR levels are intertwined with the trading levels, so that the objectives for the AVR levels are more interesting than those of Microcorruption. You won Microcorruption mostly by figuring out how to exploit simple memory corruption bugs. My hope is that Starfighter requires a bit more in-sim programming.

afy• 11 years ago

I think that is just to refer to the instruction set that is being emulated (8-bit AVR - see Erin Ptacek's blog post about it on sockpuppet.org), used by the microcontroller on many Arduino boards

pja• 11 years ago

So, do I prioritise Stockfighter or TIS-100?

Decisions, decisions :)

MichaelGG3• 11 years ago

This is awesome. I still don't see how this isn't easily cheatable. What's stopping me from writing up a little tool that can win these levels and providing minimal instructions?

At one point, it even says they will award people that write OSS for it. Doesn't that just further reduce the effort needed to pass levels?

Not to be negative. I'm very, very much looking forward to this.

kasey_junk5• 11 years ago

Don't want to speak for them, but why would they want to stop you from writing that tool?

Remember their business model, it isn't "get people to the end of the game following the rules", it's "identify people with talent to recruit". Your tool would just be evidence of the latter.

pbh101• 11 years ago

The fear is that it would introduce noise into the signal of those who are using the tool, rather than the creator of the tool.

patio1111• 11 years ago

I have some ideas jumbling in my head about this, but it's getting late and I'm not sure they're maximally coherent. Briefly:

a) What's the worst thing that possibly happens? Someone scams a row out of our database? It has more rows. 10,000 someone scams a row out of our database? I will write a for loop which scams a million rows out of our database. What about the actual world or Starfighter's capability to positively impact our players, our customers, the industry, or our own situations just got worse as a result of that for loop?

2) I think you will find that a solution which beats level six also sails over the Turing Test, so if you have a tiny script which does so, I recommend not burning it on Starfighter, but would be honored if you decided to try.

3) At the end of the day, the model is not "Someone pays us for a PDF automatically spat out of our database", the trivialization of which PDF would be dangerous to the business model. It is "We become sufficiently enthusiastic about someone's performance that we successfully introduce them to a company which is not us and then that company hires them." Is the threat model "You use a script which trivializes our levels to win a free conversation with Patrick about your career?" Anyone can get that! At any time! Save yourself a git clone; just send me an email!

The more interesting question is then, given that you've learned absolutely nothing from Starfighter because you've git cloned your way to the finish line, whether you can (in thirty minutes with me, or Erin, or Thomas) convince us "Yes, this person is totally someone I should burn my stack of accumulated karma with the CTO of $CLIENT with to suggest they interview them."

MichaelGG3• 11 years ago

Ohhh. Ok I get it. I was under the impression that the goal was to have a Starfighter profile publicly, like a new type of credential/cert. Which, I guess now that I'm thinking about it, wouldn't be much better than those MS certs. If it's just a way to generate great leads for you then it makes total sense. And I imagine even if there are public scoreboards, the end point of your company isn't to provide an embeddable seal like those SSL resellers.

Thanks for taking the time to respond.

mbesto2• 11 years ago

The more interesting question is then, given that you've learned absolutely nothing from Starfighter because you've git cloned your way to the finish line, whether you can (in thirty minutes with me, or Erin, or Thomas) convince us "Yes, this person is totally someone I should burn my stack of accumulated karma with the CTO of $CLIENT with to suggest they interview them."

So, how is this any different from $CODER burning their accumulated karma directly with CTO of respective $CLIENT ? Why should we expect them to behave any differently because it's you, Erin or Thomas?

EFruit3• 11 years ago

I have to say, this is both impressive and infuriating; they beat me to it!

My effort was started in Nov. 2014, and is similarly written in Go. Clocking in at ~1600 LoC, it is certainly nothing special. I have no background in finance and only a cursory understanding of the technical aspects of the exchanges. It's mostly working, but I've really stalled because I can find neither a real purpose to continue, nor any resources on building interesting models. All it does is make financial noise (random trades between similarly irrational parties)

All in all, I'm happy for the guys working on this for doing what I apparently couldn't.

If anyone's interested, I'd be happy to post it to Github.

mattchew2• 11 years ago

I'm interested.

EFruit3• 11 years ago

Alright, I'll clean it up and post it ASAP. Beware, it's (probably) awful, terribly unidiomatic code; I'm not a professional.

EFruit3• 11 years ago

After a bit of stage fright (and Git/Markdown reeducation), here it is.

https://github.com/Efruit/marqit

BillyParadise2• 11 years ago

You'll be a little bit late to the party, but why not repurpose into a bitcoin exchange :) Everyone else seems to have done it.

quot665552• 11 years ago

I'm very interested in Starfighter and am looking forward to the release. However, there's something from your original announcement that still worries me. In this post you said:

The designed operation of our trading levels is “Play them, lose horribly, write a program, reset the level, run the program, have it crash with a bug, fix and re-run, lose modestly, adjust your approach, re-start the level, run the program, win.” (Folks have asked whether we’ll hold this against you. No, no, of course not. We log everything under the sun, but losing/restarting levels is planned. That capability is an advantage of doing this in a rich simulation as opposed to real life — you can go Knight Capital as many times as you want and no one loses their job!)

This feels good. But earlier, this was said [1]:

We will track player behaviors and skill at incredible levels of granular detail, instrumenting them like they were built by the Orwellian MiniPeace... We can tell you exactly what happened when your candidates tried to implement a REST API. We can compare their performance against hundreds of other talented engineers (including your current employees) on the same task.

Can you please elaborate on what exactly is going to be tracked? Is any of this configurable by the player? Personally, I'm going to have a hard time enjoying and learning if I feel like every mistake I make, or every extra piece of time something takes is going to be judged and count against me later on.

[1] http://www.kalzumeus.com/2015/03/09/announcing-starfighter/

kasey_junk5• 11 years ago

Remember their incentives. They want you to perform well (or more precisely they want performing well in their game to correlate with you being a good employee).

It is not in their best interests to count against you, quite the contrary they want to sell you to their clients.

pekk5• 11 years ago

What is a better way to sell a candidate to clients than to say she is in the top 1%? They have every reason to count out as many people out as possible. Nobody looking at this kind of service wants to hire candidates who are not the best.

patio1111• 11 years ago

I feel like this is largely an issue about expectations and disclosure more than it is about logging per se, right? We have a privacy settings page, which I'll show you in a minute, but here's the public view of myself on the staging server. I have it configured to Maximally Public (and will have that on in prod).

https://www.evernote.com/l/Aacj0YC7tCRIdbau2AgTrBORPc6kgX442...

The redactions are just to avoid spoiling levels. The "replay" link is a UI templating bug. (It isn't for showing a replay, it is a convenience method for replaying the level, and it's now supposed to show unless you're logged in and able to play the level yourself. Fixing this in the next 5 minutes.)

Now, if I (as a player) log into and view my own profile, I get a better sense of some stuff the system knows about me:

https://www.evernote.com/l/Aae3c5I6TdZA7ZtONMeM_z-ammvZA_3FD...

That is obviously not all the information the system has about me. If a sparrow yawned in a world simulation that I took part in, that fact is dumped onto an S3 bucket, somewhere. To be maximally explicit: every order I sent in. Every execution I got back. Snapshots of the GM's memory state as I passed through the levels. etc. We grab all that. That isn't an exhaustive list. Assume if it is amenable to being discovered by a computer that it will be discovered and persisted until the end of time.

This is not exposed publicly.

Here's the preferences UI. Verbose but it gets the point across, right. Defaults are "Anonymous" and "No Public Profile", respectively.

https://www.evernote.com/l/AadESN5EzgdCzpFAQnR0rIuxA5Ghb7pKX...

Now to a question which may be implied: "OK, I now understand that my info is locked up pretty tightly within Starfighter. But you want to tell people that eventually, right? That's how you justify your service to clients." At the point I'm on the phone with the CTO of $CLIENT I am a contigency recruiter. That means I only get paid if that CTO decides to interview you and ultimately hire you. You can bet that I will be doing my best Enterprise Sales work and showcasing your performance in its most positive truthful light. If I did not feel like that would be possible, that phone call never happens. If the CTO had previously mentioned to be "Be on the lookout, specifically, for people who are good at analyzing mountains of data. We need them for our fraud team.", and I had relevant signal on that question from level X, you can be reasonably assured that if you did really well on it I'd open the call with "The reason I'm recommending Foobar to you is because you told me you wanted folks who are really good at analyzing data. Let me walk you through the solution Foobar came up with -- it's a hoot."

(Worth re-iterating explicitly -- that that point, we've talked on the phone and you've explicitly given me the go-ahead on placing you.)

To the question: "OK, so I get that people outside the system don't get arbitrary read on my history, and I understand that you're essentially a firewall which is incentivized to only allow the egress of the best possible true packets: what about you? Aren't you going to be influenced by, you know, taking a look directly at my stream of HTTP requests and seeing that I fumble fingered a URL the first time I played the level?" To which I say: I don't care and don't have time for that nonsense -- I'd much rather concentrate on things which actually matter to an engineer.

quot665552• 11 years ago

Thanks for the thorough response. Yeah, pretty much. It's good that you all have thought about this stuff. There's always a PVP aspect if you're ever going to put yourself public or on the leaderboards, but that's just part of the game. I can always start out anonymous and go public if I turn out to be any good...

mdaniel2• 11 years ago

As a piece of unsolicited UX feedback, if you chose a different word for one of "cleared"/"cleaned" it would make those two radically different actions stand out from one another in the list. I do see that "cleared" is accompanied by a score, so it's not the end of the world, but I did warn you it was unsolicited feedback. :-)

Also, do you know why evernote chooses to send "Content-Disposition: attachment; filename=skitch.png" forcing my browser to download instead of display the png inline?

bliti3• 11 years ago

@patio11:

Why should I "play" this and not blog about programming? I still have to go through the usual technical interview when applying for a job. What's my incentive? I'm trying to find a reason to spend time learning this, because it doesn't translate into marketable skills. I can't put "Level 65 player in StockFighter" (because this might turn out to be a niche product that no one outside HN will know about), but I will definitely include "XSS research on Java Spring".

Note: I'm not trolling or trying to put the project down. I'm interested in it but unsure.

mattmanser• 11 years ago

That's not how it works. Enter the CTF and you get supplied as a lead to companies as a potential hire for doing well (if you want), it's not that you use it as a competency canary on your CV.

bliti3• 11 years ago

That's my point. I'm being supplied as a lead for investing time learning something that I will not be able to use in any potential position. Why should I do it? I'm genuinely interested in understanding this because it might blow up and I'll have to learn it anyway.

djloche• 11 years ago

It's a recruiting firm. Their business plan is to help companies with hiring (for a price) by matching the people who demonstrate skills in the game and opted in with their partner companies that want to hire people with those skills. The thesis is that successful starfigher candidates are all-star people that companies want on their team, but for whatever reason don't make it through the traditional hiring processes.

I suspect the right answer is to do both.

no_wave• 11 years ago

You would play instead of blog if you have high inhibition... very common in this industry

bliti3• 11 years ago

Great point! I hadn't considered that since I'm not shy at all.

grayclhn3• 11 years ago

(because this might turn out to be a niche product that no one outside HN will know about)

If this is the outcome, doing well at starfighter may not translate into much additional career success. For it to be clearly worth your time, they probably have to do reasonably well. But HN-famous might be enough for a tech recruiter.

borkabrak2• 11 years ago

Well, I think at least one intended reason is: because it's fun. I get to play games/solve interesting puzzles, and I might get contacted about a job that some person in a position to know thinks I might like and do well at? Count me IN.

galois198• 11 years ago

Looks great - I get that this was built using Rails/Go, however what language(s) will players be coding in?

patio1111• 11 years ago

For trading? Anything which can drive a REST API. For the compiler/emulator levels, a subset of C and assembly.

runevault• 11 years ago

If nothing else reading this has me interested in trying Go. Kind of wish the new book by Kernighan was out before November, as that book feels like a great excuse t do a deep dive on the language.

lewisl9029• 11 years ago

On the topic of automated trading:

I strongly encourage anyone interested in real-world algorithmic trading to take a look at Quantopian.

https://www.quantopian.com/home

It's not HFT by any measure, but it manages to make algorithmic trading extremely accessible by providing a platform where users can develop their algorithms, backtest them against 13 years of market data, live trade them with paper money, or live trade with real money by linking a brokerage account.

They have also open-sourced the algorithmic trading engine used to run their platform:

https://github.com/quantopian/zipline

tptacek21• 11 years ago

Yep. There is like, zero overlap between Quantopian and what we're doing; they're both good things to play with.

jbredeche• 11 years ago

(I'm the CTO and cofounder of Quantopian)

I am incredibly excited to try my hand at Stockfighter. Hopefully some of the things I've learned over the last few years at Quantopian will be useful, but I expect I'll learn quite a few new things as well.

smoyer• 11 years ago

People who are at the leading edge of HFT do everything they can to trim microseconds off their trading time ... it can get pretty insane. One talk on HFT performance at JavaOne [1] included the recommendation that you buy more memory - so you can turn off garbage collection completely during trading hours.

[1] http://www.slideshare.net/PeterLawrey/writing-and-testing-hi...

FLUX-YOU• 11 years ago

Do you guys model regulation through world simulation (e.g. the twitter feeds) and is there a Federal Reserve/Mint with related data?

hawkice• 11 years ago

Really simple question: do the privacy settings have any impact on the recruiting stuff? I would love for you guys to poke around my data, I trust you guys, and would love for you to be able to get my name, but I would otherwise prefer to be anonymous. Ideally it would be a privacy-orthogonal checkbox, maybe?

tptacek21• 11 years ago

YES. Reminder: most people who play with Starfighter CTFs will never be interested in a new job. We're designing this stuff for those players just as much as job seekers; we're competitive about CTFs, and trying to make a mark.

Look at the leaderboard on Microcorruption.com and notice how few real names are there. That's what CTFs are like. If you want to have a profile page, we'll give you one, if you want to link up your Twitter account, you can do that, but if you want to be secretsquirrel1235813, you can do that too. From experience with CTFs: I think a slight majority of people will be anonymous.

There is nothing we will do with your data that won't be opt-in.

Pyxl101• 11 years ago

We do not intentionally expose you to the speed of light as a limiting factor early in Stockfighter… but we can’t remove latency as being an actual limitation distributed systems have to overcome.

Couldn't you remove latency as a factor, at least in a game world? To play devil's advocate, it seems like you could build a game where time advances according to the player's control by API.

Imagine if each API call optionally took a parameter describing by how much time should have advanced since the last call, upon the arrival of the current call. If the parameter is omitted, the server advances the world by the corresponding amount of real clock time. The game simulates that time passing, and simulates receiving the call at the time specified. Perhaps an API to stop and start the flow of game time, or run the game for a certain time and stop.

Players can interact with the game naturally, and solve problems without controlling time, but if they run into latency issues they can orchestrate the advance of time on the server precisely. I understand this will not be simple to implement, especially if the game consists of multiple communicating agents, but it could allow you to remove the uncertainty of players interacting with the game over a network, and more easily set up complex race conditions or timing attacks. It will also more gracefully handle any slowness that occurs in the game or bot simulations, since while game calls might take longer real time to return, they will presumably not take more game time. Last but not least, it could allow you to design test cases that are deterministic in ways that involve time, i.e., this event occurs exactly 100ms after the game world begins, every time. Players can exactly reproduce situations that would otherwise occur nondeterministically, while you retain your ability to simulate interactions that you want to be nondeterministic.

(I'm just exploring the idea because it seemed interesting, not making the case that this is especially important in the scheme of things.)

donkeyd• 11 years ago

I've just started programming professionally. I've noticed I'm a quick learner and I'm pretty good at problem solving. However, because I just started I lack quite a lot of knowledge. Will I still be able to play this game, or is 5 years of full-time work on multiple languages a prerequisite?

patio1111• 11 years ago

The first few levels are specifically designed to be solvable by someone near your skill level, and after you get through them, you might surprise yourself on the "hard" ones.

sireat• 11 years ago

Two anecdotal data points on somewhat similar sites from an employed programmer and father.

Codehunt (from Microsoft Research), very addictive yet nice concise format, polished interface, allowing Java or C# a plus.

Minuses - Requires some unnatural code golf for maximum stars. Needs new problems and difficulty is too variable. I got in Top 5 but realized it was a completely worthless achievement as it just was not hard enough, somehow I hoped for a short note from MS saying GJ here is the next challenge level.

Bonus: Did not feel bad about playing this at work on breaks.

Codecombat - program in javascript, using classical MMORPG terminology, this should have been more addictive but something was off, maybe it was the latency or browser glitches.

My bot using simple strategy got in top, but I realized that I would rather go back to writing my scripts for real MMORPGs where my bots make actual money from RMT.

I still get Codecombat monthly e-mail reminders of new features but somehow the appeal of fake MMORPG is not there.

Also, felt very guilty about firing Codecombat up at work...

So you guys not going for the standard DnD type of setting might be a good idea.

ExploitsforFun• 11 years ago

Out of curiosity, have you gotten any press inquires about your launch from major tech publications? I know that your company is outside of the VC funding publicity bubble but I am curious if anyone has reached out?

Designing and Building Stockfighter, Our Programming Game

Related Stories