NYSE/NYSE MKT has temporarily suspended trading in all symbols

https://www.nyse.com/market-status/history

by mmastrac • 11 years ago

326 105 11 years ago

Seems like they were having issues earlier today, since before market open. Looks like a network issue based on the previous reports. Those were marked as resolved; unsure if the current issue is related to the previous but it seems likely.

Other exchanges and trading venues appear to operating normally.

efuquen• 11 years ago

Pretty bad timing, markets are already rattled because of China:

http://www.bloomberg.com/news/articles/2015-07-08/u-s-index-...

Regardless of the technical cause have a feeling this will make things worse and people more nervous.

washedup• 11 years ago

The technical glitch is probably related to China news somehow, but it is rather meaningless for trading in general. NYSE has pretty antiquated technology compared to other exchanges.

usefulcat2• 11 years ago

The technical glitch is probably related to China news somehow

If that were true, you'd expect to see volume spikes on multiple exchanges, which so far I haven't seen any evidence of.

thrownaway2424• 11 years ago

Maybe their network administrator is stuck on the ground in a UAL plane.

scrozier• 11 years ago

Well, at least he/she can read the Wall Street Jo...oops.

minimax• 11 years ago

This doesn't mean all trading is suspended, it just means you can't trade specifically on NYSE or NYSE MKT (formerly Amex). You can still trade NYSE listed stocks at all the other exchanges (BATS, NASDAQ, NYSE Arca, etc). Obviously a black eye for NYSE but it's not as big a deal as the media are making out.

Q6T46nT668w6i3m• 11 years ago

CNBC and Bloomberg have been repeatedly saying this

swasheck13• 11 years ago

all open orders cancelled.

partiallypro• 11 years ago

It's still pretty bad. Imagine being the IT dept at the NYSE right now. The trading is likely being rerouted to other exchanges. The Wall Street Journal website is also down.

ianhawes• 11 years ago

Possibly related: wsj.com is also down.

vonmoltke2• 11 years ago

Only the front page is generating an error for some reason. Deeper links like http://www.wsj.com/news/economy still work fine.

swasheck13• 11 years ago

UAL went down. WSJ down. NYSE halted. coincident?

edit: based on everything that i've heard/read, the incidents are most likely not related. sorry for asking the question.

notNow2• 11 years ago

Does this have anything to do with the Chinese stock market rout and the rumors circulating that it's synthesized to a certain degree?

swasheck13• 11 years ago

interesting question and i'm not familiar with the rumors. is it that the shanghai index crash is synthesized? not being completely fluent in the financial sector, how would this be achieved?

speeder• 11 years ago

I doubt it is synthesized, they managed to make 52% of all listed stocks enter automatic freezing at the same time...

knowaveragejoe• 11 years ago

Most likely, yes.

vonmoltke2• 11 years ago

The Journal's issue appears to be only with their front page. The rest of the site is working correctly.

swasheck13• 11 years ago

mad rush to publish crushed the front page? wsj.com/asia 504s as well.

maxerickson4• 11 years ago

The TV is stating that US government sources do not see a connection.

Edit: Now the Secretary of Homeland Security is saying that there is no evidence of United or the NYSE being due to malice.

Edit (about an hour later): Now the White House Press Secretary is reiterating the point about no indication of malicious actors.

swasheck13• 11 years ago

ok. i'm not near a tv so i can't tell. thanks for the info.

maxerickson4• 11 years ago

Now 2 hours later Tom Farley, President of the NYSE, is stating that it is an internal technical problem.

ubernostrum• 11 years ago

As the saying goes, never attribute to malice what can be explained by incompetence.

In the case of United Airlines, be prepared to adjust your threshold of expected incompetence dramatically upward; since their merger with Continental multiple years ago, system-wide outages have been an every-couple-of-months thing.

dingbat• 11 years ago

HN was down earlier today too, or at least timing out on my end. never saw that before.

swasheck13• 11 years ago

that happens to me quite a bit. nginx gateway errors are pretty common on HN in my experience

smackfu• 11 years ago

Ha, you have been lucky then.

davesque• 11 years ago

Maybe there's a stock market hacker flying around on planes...kinda like in the movie Hackers. Of course, in that movie they had no trouble cuffing him without grounding the plane. :)

mangeletti• 11 years ago

ZeroHedge was down for a bit as well. I thought maybe it was an AWS (or other cloud provider) issue causing all the problems, until I read about the NYSE issue.

woodchuck64• 11 years ago

No need to apologize, my Bayesian inferences are going wild too.

omnipedal4• 11 years ago

they're highly likely related, your intuition is telling you that for a reason.

lectrick• 11 years ago

I have a possible diagnosis- Technical debt has reached critical mass!

chollida13• 11 years ago

This isn't much of an issue for traders That's a nice benefit of having 40+ trading venues in the US. They had problems at the open as well with connectivity on one of their gateways.

Issues like this crop up all the time and most of the time they are resolved before the open. The good news is that they aren't reporting any lost trades or trade busts yet so this isn't as bad as the BATS open at BATS:)

Having said that, they've announced that they will cancel all open orders, that is a huge deal, I can't remember the last time they did that.

To put that into perspective, cancelling all open orders would be the Silicon Valley equivalent of Ebay loosing all bids on their current auctions.

NYSE has always been considered second rate in their IT compared to the NASDAQ and BATS and this won't do much for their reputation.

EDIT meant BATS not FB, thanks!

One other point to keep in mind when throwing around hacking conspiracies. The exchanges aren't running on public networks. You can't DDOS them or hack directly into the matching engines. Though I'm sure you can break in via some other NYSE owned network and make your way to the matching engine somehow.

To hook into the exchange you either go through a blessed intermediary like GS or you plug directly in via colocation. You just can't keep pinging the NYSE on port 80 to bring it down.

swasheck13• 11 years ago

right. one trading head just noted that it's a "non-issue." the timing is interesting, though. it's midday so there's lighter volume. big problem is going to be with index calculation. other exchanges are functioning just fine and other venues are seeing plenty of action and stability.

late2part• 11 years ago

Nothing to see here, move along.

tptacek• 11 years ago

Are there serious trading shops that don't route orders dynamically to multiple venues?

chollida13• 11 years ago

Yes in the sense that about 90% of hedge funds just use a SOR(Smart Order Router) from their executing broker. So from the funds perspective they are just sending an order to GS that should execute on the "US Market" and they expect GS to send it to the appropriate venue.

Most funds just trade through an intermediary like GS and all intermediaries have SOR's that route to each venue in the US as per RegNMS

The remaining 10% are your market makers and HFT's and for them this is a bit of a headache but it happens and they are well equipped to deal with this. No fund that I know of is a market maker for just the NYSE, but if there is then I guess they just go home early?

usefulcat2• 11 years ago

Yes, some strategies are highly exchange-specific.

mrdrozdov• 11 years ago

I'm really happy to see that they're canceling all open orders. It's much better than letting them stick around and then execute with unexpected behavior. It looks like NYSE learned from what happened related the FB opening (just as you alluded to), and what happened with Knight around the same time. Playing it safe can be a smart move, even in the stock market.

runeks• 11 years ago

My immediate reaction is that their system shouldn't require canceling all open orders. But I admit that I don't know much about matching engines.

If they roll out a new version of the module that creates orders, I would think it makes sense to include the version of this order module in the order data structure. This way they can delete only the bad orders, created by the faulty module version, instead of all open orders. But it's also possible that I'm way in over my head here :).

anonu• 11 years ago

Actually, the FB IPO open fiasco was on Nasdaq not BATS. Nasdaq, has had way more serious issues in the last 2 or 3 years that I can recall compared to NYSE.

[EDIT] The BATS issue you may have been referring to is BATS IPO when they tried to list on their own exchange. The technology failed miserably that day, so much so that they decided to pull the IPO altogether. One of the biggest fails I can think of.

chollida13• 11 years ago

Yep, I was wrong, you are right! I meant BATS IPO on the BATS.

TeMPOraL2• 11 years ago

RE hacking, this is pretty much a case of https://xkcd.com/932/.

jeffmcjunkin• 11 years ago

Great info, thanks!

(Very minor, but "ping" usually implies ICMP Echo Requests, and ICMP doesn't have ports)

smoyer• 11 years ago

UDP and TCP include echo ports (port 7 just like ICMP) and many tools that can run in non-privileged mode actually use the UDP echo instead of ICMP.

jimktrains2• 11 years ago

It's very common to "ping" a service by sending a valid request and seeing if you get a valid, expected, non-error response.

el_duderino• 11 years ago

A tweet from last night: "Wonder if tomorrow is going to be bad for Wall Street.... we can only hope."

(source: https://twitter.com/YourAnonNews/status/618626955433349120)

Good call on their part. Coincidence?

maxerickson4• 11 years ago

That tweet is probably about the market situation in China.

socceroos• 11 years ago

I get the impression they'd heard on the wire that something might go down.

Given the blame coming out of China, it wouldn't surprise me if they tried to crash the market on opening with a flurry of sells. Looks like NYSE got a whiff of the pending trades and the NSA stepped in. All trades dropped. Crisis averted. For now...

this_user• 11 years ago

It should be noted that this concerns only the regular NYSE and NYSE MKT (formerly known as AMEX). Their ECN NYSE Arca is working properly. So, this is really not that big of a deal for the market.

briandear2• 11 years ago

Combining this with the United airlines "glitch" -- it is definitely suspicious. What are the odds of two high profile failures happening at the same time?

tuckermi• 11 years ago

Sounds like a similar problem to the birthday paradox - only instead of looking at the likelihood of any two people in room having the same birthday, we are asking what is the likelihood of two major companies having the same major, visible failure day. There is a nice explanation and calculator here:

http://betterexplained.com/articles/understanding-the-birthd...

The parameters that you provide would be the number of companies that would be newsworthy and the number of possible "dates". You could probably back out the latter from an expectation on reliability (five nines, as an example) or think of it as the time between major, visible failures (e.g. enter 3650 if you think of it as a once in ten years kind of failure).

swasheck13• 11 years ago

thanks. that's an excellent read and adds tremendous balance.

omnipedal4• 11 years ago

If there's smoke there's fire. In my view, Occam's razor might indicate that coincidence is unlikely, and that the two major incidents in US on the same day China's market crash suggest it's a state sponsored cyber attack.

oldmanjay• 11 years ago

Also I've been having real trouble getting HBO Now to stream consistently. That's triply suspicious! Or a total coincidence.

swasheck13• 11 years ago

your GoT frame-by-frame analysis will certainly be delayed

ceejayoz2• 11 years ago

What are the odds of two high profile failures happening at the same time?

Asymptotically close to 100%, given a little time.

briandear2• 11 years ago

And combined with the United Airlines ground stop happening now.. Something is certainly happening.

xhrpost• 11 years ago

That has already passed http://www.latimes.com/business/la-fi-united-flights-grounde...

BinaryIdiot• 11 years ago

That's a pretty big leap to make. You can't take disparate events, equate them and purport "something" is happening. That just doesn't make any sense.

jjulius• 11 years ago

"NBC News quoted two unnamed U.S. officials as saying that there is no indication that the market shutdown was related to the computer problems that temporarily grounded all United Airlines flights Wednesday morning."

http://www.washingtonpost.com/business/economy/nyse-trading-...

saryant• 11 years ago

SHARES crashes every few months like this, that's not news.

acscott• 11 years ago

Consider this: what are the odds that a coincidence won't happen?

task_queue• 11 years ago

Someone bought the "hackers" narrative.

davidf18• 11 years ago

Time to make the market makers fully electronic. No point in having to physically go to them. Why is NYSE so backwards compared with NASDAQ?

ExpiredLink• 11 years ago

BTW, is NYSE still run by a Tandem?

Edit: I hope so! Great platform.

vonklaus• 11 years ago

It is now owned by Intercontinental Exchange[0]

[0]http://www.nytimes.com/2015/07/09/business/dealbook/new-york...

nickpsecurity• 11 years ago

They're called HP NonStop now. Great platform. Yet, even the best platforms can't stop stupid code from ruining NYSE's day. ;)

bra-ket• 11 years ago

probably it was a huge volume of trades at the open due to chinese market halt

curiousjorge• 11 years ago

wsj goes down, nyse goes down due to technical glitch. seems like a crazy coincidence or some very large state who have a track record of infiltrating and disrupting America electronically at a time when the leaders of that state is feeling the heat and need to stop speculators.

swasheck13• 11 years ago

so you're saying that china has initiated some sort of half-cocked single-exchange attack? all other exchanges are operating, and trading is maintaining volume, and asset prices are stable. not sure i'm seeing the reasoning for such an attack.

jaketay• 11 years ago

Its a distraction. We are here talking about the NYSE not the colossal Chinese market collapse.

ratsimihah• 11 years ago

It's just another advertising campaign for Mr. Robot.

mark_l_watson• 11 years ago

Good one! BTW, I like that show. Technology wise, sort of believable, and entertaining.

PhoenixWright• 11 years ago

This is what happens when you pay ENGINEERS less than 100k in NYC! This is what happens when you hire a bunch of H1Bs! This is what happens when you don't invest in tech infrastructure!

Why aren't Vladimir Tenev and Baiju Bhatt being acquihired? Why isn't David Byttow making 1 million plus as a full stack engineer for the NYSE? It's because companies, even those heavily dependent on their systems, see tech workers as a cost instead of THE business.

This will more than likely never change. Underpaid engineers vastly outperform their salaries. But when things like this happen I can't help but feel a little glee.

blobbers• 11 years ago

What are you trying to say? That somehow American born engineers are better than those from the rest of the world?

The attitude of a small subset of people in this country is a constant reminder of why I don't feel at home here.

littletimmy• 11 years ago

It looks like Anonymous tweeted something about this yesterday...

https://twitter.com/YourAnonNews/status/618626955433349120

kevinpet• 11 years ago

We all knew today would be "bad for wall street". The Chinese stock market is crumbling (due to economic, not technical reasons) and Greece is in the fiscal equivalent of the crazy guy barricaded in his house with this gun collection.

justinzollars• 11 years ago

Apparently its an staypuft attack https://twitter.com/staypuft/status/618808119695544321

NN88• 11 years ago

More info on them/this?

ablation3• 11 years ago

Well, one thing this has done is bring the paranoid out of the woodwork. I've rarely seen such a display of tinfoil hattery on HN as I have done in the last few days.

swasheck13• 11 years ago

this is nothing compared to the comment thread on zerohedge. they're connecting jade helm, china, greece, and this into something pretty extraordinary.

ablation3• 11 years ago

Yeah, but that is Zero Hedge's audience though: the upper echelon, rarified air, pure-bred market conspiracy theorist. What surprised me is seeing it start to creep in HN in such a volume.

ddeck• 11 years ago

From the NY Times:

"A trader on the floor of the exchange in lower Manhattan, who spoke on the condition of anonymity, said that after the suspension began, traders were told that the problem was related to updated software that was rolled out before markets opened on Wednesday.

According to the trader, the exchange said that the new software caused problems soon after trading began on Wednesday and the exchange decided to shut down trading all together to fix the problem.

A representative for the exchange did not respond to a request for comment on the trader’s account."

http://www.nytimes.com/2015/07/09/business/dealbook/new-york...

swasheck13• 11 years ago

three cheers for SDLC!!!

ocschwar• 11 years ago

Good thing I still have all the MREs I bought for Y2K

noname123• 11 years ago

Very interesting, obviously matters very little to retail traders as most retail brokerages either sell order-flow to marketmakers (e.g., Ameritrade) or have their own smart-router that looks at the liquidity of all exchanges/ECN and decides how to route their customers' orders (Interactive Brokers).

Quick question for trading peeps out there, is there a reason why one would want to direct orders directly to NYSE? Is it because it's still the place to trade bulk orders? (vs. say BATS or ARCA).

naqeeb• 11 years ago

Systems have bugs all the time. Unfortunately, it was bad timing that all of these systems were affected by different issues.

You're better off using a jump to conclusions board rather than speculating on the correlation of these events.

themeek• 11 years ago

This is a 'networking issue' not yet attributed to an adversarial compromise.

There is also an outage at United Airlines and the Wall Street Journal - none so far attributed to an attack.

NYSE/NYSE MKT has temporarily suspended trading in all symbols

Related Stories