Contractors See Weeks of Work on Health Site

http://www.nytimes.com/2013/10/21/us/insurance-site-seen-needing-weeks-to-fix.html?pagewanted=1&_r=0

by noahlt • 13 years ago

76 108 13 years ago

conover• 13 years ago

That's a staggering amount of code. Nearly unbelievable. To put that in perspective, Windows XP was 45 million lines of code[0].

[0] https://m.facebook.com/windows/posts/155741344475532

EdgarVerona9• 13 years ago

Yeah, I'd call total BS on that number, unless they did something absolutely insane... I mean, we're talking "instead of saving user information in the database, we codegen'd a set of classes specifically for each regisered user and save them to our repository" level of insanity. Has there ever been software that's even come close to approaching 500 million lines of code?

Houshalter• 13 years ago

Imagine someone threw together some automatic code generation thing. It tries every possible input and then creates a special if-then statement for it.

Don't worry, the compiler will optimize it.

tonyfelice• 13 years ago

My reaction as well. Even 500 million bytes would be hard to believe.

While "efficient" is not a word that comes to mind when thinking about the government, 500 lines of code would be 1.6 lines for every man, woman, and child in this country.

EdgarVerona9• 13 years ago

This is where my hypothesis comes in. Instead of a database, they're codgening a small class and adding it to their codebase for every registered user, and recompiling the entire site immediately following that operation, blocking requests until the recompilation is complete. This hypothesis neatly explains both the terrible performance and the unimaginable LoC estimation.

jussij4• 13 years ago

My guess is they're counting code from existing packages used to create the site.

So for example, if they used Java, that line count is including the lines of code found in the JDK.

I see no other way of getting to such a large number.

EdgarVerona9• 13 years ago

I don't see it approaching that even with the lines of code for their existing packages. For instance, this is old and out of date, but you could extrapolate a ballpark estimate out of it... looks like back in 2007 or so JDK was about 6 million LoC (http://cr.openjdk.java.net/~darcy/OpenJdkDevGuide/OpenJdkDev...). Even if it increased tenfold since then, it's barely making a dent in that 500 million LoC number.

mag4872• 13 years ago

It's hard to imagine that even those could total more than a small fraction of the purported 500 MLOC. The JDK is only a few million lines of code.

joeblau• 13 years ago

What was the site written in?

bluefinity• 13 years ago

Somewhat surprisingly, Visual Studio 2012 has 50 million lines of code. [0]

It's kind of weird to think that VS 2012 has more lines of code than Windows XP.

[0] http://blogs.msdn.com/b/visualstudio/archive/2012/06/20/the-...

MartinCron• 13 years ago

VS 2012 does a lot more.

carterparks• 13 years ago

That's a lot of eating your own dog food

jussij4• 13 years ago

From the page:

One specialist said that as many as five million lines of software code may need to be rewritten before the Web site runs properly.

It sounds like they've been doing a little too much coding and not enough designing.

jlgreco2• 13 years ago

My first thought was the exact opposite.

jussij4• 13 years ago

Maybe half the developers were coding while the other half were designing ;)

EdgarVerona9• 13 years ago

Was that specialist in the running for being paid to fix it, and thus was overinflating the estimate so he could get paid a few hundred million to fix it up?

I like the idea of a government healthcare program, I really do. But this website... man, someone lined their pockets on this scam, big time.

hga10• 13 years ago

Not even vaguely a scam, as I reply to you elsewhere in this discussion: https://news.ycombinator.com/item?id=6586603

Unless you want to posit that the contracts were accepted by companies that knew it could not be delivered in the time required. And even then, people are optimistic, and I strongly suspect they didn't think HHS/CMMS would be so insane about the number of changes demanded during the process, going all the way through the week before launch (!!!).

chill1• 13 years ago

Use mobile version to get around the pay-wall: http://mobile.nytimes.com/2013/10/21/us/insurance-site-seen-...

ddoolin• 13 years ago

I understand this website is probably more complex than your average website, but I would put money -- a lot of money -- on it being nowhere near 500 million lines.

chubot• 13 years ago

Yeah it's got to be off by at least 3 orders of magnitude. 500K LOC might be industry standard bloat, but it should be doable in 50K if you hire good programmers.

But 500M is not credible, unless you are doing things like counting auto-generated code and open source libraries. Even then, you would have to be using 100,000 open source libraries to get near 500M LOC :)

codygman• 13 years ago

No wonder it doesn't work. It's very unlikely that healthcare.gov is more complex than an OS (such as XP noted by conover in this thread).

tunesmith• 13 years ago

I wonder what languages it focuses on. I haven't heard.

I find some of it amusing - I've worked on some enterprise-level projects with business systems talking to other business systems, and I can only imagine how much eye-rolling is going on among the programmers that are actually trying to do the work. This is a project where some very visible, loud people have a vested interest in it looking bad even if it were flawless.

I actually think it would be kind of fun to be close enough to the technology to really identify the core constraints and be in a position to help alleviate them.

EdgarVerona9• 13 years ago

The sad thing is that the site being bad should be really tangential to the politics. I personally would love to see outright socialized healthcare, where everyone was mandatorally enrolled in a government created program. I feel like we'd have the most leverage to bring costs down under a centralized and non-profit system. However, this system being as broken as it is - and costing as much as it has - did nothing to help that cause, even though it should be tangential to it. Very frustrating.

hga10• 13 years ago

I guess you can't see how giving the government even more power of life and death over its subjects can be anything but political?

Certainly the U.K. NHS experience shows the running of such a system is political. I don't watch it closely, but as I understand it Tony Blair/New Labour doubled spending, but somehow functionaries and bureaucrats got the new money, not the front line people and institutions.

You think we can have anything less than a totally political discussion about euthanasia, most recently known as the Liverpool Care Pathway in the NHS?

hga10• 13 years ago

"This is a project where some very visible, loud people have a vested interest in it looking bad even if it were flawless."

Are any of the managers in the White House, HHS and CMMS in that category? Certainly not loud (nobody is speaking on the record: https://news.ycombinator.com/item?id=6586119), but perhaps anti-Obamacare moles?

Because the people running this show every sign of being true believers, and I seriously doubt they're playing much attention to the "waco birds" who are totally against it.

r0h1n2• 13 years ago

$400M for 500M lines of code. That's roughly $800/KLoC. I wonder how that compares to any other modern software.

greenyoda2• 13 years ago

$800 pays for roughly one day's worth of work for a senior developer in the US. One thousand lines of working, debugged code (which is presumably what's being paid for) sounds like an awful lot of code for one person to write in a day, if the problem domain has any degree of complexity. Some parts of healthcare.gov could be quite complex, e.g., interfacing to other government systems whose APIs are not well documented.

And I agree with the people who think that 500M lines of code is unlikely. Judging by some of the statistics that were cited in other comments (e.g., "Windows XP was 45 million lines of code"), this may be more lines of code than Microsoft has written since the company was founded (many of the lines of code in XP were originally from NT, etc.).

r0h1n2• 13 years ago

It costs ~$300K for a senior web developer in the US currently?

ivanbrussik• 13 years ago

they might be exaggerating that by including all of the respective libraries used etc, or just exaggerating in general (like the gvt tends to do)

jlgreco2• 13 years ago

Even if they counted PHP's lines of code, that would only throw on less than a million more lines.

I strongly suspect that somebody is full of shit.

mag4872• 13 years ago

I'm having a really hard time believing that figure.

Stratoscope• 13 years ago

On Monday, Mr. Obama will host a Rose Garden event with people who have successfully enrolled in the health care exchanges.

It's great that they can all fit in the Rose Garden!

DigitalJack• 13 years ago

I'm pretty sure they'd all fit in my basement.

DigitalSea2• 13 years ago

Sounds like a load of crap to me. Seriously, 500 million lines of code? I bet not even mission-critical code written to power NASA spacecraft and satellites are anywhere near that length.

I propose they give the project to the HN community, divide up the $400 million figure amongst the participants and give us a weekend and I bet we'll have more to show than that of the so-called developers the government hired.

I am not underestimating the complexity of such a site, but seriously, it's not like it hasn't been done elsewhere to a degree. if I can do Internet banking, the a site for providing healthcare can be built a lot easier. They're both of similar scale and security requirements I would imagine.

This site seems like the greatest scam ever perpetuated. I bet the contractors are laughing themselves to the bank literally as we speak. Even though it's bad publicity, money is a higher priority than credibility for some businesses out there it would seem. Is it any wonder people don't trust the government? They can't even hire competent people to build them a website. This seems to be a recurring theme for local councils and governments elsewhere as well, so it's not just the US this is happening.

People need to get mad about this. That's $400 million dollars that could have been invested elsewhere like medical research, charities, food programs, rehabilitation for the homeless... A $400 million mistake has been made, and another $400 million will probably be spent before it's even ready.

What a waste of money.

javert3• 13 years ago

While I completely agree with you in general, a nit pick:

I bet not even mission-critical code written to power NASA spacecraft and satellites are anywhere near that length.

That doesn't seem like a good example. You want to keep the code in those kinds of things to a minimum, and that's probably not too hard.

DigitalSea2• 13 years ago

Yeah, that's the best example I could come up with at the time of writing the comment. It's been a long day today, and I don't mind the nit-pick, you are right.

code_duck3• 13 years ago

It's best to keep the code in everything to the minimum!

keypusher• 13 years ago

Yeah, I remember reading somewhere that they have extremely strict standards at NASA, to the point that multiple reviews are mandatory on even single line changes for mission critical systems.

Here's an example: http://spinroot.com/gerard/pdf/P10.pdf

And another: http://lars-lab.jpl.nasa.gov/JPL_Coding_Standard_C.pdf

Dunno how closely these are followed in day-to-day operations, but they clearly put some thought into it.

EdSharkey5• 13 years ago

Yes, we've burned at least $400 mil. Last week, everyone insisted it only cost $55 million, and the $768 million figure was misreported. I can only speculate what the final cost will be after victory is declared over these bugs.

It is too late to get mad about any new/rising costs related to the ACA. It is here to stay and might as well be Medicare 2. "Law of the land" as the president is fond of saying.

Here's a fascinating visio diagram that shows just how much government has grown under the ACA. Anything red indicates new or expanded government:

http://www.jec.senate.gov/republicans/public/index.cfm?a=Fil...

Seems like the ACA created very painful integrations between federal systems and insurers, asking them to do many new things. Not a big surprise they're having problems. Some federal IT systems are archaic and integrations/change are costly and fraught with danger. The insider stories I've read about the Soc Security mainframe-based system are frightening!

EdSharkey5• 13 years ago

Here's the frightening story I recalled on Social Security from 2011. http://www.informationweek.com/government/enterprise-archite...

Woe be to the phalanx of ACA consultants that has to integrate with those systems... :/

hga10• 13 years ago

"It is too late to get mad about any new/rising costs related to the ACA. It is here to stay and might as well be Medicare 2. "Law of the land" as the president is fond of saying."

Reading this article, which at the highest level tells us the people in charge, the White House and HSS's CMMS (all government workers, the latter the integrator of the project) are still discussing how to fix this mess, hoping to have a plan by Thursday, combined with e.g. CMMS delivering changes to the contractors through the week before launch, and per this article "In the last 10 months alone, government documents show, officials modified hardware and software requirements for the exchange seven times.", suggests to me it could easily be years before the exchange works, and possibly never. We've seen this before (FAA, FBI, IRS, etc.).

All depends on CMMS (non-specialist civil servants and perhaps political appointees) learning big project management "on the job" or the vast majority of them getting replaced by an integrator who has a clue, either of which will take months.

That's just what's required to get sane high level management of the project.

In the meanwhile, can the individual parts done by different contractors get some serious progress? Well, first it depends on CMMS freezing requirements, and doing a good enough job of deciding what requires a total rewrite. Per this article:

"Quality Software Services Inc., or Q.S.S.I., a unit of the UnitedHealth Group, developed the identity management system, another major component that allowed consumers to register and establish accounts.

The identity management system from Q.S.S.I., which also taps into government databases to retrieve users’ personal information, was a particular source of trouble when the exchange opened. Change orders show that on Oct. 4 — after millions of people had been trapped in technological loops trying merely to log in — the government asked CGI to help it devise a new identity management system to replace the one provided by Q.S.S.I. But specialists said that approach was abandoned as too risky. Ultimately it was decided to fix the current identity system."

And we've heard from a HN member who was involved in a big university project that Oracle's identity software is fairly gnarly to get working. Which I have absolutely no trouble believing, because the underlying problem is gnarly.

But one wonders just how much panic was behind CMMS's idea of punting the existing one 3 days after launch. And you know the managers of both contractors had to spend a lot of time working with CMMS on just this detail ... working with people who are simply not (very) experienced in software development and management.

(Ask for my worst experience working for a government for the level of cluelessness that's likely involved here.)

So, getting to my point, you may say "It is here to stay", but "You Can't Fight Mother Nature" or the hard reality that computers do exactly what you tell them to do, not what you want them to do. How many months or years do you think people will stand for being fined by the IRS for not enrolling when they can't? How many will die because they've lost their existing coverage due to the ACA changes but can't sign up for anything, or anything they can afford without the subsidies which only the exchanges can (now) deliver? What happens if this not entirely nonfunctional system develops an adverse selection death spiral?

That's where only the very dedicated, the ones who really need a lot of healthcare, jump through the hoops, whereas the healthy 20 somethings and beyond, who are asked to do a 30 step (!) application on an unreliable system, punt, pay the fine but not the much larger sums needed to subsidize the 50-64 age bracket. Which raises the prices of the policies offered, which further decreases enrollment....

unreal37• 13 years ago

There's no way that's true.

No possible way. For a website. Did they write their own operating system from scratch? And database?

adventured• 13 years ago

It's definitely a lot worse than that.

500 million lines of code would represent nearly every line of code that has ever gone into all shipped Microsoft software spanning 37 years.

yen223• 13 years ago

They probably count generated code. SQL generators aren't known for their LOC efficiency.

mattdeboard• 13 years ago

I... dunno guys. Multiple disparate systems designed by separate government contractors who no doubt pumped up their work estimates tutu get even bigger contracts. Remember this is a $400 mil project. Maybe I spent too long in the government but this doesn't seem unrealistic to me.

EdgarVerona9• 13 years ago

Yeah, after reading that article, I get the very strong feeling that these "federal consultants" are in for a second round of greedy-greedy-grabfest. They're throwing insane sounding numbers out there so that they can justify egregious compensation. Someone needs to call these people out on their bluff, there is no way that this terrible site is even close to the LoC numbers they're throwing out there.

I'm so sick of seeing contractors fleece our government (and by extent, us) by pulling the wool over the eyes of those who don't know any better and making sweetheart arrangements.

hga10• 13 years ago

Geeze, how many times are you going to make this point in this discussion? See my first reply here: https://news.ycombinator.com/item?id=6586603

I point out there's absolutely no way the contractors could have won with the insane management they were under, unqualified CMMS government bureaucrats doing the integration and constantly requiring changes, through the week before launch, some of which is detailed in this article.

kiddz• 13 years ago

The USPTO should see this as an opportunity to have a mail-in form which would create redundancy and also obscure these problems.

burgreblast• 13 years ago

ha. I'll bet it's: LoC * Num_servers = 500M

At least, that sounds like a great way to charge gubbmints by LoC.

fmax30• 13 years ago

Who told them to use vanilla php to make everything , 500 million lines of code is an astounding amount of code . Next step put up a disclosure/FOI request for the source code of the healthcare.gov so that we can know what they were writing in there.

marginalboy• 13 years ago

I doubt you'd get it from an FOIA request. I believe there's an exemption that covers work like this.

valleyer• 13 years ago

Right — no one there knows how to edit the database credentials out of the PHP source. :)

spikels2• 13 years ago

"One specialist said that as many as five million lines of software code may need to be rewritten before the Web site runs properly."

Yikes! I don't think these "experts" are going to be fixing this anytime soon.

EdgarVerona9• 13 years ago

The sad part is that it sounds like they're already being paid to do so - they were quoted as being Federal Contractors.

We really need a public peer review system for technical contracting of government projects. This is getting insane. These schmucks fleeced 500 million from our government (us) for a barely functioning website, and now some snake oil contractors are going to come in to "fix it" and fleece us for even more.

hga10• 13 years ago

Given the obviously appalling management "these schmucks" were subjected to, per the article "In the last 10 months alone, government documents show, officials modified hardware and software requirements for the exchange seven times.", plus our hearing from multiple sources that there were getting change orders through the beginning of the week per launch, can you honestly say they fleeced us?

I can't imagine any way the contractors could have delivered a working system under the constraints we've been hearing, including most especially the unqualified Federal HHS CMMS government bureaucrats taking upon themselves the role of integrator of the 50+ parts of this system.

For that matter many people at all levels of this have been waving red flags, which the people running the show have studiously ignored. I mean, you know someone told them "you've got to freeze the requirements and give the programmers some time to get the job finished", not to mention "it needs more than one week of testing"....

Some of the remedial steps we're hearing about, like monitors for system bottlenecks, sound to me like things the integrator should have made part of the system from the beginning....

strict9• 13 years ago

While 500 million sounds really far fetched, this is the point at which projects explode with bloat and repeated code:

the separately designed databases and pieces of software from 55 contractors work together.

Factor in dealing with the multitude of state laws and companies they are interacting with, and you have a recipe for problems.

A lot of armchair quarterback technologists, but not a lot of precedent to compare computerizing a healthcare system in this way to.

ChuckMcM2• 13 years ago

Pretty sure that is a direct rectal extraction number not a real number. If there were 500 million lines of code you would have a prima facie case for fraud on the part of the contractor since government contractors are required to use best industry practices and no industry practice supports 10x code expansion.

spikels2• 13 years ago

I can just imagine some clueless bureaucrat bragging to his colleagues that his project has 500 million lines of code.

growupkids• 13 years ago

That's just not true. Contractors are only required to deliver what they proposed and the government agreed to.

ChuckMcM2• 13 years ago

Is it possible its different for DoD contractors? I ask because Lockheed in Sunnyvale took some heat (I believe a fine and a restriction on being able to bid on contracts for a period of time) for failing to use 'established industry practice' in the testing and delivery of what ever it was they were delivered. My neighbor at the time worked for them and recounted the gruesome (or in this case non-standard) details. I never really thought to question whether or not it was actually written that way, it just made sense that it would be. When I was with Intel and Intel was building milspec 8080s there were a list of standards that had to be met and all of the test equipment was calibrated based on that as well. Seemed like a pretty natural part of the government contracting business.

So you're saying that Contractors are under no obligation to meet standards? Or just that this contractor would be exempted? Or perhaps software in general would be exempted?

( Apparently NIST is involved somewhat : http://www.informationweek.com/government/enterprise-applica... )

potatolicious• 13 years ago

Ours is not the only corner of the industry - there are still a great many places where programmer productivity is measured by LOC or some other bullshit metric.

Hell, when I worked at Amazon my manager used sprint story points as individual productivity metrics. Oh God.

Our industry doesn't have a consensus on much of anything, least of all anything regarding counting lines of code.

derleth2• 13 years ago

They might be including all of the HTML in addition to all of the library code and all of the code in all of the interpreters and compilers they used, but even that probably comes up short.

Maybe if they include all of the possible HTML the site could generate, but counting potential code seems perverse even beyond the usual cluelessness of the mass media when talking about software.

eliteraspberrie• 13 years ago

Less than 10 million lines of code are required to fly an F-35 -- and that is C++.

Who is the contractor responsible for this? Perhaps the same people responsible for the Ontario EHealth scandal:

http://www.cbc.ca/news/canada/toronto/ehealth-scandal-a-1b-w...

bch• 13 years ago

Perhaps the same people responsible for the Ontario EHealth scandal…

I don't know if you were being facetious, but you're not far off:

http://m.washingtonpost.com/blogs/wonkblog/wp/2013/10/16/mee...

robomartin4• 13 years ago

I've said this before. The fact that these people (government and it's contractors) are going to have and handle our most intimate information AND that the IRS is also in the loop scares the living daylights out of me. The level of incompetence being revealed here is almost without description.

To go all the way up to opening day, have these problems and then offer that five million lines of code need to be redone is, well, I'm at a loss for words. I am trying to think about how badly one could fuck up five million lines of, say, Python. That's a lot of code. It's almost like saying that you set out to write a portal for the ACA and mistakenly wrote an navigation system for a new spacecraft.

In many ways this is great. I can't think of a better example of how bad big government can be and often is. It is hard to argue with what is unfolding before our very eyes: An attempt at centralized control that succeeded on the legislative front but is failing miserably in execution due to nothing less than incompetence at all levels. And, to boot, it is costing hundreds of millions and will probably cost billions if it doesn't implode under it's own mess.

Look at this in sharp contrast with what I happen to think would have been a far better approach: Free market decentralized healthcare. All they had to do is open interstate borders to launch all available companies into competition and slowly open other gates in order to promote free and open competition. Yes, judicious regulations would have been required here and there, but always with the goal of having an ecosystem that is as free as reasonably possible.

In this scenario at worst the government would launch a open-source projects to develop open and well-debated APIs for publishing and sharing all pertinent information. Various sites could then connect to any available insurance company through these standard API's in order to gather, present and sell insurance to anyone. Instead of having a badly executed centralized attempt to develop a portal you'd have thousands of entrepreneurs jump into the fray for an opportunity to offer the best possible services. Very soon the best-in-breed services would rise to the top and continually evolve better and better offerings.

Cheaper, faster, better. All three.

Instead we got this shit.

Zancarius• 13 years ago

To go all the way up to opening day, have these problems and then offer that five million lines of code need to be redone is, well, I'm at a loss for words.

There was an article on here last week (I think, my memory might be failing to that end) interviewing some individuals who claimed to be former contractors for CGI Federal. They also claimed that CMS, the agency responsible for integration (with no integration experience) essentially demanded that they perform no testing on the system until about 5 days before October 1st. I don't know how true these claims are, but given the nature of this whole fiasco, it somehow doesn't seem all that surprising.

Look at this in sharp contrast with what I happen to think would have been a far better approach: Free market decentralized healthcare.

Yeah, and the US government seems to favor regional monopolies that almost cleanly map to congressional districts. Maybe there's a correlation and someone's getting a kickback somewhere along the lines...

Cheaper, faster, better. All three.
Instead we got this shit.

More expensive, slower, worse.

I do have to wonder. If the government elects to go the route of a single payer system after they've largely screwed up the launch of this mess (and remember, ACA isn't fully implemented--the last bits of the law don't come into effect until 2018), is anyone is going to trust them at this point to do so?

jjoonathan4• 13 years ago

It is hard to argue with what is unfolding before our very eyes

What we are seeing right now (~.5B waste) is a drop in the bucket compared to waste in the US system as compared to socialized systems. We pay ~2x as much for comparable health care (per person per mean PPP-adjusted GDP/capita) [1]. That is about 250b/yr we are spending on a (more closely) free market system that keeps promising to be more efficient than its social counterparts and keeps failing to deliver, or even deliver in the same ballpark. I realize that HN is a site about code, but it's important to keep some perspective here. $500M (itself an inflated figure) is 0.2% of $250b. If it is hard to ague against $500M of waste, it surely must be nearly impossible to argue against $250b of yearly, recurring waste, no? How do you ignore these figures?!

If you're going to argue that our system makes up the difference with better outcomes, I want to see numbers. I happen to know that they don't support this argument, but I want you to do the search so that you can't dismiss my sources by assuming they were cherry-picked.

I happen to think ... Free market decentralized healthcare ... would have been a far better approach

Of course, $250b/yr includes the cost of providing care as well as administration. Perhaps administration of a free market system is more efficient? Not when you compare the US (31% administrative overhead) to Canada (16.7%) or the UK (6%) [2], [3]. This isn't a surprising result, either: I recently listed 7 reasons why exchanges should be more efficient than current individual plans [4] and you can add the profit margin and advertising costs to that list to come up with 9 advantages a single-payer system would have over a free market health care system.

Markets are good at optimizing things that can be quantified, so perhaps a better comparison would be between medicare and private insurance (not overall administration, just the insurance part). Again, the government systems come out ahead: Medicare overhead is 3.6%, Canada's insurance-related overhead is 1.3%, and overhead among private US insurers is 11.7% [2].

Cheaper, faster, better. All three.

[citation needed]

See [5] for a more complete overview of the pro-single-payer argument and a truckload of reputably-sourced studies and comparisons (as opposed to hand-wavey promises) to back it up.

[1] http://en.wikipedia.org/wiki/List_of_countries_by_total_heal...

[2] http://www.pnhp.org/single_payer_resources/administrative_wa...

[3] http://nhsrationing.org/tag/administrative-costs/

[4] https://news.ycombinator.com/item?id=6582753

[5] http://www.pnhp.org/facts/single-payer-resources

robomartin4• 13 years ago

> Cheaper, faster, better. All three. [citation needed]

Lookup the history of laser eye surgery over the last, say, ten years.

Free markets work. That's irrefutable. The problem is we've never had a free market system when it comes to medicine.

bluemanshoe2• 13 years ago

The entirety of the source code available in the debian repositories as of lenny is 324 million lines of code: [source](http://gsyc.es/~frivas/paper.pdf).

lifeisstillgood• 13 years ago

Again the argument that Government paid for code must be released under an Open License - all of the speculation and questions here would get wiped out by a simple git push from the right contractor.

And we could more fruitfully spend our time contributing to the automated test pack.

nostromo• 13 years ago

According to one specialist, the Web site contains about 500 million lines of software code.
One specialist said that as many as five million lines of software code may need to be rewritten before the Web site runs properly.

Oh, so only 1%!

Tangentially, why are so few quotes in this article sourced? I see no reason why they shouldn't be. It's all throughout the article... They keep quoting "specialists."

rwallace2• 13 years ago

In something this loaded with politics, I wouldn't imagine anyone involved would think it a good career move to have their names attached to bad news quoted in the press?

hga10• 13 years ago

The only people I've seen willing to go on the record have been:

"Stan Z. Soloway, president and CEO of the trade association that represents hundreds of government service companies" who parsed the contracts and independently confirmed that there's no prime integrator (i.e. the HHS CMMS is doing/failing that job, https://news.ycombinator.com/item?id=6569042).

One or more big insurance company CEOs; they're the ones who have to deal with the gubble the backend is delivering to them.

Various mostly bloviating politicians like John McCain.

Even yesterday's HHS blog posting (http://www.hhs.gov/digitalstrategy/blog/2013/10/making-healt...) is unsigned.

joe_the_user• 13 years ago

Perhaps because people working on the project would be fired for talking to the press?

You know, "war on whistle-blowers" and all.

harold• 13 years ago

From March of this year:

Chao said that he’d once held high hopes that the exchanges would run smoothly from the beginning, but that those hopes had been dashed. “The time for debating about the size of the text on the screen, or the color, or is it a world-class user experience, that’s what we used to talk about two years ago,” said Chao. “Let’s just make sure it’s not a third-world experience.”

http://www.forbes.com/sites/theapothecary/2013/03/22/cms-on-...

lmkg• 13 years ago

My own conspiracy theory: XML "code" got included, even the stuff that is unambiguously data. This may or may not include HTML templates (possibly machine-generated) and the data files for each individual health care plan. This is the result either of naive use of a line-counting utility, or a miscommunication between technical and non-technical folk, or both.

azifali• 13 years ago

Did they do a character count and submitted that as LOC to justify the high costs?..

downandout• 13 years ago

Let's be clear: This number could not be accurate, even if it included the line counts of licensed code (SDK's, database engines, etc.). The article failed to even cite its source, beyond saying that "one specialist" said this.

The NYT is a left-leaning publication. This seems to be a transparent attempt to try to soften the blow to the Obama administration's image by saying that implementing healthcare.gov was a Herculean task that no one could have gotten right on the first try because it required a half billion lines of code. It's no coincidence that this article emerged the day before Obama himself plans a press conference on this very issue.

semerda• 13 years ago

hahaha what a crock of bull. Probably the same experts that built it were quoted as saying that. Or maybe it is true... when you cut and paste every code snippet from the web and mush it up to create this site it does add up to 500m lines of code..

Anyone have detail on language they used and/or frameworks(s)?

Someone should be held liable for this. They just wasted tax payers money on would be experts and have to spend more tax payers money to fix it. Disaster!

noonespecial• 13 years ago

I suspect that as unbelievable as this sounds, it might actually be true. There must be a ton of "repeated code" to handle all of the different states and rules. I'm guessing that they started with one huge bloated "template" and then wrote automatic code generation systems that just copied everything hundreds or even thousands of times.

They could very well now find themselves in a position where it is humanly impossible to untangle it all and make even the smallest changes. I'm calling it failed at this point.

Edit: Title was changed from: Healthcare.gov "contains about 500 million lines of software code". This comment reflects this fact which seems unbelievable, but if true, I feel would likely mean that it was mostly created by code generators that probably copied and ossified original errors and misconceptions to the point that it is no longer possible to fix. Its a total failure, the money is gone, but they'll keep throwing even more after it, until the tiny, disjointed systems they develop in the mean time as "temporary solutions" eventually become the "real" system.

lifeformed• 13 years ago

Has anyone actually tried to look for health insurance using the site? It's indeed as bad as they say. It's probably the buggiest site I've ever used.

I've tried a dozen times over the span of several weeks, and get to a new impassable failure state each time. I'm literally sent to broken URL's that have variable names "?id=%applicationID%" in the URL instead of the actual value. Sometimes clicking "next" just simply refreshed the page with no new content.

jerrya• 13 years ago

One specialist said that as many as five million lines of software code may need to be rewritten before the Web site runs properly.

In comparison:

  Windows 95 had 15 million lines of code.
  Windows 98, 18 million
  Windows XP, 35 million

http://www.nytimes.com/2006/03/27/technology/27soft.html?pag...

Luckily they have about 8 weeks.

Contractors See Weeks of Work on Health Site

Related Stories