Doge Developed Error-Prone AI Tool to "Munch" Veterans Affairs Contracts

https://www.propublica.org/article/trump-doge-veterans-affairs-ai-contracts-health-care

by afavour • 1 year ago

156 111 1 year ago

"AI is the wrong tool to do this" someone said.

Different things need to be addressed, like how someone is able to act outside all three branches of government, and have enough power to destroy the lives of those who protected our country when we needed them most.

It should be crystal clear to even the most skeptical people should turn their opinion on how they bankrupt the poor and bail out the rich every single time.

rsynnott8• 1 year ago

This whole thing is just kind of astonishing.

“I think that mistakes were made,” said Lavingia, who worked at DOGE for nearly two months. “I’m sure mistakes were made. Mistakes are always made. I would never recommend someone run my code and do what it says. It’s like that ‘Office’ episode where Steve Carell drives into the lake because Google Maps says drive into the lake. Do not drive into the lake.”

Well, then why the hell did you write it? For purely entertainment purposes? I mean, what.

EDIT: Oh, wow, just got to the prompt. Did he... did he just think LLMs were _magic_? "Level 1: Necessary consultants that can't be insourced" - how the hell is ChatGPT supposed to know what can and cannot be insourced?

In late March, Lavingia published a version of the “munchable” script on his GitHub account to invite others to use and improve it, he told ProPublica. “It would have been cool if the entire federal government used this script and anyone in the public could see that this is how the VA is thinking about cutting contracts.”

No, that would not have been 'cool', you idiot. Bloody hell.

John23832• 1 year ago

... and get paid to do it while sidelining anyone with actual experience in the sector/topic.

I called this when it started. They would come in, be totally ineffective to the point of breaking things, and then we would get a "shucks, sorry" later.

rsynnott8• 1 year ago

I just can't believe how totally incompetent this whole thing is. Even for a Musk/Trump venture, I didn't expect quite this level of just complete nonsense.

itsdrewmiller• 1 year ago

He didn’t get paid.

nxobject2• 1 year ago

The only silver lining here is his transparency about being wrong -- and providing evidence of that publicly. Given DOGE's culture of prioritizing avoiding accountability and transparency measures like FOIA [1], it's almost sadly laudable.

[1] https://www.cjr.org/feature-2/trump-100-legal-battle-for-dog...

rsynnott8• 1 year ago

But he still doesn't seem to get the seriousness of the situation, at all. Like, he's saying no-one should have used his stuff, but also "It would have been cool if the entire federal government used this script".

For instance, imagine you have surgery, and die of a complication. Your relatives will feel very differently, and be in a very different legal situation, if the hospital's explanation is "surgeon was the bottom of their class" vs "yeah, we just handed a visiting 5 year old a scalpel to see what would happen". Neither are ideal, obviously, but the latter is _criminal_.

The idiot in the article is clearly closer to the 5 year old than the scraping-a-pass surgeon; he is simply not in any way a proper person to have been given the sort of access or authority he was, this would have been obvious to whoever gave it to him, and whoever gave it to him is at least morally culpable.

dylan6043• 1 year ago

DOGE isn't SpaceX where getting it wrong is a chance to learn for the next iteration. These are human lives being affected by all of these mistakes that they are so nonchalantly making as if the humans are just NPCs in whatever game they are playing. Not only are these mistakes directly affecting humans with the loss of government jobs, they are also affecting citizens that depend on the services these employees worked. I don't care about your silver lining. The damage is done, and it is not easily unwound. All because they had no shits to give and too young to have the life experience to even come close to understanding just how damaging their decisions were.

afavour4• 1 year ago

IMO this is indicative of the kind of questions anyone with DOGE on their resume will need to answer at their next interview.

- why did you do it?

- why did you not quit as soon as you realized how dangerous this was going to be?

"I did what I did, I knew there were mistakes, I don't really know what was done with the output of my work and I never tried to find out" are absolutely disqualifying statements for anyone above associate level engineering.

nilamo2• 1 year ago

"I needed money." Why is that not a good enough answer?

krapp• 1 year ago

Incompetent and destructive as they are, DOGE only did what every company and startup in the world is trying to do, aggressively integrate AI in a way that radically upsets the societal paradigm and makes a ton of money. And much of tech buys into the politics and narrative. They're rock stars. They'll never even need to interview to find work, work will come to them.

delusional2• 1 year ago

disqualifying statements for anyone above associate level engineering.

Only if you're an engineer. The Silicon Valley folks love this sort of lawless disregard for wider society.

RickJWagner2• 1 year ago

I’d do it, gladly.

I worked on the Obamacare site after it rolled out, crashing. I gave it my best effort.

I’m in favor of progress and will work for the common good. I think that’s the best way.

kjkjadksj• 1 year ago

They will probably get a favorable role at a thiel or yc associated company

andrewflnr• 1 year ago

Employers inclined to ask those questions will more likely just toss any resumes mentioning DOGE at an earlier stage of filtering.

nop_slide2• 1 year ago

did he just think LLMs were _magic_?

If you happened to follow this particular developer, yes he does think they're magic.

zeta0134• 1 year ago

It doesn't even take very long messing around with the things to quickly realize how much smoke and mirrors is involved. Next time you get a good response from your favorite LLM, go ahead, hit "regenerate" a few times. Look at how different the responses can be, from the same initial prompt, with just a liiiiitle push in the RNG weights. Sometimes it's remarkably correct! Sometimes it's remarkably *incorrect* and confident about it. Which one did you get this time?

esafak• 1 year ago

@sahillavingia is a member and wrote about his DOGE experience recently: https://news.ycombinator.com/item?id=44116061

FrustratedMonky2• 1 year ago

Maybe this engineer is just saying, like all big projects, there are mistakes, and there should be more vetting, debugging. It was rushed, so yes, there are mistakes, but not, "generally, all my stuff is bug ridden crap"

mdhb• 1 year ago

But in this case it WAS all bug ridden crap. Casually rolling the dice with people’s lives like this means you probably won’t be able to live without some fear of people coming looking for you when this is all said and over.

Maybe they don’t realise it yet but their lives have just changed for the worse in a very significant and hard to reverse way. This is going to catch up with them on a reasonable timescale.

davidcbc3• 1 year ago

This isn't your typical SV startup. Move fast and break things works for garbage like Juicero or Uber for Dogs or whatever silly idea gets VC funding this week, it does not work when you're talking about the lives of millions of people

FrustratedMonky2• 1 year ago

I was making a comment on software projects, not a political statement. Thanks for the downvotes. Of course the situation is awful, but not every programmer gets to cherry pick the picture perfect project to work on.

The job market is difficult right now, so who wouldn't take this one? That's the more insidious message from capitalism, there are a lot of cogs in the machine that are fully aware they are cogs, but you still have to eat, so you keep doing cog like things.

TrapLord_Rhodo2• 1 year ago

I think you are misinterpreting the comment here.

I would never recommend someone run my code and do what it says.

Using this as a tool to signal can be a huge time saver. But not taking the code output as "truth" is what the engineer meant.

It’s unclear how many have been or are on track to be canceled.

This article, just like you, are taking all of these out of context.

Google Maps says drive into the lake. Do not drive into the lake.

He's using LLM's as a potential flag, then you go and human review those contracts with lawyers, supply chain, business specialists, nurses on the ground, etc. etc.

Calling him an idiot for being transparent is a personal attack, and has no place here.

Directly from the github: Tools I have written to do stuff more effectively at the VA. Code exists to make humans more efficient; not to replace them. All code leads to human review. Feedback appreciated!

thisisit• 1 year ago

Spoken like a developer who doesn't understand how real world works.

Using this as a tool to signal can be a huge time saver. But not taking the code output as "truth" is what the engineer meant.

Lavingia didn't provide any provable metric that this tool was going a "huge time saver". Additionally, if the output isn't the "truth", what can kind of time saving is expected? That a person needs to run the code, get result and then spend time analyzing results and cross verifying contracts again?

You know what will be a good time saver? Open the contracts and cross check it if it is valid. No AI slop required.

This article, just like you, are taking all of these out of context.

Real people are impacted by VA contracts. If there is even 1 contract impacting even 100s of people that is enough damage. Saying that this is people taking things out of context shows your of empathy for other human beings, at least if they are not coders.

Because apparently Lavingia should get kudos for his transparency. But calling him an idiot for his lack of self-awareness that he had no clue about government contracts and should not be writing AI and creating this mess, all the while impacting people's life? Well we can't have that.

He's using LLM's as a potential flag, then you go and human review those contracts with lawyers, supply chain, business specialists, nurses on the ground, etc. etc.

If human review is the point then start from there. Don't write AI slop, generate wrong result and then ask for human review.

To the larger point - Benjamin Franklin said - 'It is better 100 guilty Persons should escape than that one innocent Person should suffer'. In the same vein every well meaning contract/legislation which is helping 100s will have 10s exploiting the loophole. People like Lavingia and from DOGE seemed to have decided that rooting out those 10s is worth hurting the 100s. So, lets write an AI for that.

ryandrake• 1 year ago

Mistakes Were Made

https://en.wikipedia.org/wiki/Mistakes_were_made

bix6• 1 year ago

This is honestly unbelievable. Sahil wrote code he knew was bad and it was used to terminate vital medical contracts… how is that not fraud against every person who pays taxes?

xnx• 1 year ago

This is all facilitated by congress and the president. Destruction under any pretense was the goal. They must all be voted out.

glookler• 1 year ago

Voted out? Crimes have been committed and the President as dictator nonsense has to be cancelled so these people can all go to federal prison where white collar criminals belong.

masfuerte• 1 year ago

But there is no way that is happening while they remain in power.

eviks4• 1 year ago

Expected levels of YOLO incompetence, but still fascinating.

joshstrange• 1 year ago

“I think that mistakes were made,” said Lavingia, who worked at DOGE for nearly two months. “I’m sure mistakes were made. Mistakes are always made. I would never recommend someone run my code and do what it says. It’s like that ‘Office’ episode where Steve Carell drives into the lake because Google Maps says drive into the lake. Do not drive into the lake.”

Sahil Lavingia is a snake. I thought his blog post that was on HN a week or so ago displayed a frightening level of cognitive dissidence and now he wants to distance himself from it all. No way buddy, this will be a millstone hung around your neck. You encouraged this, you participated in this, you cheered this, you will live with the consequences.

mvid• 1 year ago

He will probably become a Thiel fellow or raise from YC

smackay2• 1 year ago

I'd be very interested in learning more about how DOGE got staffed.

If your goal was to dramatically cut government spending, then hiring bright, young people, with no prior experience, who axed first, and asked questions later would be the way to do it, otherwise you'd get bogged down in details since there was probably a good reason, at least initially for the said spending.

However, if you really, wanted to make a spectacular mess then hiring bright, young people, with no prior experience, who axed first, and asked questions later would be the way to do it.

Somebody, somewhere, thought this was a good approach. How could they not know it would turn into a massive clusterfuck. Hubris?

cwillu• 1 year ago

Searching the page for “hiring bright, young people, with no prior experience, who axed first, and asked questions later” shows two results, was that intended?

smackay2• 1 year ago

Not at all, the slightly snarky wording, was typed on the spur of the moment. Incidentally, I tried the same terms on Google and the AI-powered answer was rather informative with the pros and cons of the approach. Indeed the cons seem to be playing out before our very eyes.

polygotdomain• 1 year ago

To start, it was clear that many were Musk fan boys. I don't think "bright" was a driving force, but "young, with no prior experience" would be great if you wanted people to make decisions that have the potential to negatively impact millions of lives. You need people with a lack of context to realize those are the stakes at hand. Since it was also clear they were going to "throw AI at it", I also doubt they were looking for anyone who acknowledged the shortcomings and pitfalls of an all AI approach.

I think this approach got them exactly what they wanted. Musk clearly wanted access to as much data as he could get his hands onto; legally or not. There were contracts and investigations into Musk's interest that he wanted control over, and got. Overall, chaos is not seen as an issue for those in this administration. The attitude from the top was clearly that these agencies and contractors "deserved it" for some reason, even though they have no idea what they really do or why they're doing it.

crypto_throwa• 1 year ago

I'm in a chat with one of the engineers on DOGE (young college dropout), and they're trying to recruit more young college dropouts to work on DOGE.

I would characterize some in this group as believing they're smarter than everyone else or anything that's been done before, so yes I think it's pure hubris.

There are a lot of bright people in the chat working on very important things, but they're not the ones joining DOGE.

TrackerFF2• 1 year ago

Here's one guy that posted about his short stint with DOGE, when it was still in the planning stages

https://vinay.sh/i-am-rich-and-have-no-idea-what-to-do-with-...

https://news.ycombinator.com/item?id=42579873

"When I got back home and regaled my friends with my mountain stories, one of my friends joked that I should work for Elon and Vivek at DOGE and help America get off its current crash to defaulting on its own debt. So I reached out to some people and got in. After 8 calls with people who all talked fast and sounded very autistic smart, I was added to a number of Signal groups and immediately put to work.

Working for DOGE for 4 weeks, remembering the power of urgency

Within 2 minutes of talking to the final interviewer for DOGE, he asked me if I wanted to join. I said “yes”. Then he said “cool” and I was in multiple Signal groups. I was immediately acquainted with the software, HR, and legal teams and went from 0 to 100 taking meetings and getting shit done. This was the day before Thanksgiving.

The next 4 weeks of my life consisted of 100s of calls recruiting the smartest people I’ve ever talked to, working on various projects I’m definitely not able to talk about, and learning how completely dysfunctional the government was. It was a blast.

I learned about the power of urgency and having an undeniable mission. Not by reading it somewhere. By experiencing it. I came to realize how laughable my robotics stint had been in comparison. And I started to realize that, although the mission of DOGE is extremely important, it wasn’t the most important thing I needed to focus on with urgency for myself. I needed to get back to ambiguity, focus on my insecurities, and be ok with that for a while. DOGE wasn’t going to fix that.

So, after 4 intense and intoxicating weeks, I called off my plans to move to DC and embark on a journey to save our government with some of the smartest people I’ve ever met. And I booked a 1-way ticket to Hawaii."

ujkhsjkdhf234• 1 year ago

All of these people sound like naive psychopaths.

davidcbc3• 1 year ago

The next 4 weeks of my life consisted of 100s of calls recruiting the smartest people I’ve ever talked to

This kid needs to talk to more people

rsynnott8• 1 year ago

So, after 4 intense and intoxicating weeks, I called off my plans to move to DC and embark on a journey to save our government with some of the smartest people I’ve ever met. And I booked a 1-way ticket to Hawaii.

These aren't serious people.

insane_dreamer3• 1 year ago

Serious amount of Kool-Aid being served up here.

quickthrowman2• 1 year ago

I remember reading this when it was first posted and being amazed it wasn’t satire.

antisthenes2• 1 year ago

Sounds like a micro-dosing narcissist nepo-baby who got rich working at a tech company and doesn't know what to do with themselves, because they assume everything in life is going to be so easy as to not worth doing seriously.

spaceisballer• 1 year ago

You have to demonize the government in order to come in and start trashing the place. Obviously there is going to be some waste to trim. But instead they came in, asked zero questions and started arbitrarily cutting contracts and jobs. In the end all the cuts will just make the government dysfunctional and cost us more than it did before.

rsynnott8• 1 year ago

They seem to have gone for option 3, based on the article; hiring _staggeringly incompetent_ people with no prior experience. No-one bright would be saying, to paraphrase the article "no-one should have acted on the output of my shit software, but also it would have been cool if other federal departments had adopted my shit software".

(It's very characteristic of a certain type of AI booster, though; "it's fine as long as you don't use the results for anything, also you should buy more".)

techpineapple• 1 year ago

I imagine it was always a weird conflict of intentions.

On the one hand if you're really trying to disrupt an industry, you want to hire at least a good percentage of people who don't understand the industry, so they're not biased by the set of circumstances you may be trying to disrupt - and Thiel and Yarvin and maybe Vance and Elon certainly wanted to disrupt the government. Like Thiel and Yarvin probably don't want people who understand how to renegotiate government contracts, they just want people who know how to burn them to the ground.

But I imagine there were very few people in government, including some of the people "over" them like the senate and house wanted real disruption, and certainly most of the population didn't want real disruption, and Trump and his administration probably didn't want real disruption that would impact their popularity.

So it was probably doomed from the start.

nop_slide2• 1 year ago

Relevant Github repo of the code he apparently used from the article:

https://github.com/slavingia/va

gorbachev• 1 year ago

Is there a way to -star a github repo?

mountainb2• 1 year ago

"That’s not possible — you have 90,000 contracts,” he said. “Unless you write some code. But even then it’s not really possible.”

This entirely possible with some lawyers, some business analysts, perhaps some hospital administrative consultants, and ordinary support staff. That team might even use LLMs in some capacity but not in the way described by the article. Reviewing 90,000 hospital and other service contracts sounds like just another project for a mid-sized or big law firm; or the government. That is how those contracts were created in the first place.

This is like the meme about someone's uncle talking about how there are these 90,000 contracts that no one knows how to review because we've forgotten how to do it. If there's something America still knows how to do it's how to review tens of thousands of turgid government contracts.

ianhawes• 1 year ago

Not possible within the 30 day timeframe they were given.

mountainb2• 1 year ago

With the right teams and resources, this could be done in a shorter time. With that team and skillset there was no way to complete this in a way that did not result in a lot of contract liability to the government and other silliness.

TrackerFF2• 1 year ago

They also put a 23 year old to essentially greenlight or stop NSF grants.

https://www.techdirt.com/2025/06/02/a-23-year-old-crypto-bro...

I just don't understand how these people can do what they do? If I was a fresh college grad, and put in that position - to have the final say on whether experts in their fields can get grants, to conduct research on something I have no expertise in...I don't think I could do that job. I'd feel like the biggest empty suit in existence. Don't they feel any shame?

Or do they suffer from some superiority complex, where they think that whatever knowledge they have in AI/ML makes them the smartest person in the room?

EDIT: Reading through it, at least the guy acknowledges his errors.

morkalork2• 1 year ago

There is a belief that by having grants address DEI and other issues, you're already putting them at the mercy of unqualified individuals and as such "what's good for the goose is good for the gander". Now of course there's some pretty big, atrocious, historical reasons why those parts of the grant application exist.

kevin_thibedeau• 1 year ago

I just don't understand how these people can do what they do

It's narcissistic personality disorder. The 45th administration had a surfeit of responsible adults putting up guard rails to contain the future felon. This one has taken the Hitler youth approach to recruiting like-minded individuals with an inflated sense of their own competence. Minimal critical thinking skills and life experience are a checkbox in the hiring process.

insane_dreamer3• 1 year ago

I don't think I could do that job. I'd feel like the biggest empty suit in existence.

You don't have that level of arrogance. But I've met people who do. They're the worst people to work with.

zenburnmyface• 1 year ago

Lavingia should be ostracized from tech for this. What an embarrassment.

mhalle• 1 year ago

His github repository also contains the code that prompts OpenAI to analyze and flag VA memos that might contradict hot button issues from Trump's Executive orders.

https://github.com/slavingia/va/blob/main/eos/analyze_eos.py

System prompt:

    You are an AI assistant that analyzes internal government memos for compliance with 2025 Executive Orders. You identify references to DEI, gender identity, COVID policies, climate initiatives, and WHO partnerships that must be removed or modified according to new directives. Provide detailed analysis and be precise about what needs to be changed."

User prompt:

   Based on this memo analysis, provide a detailed assessment of non-compliance with 2025 Executive Orders:
        
        MEMO CONTENT:
        {text[:10000]}
        
        Rules for determining non-compliance:
        
        1. DEI/DEIA Content:
        - References to "diversity", "equity", "inclusion" in policy context
        - Mentions of equity action plans
        - References to Chief Diversity Officers or similar roles
        - Performance criteria based on DEI targets
        
        2. Gender Identity Content:
        - References to gender identity or preferred pronouns
        - Content about gender ideology or training
        - Instructions to use non-binary pronouns
        
        3. COVID Policy/Telework:
        - COVID-19 vaccination requirements
        - Pandemic-era remote work extensions
        - Masking or testing policies that remain in effect
        
        4. Climate/Environmental Content:
        - References to climate resilience, zero-emission requirements
        - Environmental justice language
        - Sustainability criteria in purchasing/contracting
        - Plastic straw bans or similar procurement restrictions
        
        5. WHO Partnerships:
        - References to World Health Organization partnerships
        - Alignment with WHO protocols or frameworks
        
        6. Other Key Areas:
        - Affirmative action or racial/gender preferences in contracting
        - $15 federal contractor minimum wage references
        - Student loan forgiveness expansions
        - Paper check disbursement references

[Edit: formatting.]

liampulles• 1 year ago

On the surface, the principle of using AI to filter down contracts for human review seems like a good thing, because it increases efficiency. Especially if it makes something previously infeasible and/or very expensive now doable and/or relatively cheap.

Where it would become bad for me is if the next step is taken to remove humans from the loop, or more simply if the AI is not tested for accuracy regularly and updated as the nature of the world changes. Unfortunately, I think the chance of these outcomes occurring is high.

I've worked with LLMs to aid humans in the domain of text review. I've encountered inexplicable inaccuracies that are very difficult to robustly remove from the system (you can patch one thing and then another pops up when you try some new data). I've found it very difficult to try and explain this issue to business people: I suggested that there needed to be humans in the loop, because it seemed like some level of inaccuracy is insoluble.

This is not ideal for the business, because obviously it hurts their ability to scale (reasonable). Also from the perspective of business people I've interacted with: AI is the greatest thing since the invention of the transistor, and its only a matter of time until these LLMs are perfectly intelligent (not reasonable), and vitally important that they get ahead of the curve before potential competitors do.

I can just see how this issue is going to lead to so many important aspects of our society being eaten up by imperfect software that we are forced to accept, because there are not competitors able to justify a less scalable and accurate alternative. I hope I'm wrong, but until then I hope people will flag this sort of shit when they see it.

RickJWagner2• 1 year ago

Glad to see AI being put to use with the big wheels of government. It’s inevitable, might as well start setting up best practices.

insane_dreamer3• 1 year ago

Arrogance, inexperience, and haste, is a really bad combination when it comes to building fault-tolerant code. And when you're handling government systems, you need fault-tolerant code. This isn't some Tinder app.

Would you hire any ex-DOGE engineer? I ~~probably~~ would not.

cavisne• 1 year ago

Anyone who thinks the VA is well run should watch a few episodes of the “financial audit” YouTube channel. The most financially irresponsible people in the world (and generally bad people in general). Almost all are cashing a “disability” payment from the VA.

Doge Developed Error-Prone AI Tool to "Munch" Veterans Affairs Contracts

Related Stories