NY Times releases military surplus gear data on GitHub

https://github.com/TheUpshot/Military-Surplus-Gear

by frenchman_in_ny • 12 years ago

214 132 12 years ago

github.com

zrail• 12 years ago

This is awesome. Not really sure why they chose to release it as an XLSX document on GitHub, but it'll work!

shebson• 12 years ago

I'd assume it's an XLSX simply because that's the format they received it in from the Defense Department.

untog2• 12 years ago

Yes, the file is 1033-program-foia-may-2014.xlsx - just the fact that 'FOIA' is in there suggests that it's the original format.

aed• 12 years ago

Looking further at the metadata, the author is "Williams, John A DLA CIV DISPOSITION SERVICES" and the company is "Defense Logistics Agency"

pgrote• 12 years ago

There is a CSV uploaded as an update.

imacomputer2• 12 years ago

Maybe they are hoping that it will attract more attention from hackers who might do something cool with the data. I'd like to make an app that plots all of the purchases on a map, or finds all the military gear ordered near you. But I don't have the time.

lotsofmangos8• 12 years ago

Is interesting that this has made itself on here and not been flagged yet.

I have noticed that HN normally has an aversion to issues that are directly to do with contemporary politics outside the realm of telecoms, patents and copyright and I wonder if this seems more of an acceptable discussion for the site, given the wider issues raised, purely because the material being discussed is up on Github.

maxerickson• 12 years ago

There were similar substantial discussions a couple times in the last year or so:

https://hn.algolia.com/?q=police%20milit#!/story/sort_by_dat...

(the articles from the New Yorker, Salon and the Economist)

bradgessler• 12 years ago

In .xlsx format.

untog2• 12 years ago

https://github.com/DataGarage/node-xlsx-json

Node module to convert xlsx to JSON.

ISL• 12 years ago

No matter who was doing the acquiring, the "acquisition costs" of firearms are impressively low. Who buys a military-grade .45 pistol for $60?

bmelton• 12 years ago

I can't think of any others, but a $60 Colt M1911 is indeed a hell of a bargain. Unless it's in a dire state of disrepair, I'd say that price is about $800-900 cheaper than expected.

MBCook• 12 years ago

I see places where the prices change year to year based on when it was acquired. I wonder if it's a depreciated cost based on how old it is.

pmorici• 12 years ago

Those are probably not the actual acquisition costs the government paid. The way the surplussing process works is the people doing the surplussing just make a random guess at the acquisition value if the exact information isn't available. For most things that number isn't readily available to the person filling out the surplus paperwork.

frisbee14• 12 years ago

Imported it into Google docs: https://docs.google.com/spreadsheets/d/1AXPtmdbZ6zJph9Nly6_3...

(throwaway account)

sr-ix• 12 years ago

A contributor just uploaded the data in CSV format as well.

notjustanymike• 12 years ago

What's an "extreme mitten" ?

talmand• 12 years ago

Mittens intended for extremely cold weather, like San Francisco in the summer.

AmVess• 12 years ago

Extreme cold weather mittens.

barkingcat• 12 years ago

you can handle the Bhut Jolokia pepper with it.

crazypyro4• 12 years ago

Might just be the parts I looked at, but some of the data is bad. e.g. Repeated orders, orders with 0 cost, but then the cost is calculated into another line, etc.

Also why does the Florida Police (or anyone) need a 800k Mine Resistant Vehicle??

cwal37• 12 years ago

From the Article[1]: "The Times is now posting the raw data to GitHub here. With this data, which is being posted as it was received,"

The data they got was bad in general. Their original article[2] also discusses the lack of information out there.

[1] http://www.nytimes.com/2014/08/20/upshot/data-on-transfer-of...

[2] http://www.nytimes.com/2014/06/09/us/war-gear-flows-to-polic...

crazypyro4• 12 years ago

Ah, I didn't really read the articles, just dove straight into a python interpreter to start checking out the data. I figured it wasn't the Times' fault, but just a warning.

amcnett5• 12 years ago

I was surprised to see a number of mine-resistant vehicles associated with very rural counties in Washington State. Not being completely sure whether a mine-resistant vehicle was a tank or not, I googled the term and came across this article:

http://news.yahoo.com/as-wars-wind-down--small-town-cops-inh...

Choice quote from the article:

“Here’s the thing,” Shellmyer says. “Washington, Iowa, has 8,000 people. We have an MRAP now. We have a SWAT team. We have [police] dogs, and we have a SWAT team transportation vehicle that’s not armored.

The city councilman began to think: “Goodness, this is overkill.”

dublinben• 12 years ago

If we're lucky, those massive armored vehicles will be parked behind the station, and left to deteriorate. They're so expensive to operate, small departments are loathe to actually use them. If we're unlucky, we get the events in Ferguson, MO.

dredmorbius2• 12 years ago

Some years back I was in the position of managing a few systems for a nonprofit. It had been awarded equipment through a donation program. Some of this was usable. Some ... was not. One particular item turned out to be such a white elephant that it was literally more sensible to try to sell it and replace it with something more economical. Which other branches of the nonprofit were doing, though I never convinced the GM of ours to do.

Equipment that's not actually usable isn't an asset.

(And yes, I'm keeping this all pretty intentionally vague.)

barsonme3• 12 years ago

I live in Puyallup, Washington and we've received (what I believe to be) an MRAP. (It looks like one, but I've yet to see it outside of the building so I haven't gotten a very good look at it.) It's housed in the police/fire station downtown.

Puyallup is a "city" of just under 40,000. Our crime isn't even too horrible (http://www.piercecountycrimedata.org/NeighborhoodCrime/index...)

But we have an MRAP. It might be Pierce County's MRAP, housed in our building, but even then compared to the rest of Washington state, Pierce County is pretty tame.

Although, we do rank #9 in the U.S. for meth labs. So there's that. http://www.kplu.org/post/pierce-county-among-top-10-us-numbe...

amcnett5• 12 years ago

Maybe the MRAP was requested in the event that the Puyallup Fair (the WA state fair for the uninitiated) gets totally out of control.

RE: the meth labs, etc. it would be interesting to see usage data for the equipment. Who knows...if an occupied meth lab is raided every week, the vehicle could make sense (not that I have any tactical expertise in the slightest).

poulsbohemian• 12 years ago

I live in Walla Walla and I don't see our MRAP in the NYT spreadsheet, but here's the article: http://union-bulletin.com/news/2014/apr/16/walla-walla-swat-.... Absolute madness.

greedo• 12 years ago

Operational costs will prevent most of the MRAPs from seeing any use. 6MPG is horrible, and fuel costs are a big budgetary item for most public safety departments.

dragonwriter• 12 years ago

Operational costs will prevent most of the MRAPs from seeing any use.

Or, alternatively, MRAPs will provide the justification for greater budget demands.

baddox• 12 years ago

Does the federal government not provide funding for their operation as well?

hughdbrown• 12 years ago

From your article: "The supply of extra MRAPs is likely to only increase — the government spent $50 billion to produce 27,000 of them in 2007."

Can someone explain the math to me? It sounds like each vehicle costs $2 million.

thewopr3• 12 years ago

Quick question, is the apache license really the most relevant for a dataset? Does that license apply?

I ask only because I'm planning on doing something similar.

teachingaway2• 12 years ago

If the dataset comes from the Gov't (e.g., Freedom of Info Act), one might argue that its public domain data. If its public domain, then slapping a license on it doesn't change anything.

If its not public domain, and you want people to use it fairly freely, Apache seems like a good license.

Disclaimer: i haven't thought about this very carefully before posting. Could be wrong.

One thing I do know is that @copiesofcopies is a former EFF guy that knows open source licensing. Ask him!

thewopr3• 12 years ago

Yeah, I would tend to agree with this. These data are derived from work originally done by the U.S. federal government. If they had chosen to make it freely available without a FOIA, public domain would have applied. I don't think having it being a result of a FOIA changes anything.

pavpanchekha• 12 years ago

Roughly (I'm not a lawyer) data by itself is not copyright-able, but the organization and selection of data is, so most datasets do, in fact, have to come with a license to be usable.

rpedela• 12 years ago

Apache is probably okay for data, but I would recommend Open Data Commons or Creative Commons licenses.

http://opendefinition.org/licenses/

nightpool• 12 years ago

IANAL, but I'm not actually sure datasets like these CAN be copyrighted. Copyright rewards "original creative expression", (Feist v. Rural Telephone Service) not just effort, and I know things like phone books and other databases aren't protected by copyright. Generally, compilation protection is limited to things like charts and graphs, were the author has actually shown creative input into how the data is presented or displayed.

EDIT: But just to be sure, I would recommend something is well understood and is aimed towards a general usage, such as a Creative Commons license of your choice http://creativecommons.org/choose/

nroach• 12 years ago

The leading case on this is Feist, a telephone book case. http://www.law.cornell.edu/copyright/cases/499_US_340.htm As an interesting note, the phone company had been introducing fake listings: "Four of these were fictitious listings that Rural had inserted into its directory to detect copying." For copyright, originality is a constitutional requirement and copyright cannot attach to a bare compilation of known facts. However, "if the compilation author clothes facts with an original collocation of words, he or she may be able to claim a copyright in this written expression."

The court held that "the names, towns, and telephone numbers copied by Feist were not original to Rural and therefore were not protected by the copyright in Rural's combined white and yellow pages directory. As a constitutional matter, copyright protects only those constituent elements of a work that possess more than a de minimis quantum of creativity."

geetee2• 12 years ago

What in the world do police need parachutes for?

geetee2• 12 years ago

Monroe, Kentucky's refrigerator was a steal for only $2,394,800.0

giarc2• 12 years ago

Morgue?

pinoceros• 12 years ago

BASE jumping?

base698• 12 years ago

You certainly aren't BASE jumping with military parachutes. At least not more than once.

crazypyro4• 12 years ago

The same world where just these departments have a combined MRAP value of over $56 million.

edit: jk, thats just the first page.

ianstallings3• 12 years ago

Don't give them any ideas!

ianstallings3• 12 years ago

What I don't get about this is, this is a non-hacking topic. An outside discussion. So why is my post, obviously in jest, inappropriate?

krawczstef• 12 years ago

Where's the D3 vis of it? :D or iPython notebook on it? ;)

comrh2• 12 years ago

Yikes:

AL,MONTGOMERY,1010-00-691-1382,"LAUNCHER,GRENADE",1,Each,720,2006-04-12

crazypyro4• 12 years ago

There's 84 total grenade launchers in the data set, if you are curious. Florida took twice as many as CA (36 to 18) and the rest of the states with them are under 10. Why Florida needs so many is another question...

Also interestingly, Montgomery got all 9 of AL's grenade launchers, more than Sacramento or LA took.

edit: actually, this data is wrong because I only used the first page, didn't even see the other 3....

comrh2• 12 years ago

There was a big thing made about it in Bergen County, NJ when it was first reported that they had received one. The police basically said they had no use for it and couldn't even image a reason to use it.

giarc2• 12 years ago

Why does any police force need a grenade launcher? Unless these are used for tear gas or something.

venomsnake• 12 years ago

When you have a hammer ...

Rise of the warrior cop was very interesting read. With the gear comes the cop attitude. If you want that kind of toys in you state, you shouldn't be allowed to have them.

ceejayoz3• 12 years ago

Presumably for tear gas / smoke rounds?

amcnett5• 12 years ago

Probably not t-shirts at Gators games.

rrggrr• 12 years ago

Disappointed but I guess not surprised to see the small number of helicopters, armored vehicles and assault rifles provided to southern border states. Federal government appears happy to distribute heavy weapons anywhere except where they are needed most as evidenced by actual incidents.

idlewords• 12 years ago

You realize the Mexican War is over?

whyaduck• 12 years ago

As a resident of Arizona, I can tell you that however inefficient and/or incompetent you think the federal government's border agencies are, most of the state and county agencies in this state are far worse. I'd prefer some of them have nothing more deadly than a water pistol (especially the Maricopa County Sheriff's Office - headed by the country's most corrupt un-indicted law enforcement official). Local law enforcement officials are generally competent and savvy enough to want nothing to do with the kind of work you seem to imply they should be doing.

PhasmaFelis• 12 years ago

The equipment is distributed in response to requests from police departments. (source: http://www.nytimes.com/2014/06/09/us/war-gear-flows-to-polic...) I'm afraid you'll have to blame the locals, not the feds, for being soft on those sinister Mexicans.

lotsofmangos8• 12 years ago

How long before state police start pointing these back upward? I mean, in some areas it is basically handing military gear to secessionists, which does not seem all that bright. I certainly wouldn't like to be the federal official charged with the job of asking for any of them back.

ceejayoz3• 12 years ago

One county on the map has ~16k population and 640 military surplus assault rifles. If they're equipped with 30 round magazines, they can shoot everyone in the county without reloading once.

staunch10• 12 years ago

No, they couldn't shoot more than a few people before thousands of armed citizens made them stop.

To help clarify things: the city of LA has 10,000 police officers and 1,000,000+ armed citizens.

amcnett5• 12 years ago

Disorganized pseudo-infantry (i.e. pissed off individuals with guns) will always be inferior to organized paramilitary forces with armor.

jfoutz• 12 years ago

Yeah, that worked really well in Ludlow. I can't think of any other cases in the US where armed civilians were up against automatic weapons. I'd be delighted to see some more examples.

lotsofmangos8• 12 years ago

You already have the National Guard out in Missouri, and they have Blackhawk helicopters which have chain guns as their standard armament.

The idea that in the US civil militias could really hold their own against the state is an absurd fiction.

Unless significant portions of the armed forces defected to join an uprising and managed to take their equipment with them, then it would be very one sided.

The militias will have hand guns and rifles, the state has everything up to and including nukes.

There is a probably apocryphal quote attributed to Isoroku Yamamoto, that if you tried to invade the mainland US, there would be a rifle behind every blade of grass. Isoroku Yamamoto was born in 1884.

War has progressed since then.

Satellites can look behind every blade of grass and let you know where the rifles are, then you can send in the drones while you sit back in your bunker and sip tea.

mrexroad• 12 years ago

i might just move there. sounds like better odds than most places during zombie apocalypse.

wisty• 12 years ago

It's lucky crime rates have dropped so low. You wouldn't want this kind of hardware lying around if there was any chance the wrong king of person could get their hands on it.

It would make a great movie or game premise though.

existencebox• 12 years ago

It's called saints row. (Although more of a comical than a dystopian twist, but the latter is certainly hinted at.) One of their primary plotlines is taking down an overmilitarized police force with their own gear.

teachingaway2• 12 years ago

I see 205 of "Launcher, Grenade" followed by only 1 of "Laundry, Cleaner"

BugBrother• 12 years ago

As a European which don't follow the subject of US police, my TV habits are not crime dramas, I just have to ask:

WTF are police going to do with grenade launchers?! (Do they have 155 mm artillery too? :-) )

Are those for tear gas? I just don't believe even US gangs are doing Taliban style insurrections or Intifadas?

(Being Swedish with our really open borders for refugees from the Middle East, I should just wait a few years and find out why police need heavy weaponry? :-) :-( )

Edit: Other commenters seem to say (a) These really aren't needed by police, (b) there are lots of surplus weapons from Iraq/Afghanistan. Why aren't those extra weapons just sold/donated to countries which need them? Like most neighbors of Russia, they ought to be starting with big defense investments right now (except for the complete crazies, see Swedish politicians.)

learc83• 12 years ago

Are those for tear gas?

Almost definitely. There are a variety of less than lethal projectiles that can be fired from a 40mm grenade launcher. Everything from tear gas canisters to rubber projectiles.

Even the most overzealous SWAT team wouldn't use a grenade launcher with real fragmentation grenades in a populated area. Fragmentation grenades kill by producing shrapnel, which is just too indiscriminate for police use.

That being said, I do remember a story about a SWAT team attempting to drop a homemade bomb out of a helicopter, so you can't be 100% sure.

Edit: Found a link to the police helicopter bombing story. The improvised bomb caused a fire that spread to 50 or 60 other houses. http://www.nytimes.com/1985/05/14/us/police-drop-bomb-on-rad...

cmsmith• 12 years ago

Note that Washington, DC seems to have received 17 military cargo planes despite not having an airport (also shows up on nytimes.com piece). Is it possible that this is a mistake?

stormcloud• 12 years ago

I can only imagine this is an error, but apparently Brevard County, FL has received at least 61 helicopters... 8 of them worth $18m a piece.

Someone tell me this is misreported? There is no way a county of ~500k people could need 61 helicopters, rght?

amcnett5• 12 years ago

Imagine, or hope?

guimarin• 12 years ago

You don't think they 'actually spend $20k on a hammer, $30k on a toilet seat, do you'[1]? If this is not an error, I think we just found the best justification I've seen for making all budgets/purchases etc. in government publicly available via API, as this and the ones below are amazing finds.

[1] http://www.imdb.com/title/tt0116629/quotes?item=qt1534129

abritishguy• 12 years ago

I can't find the 20k hammer or the toilet seat.

mludlum2• 12 years ago

Jefferson county in WV did pay $189 per a latrine though...

superuser2• 12 years ago

Not to excuse it all, because I'm sure there's plenty of waste and fraud, but sometimes militaries/governments need things that are more robust than the seemingly just-as-good civilian alternatives.

Why doesn't Curiosity run on a Dell Inspiron? Why can't the President drive a used Honda Civic? Why can't the drone programs just retrofit toy helicopters?

(Hint: there are good answers to all of these questions.)

I've always found this West Wing clip interesting: https://www.youtube.com/watch?v=7R9kH_HOUXM

draker• 12 years ago

Patrick Air Force Base is in Brevard County.

"Headquartered at Patrick Air Force Base, Florida, the 920th Rescue Wing is the Air Force Reserve Command's premier combat search and rescue (CSAR) unit. The wing consists of over 1,500 Airmen, trained and equipped to locate and recover U.S. Armed Forces personnel during both peacetime and wartime military operations."

http://en.wikipedia.org/wiki/920th_Rescue_Wing

mludlum2• 12 years ago

Well,they needed 3 Kiowa(http://en.wikipedia.org/wiki/Bell_OH-58_Kiowa) & a lot(54) of Bell UH1-H's.....How do they afford to maintain a fleet that big?

grecy• 12 years ago

There is no way a county of ~500k people could need 61 helicopters, rght?

Need has nothing to do with it.

DanielBMarkham• 12 years ago

Couple of random observations after looking at the data:

1) Looks like a contest where the winners are folks who are able to pull the biggest political strings in DC. It has nothing to do with forces needing anything. This was a free-for-all giveaway. It's all political.

2) In defense of DoD, once you build this crap, it's gotta go somewhere. Maybe some of these MRAPs could only be used for local parades. Local PDs do not need this junk, but the rest of the country doesn't need it either. Better dumping it on Deputy Joe than letting it rot. As long as Joe doesn't start getting delusions of terrorism.

3) A lot of the local rural counties where I live got M16s. I guess that seems useful. The M16 is a fine rifle, and if you could get every one of your cops a proven rifle it sure beats trying to get the poor local governments to pay for them. The cities are a different story: much better connected and with more resources (and ambition?) to screw as much out of the system as possible. The nearest city to me is getting all kinds of idiotic stuff. Do we really need 20 $4k night vision sniper scopes? Perhaps if the city were attacked by protesting vampires such a purchase would be useful. But I guess guys gotta play with their toys.

The distribution alone is not worth ranting about. The problem is what happens once all this junk is distributed. If you own a bear gun, you have a tendency to go out into the woods hunting bears. You also see evidence of bears where others do not. That's the real problem.

ejfox• 12 years ago

Better dumping it on Deputy Joe than letting it rot.

This is a central assumption the DoD/Government has been making, that is turning out to be misguided if not flat-out wrong. The police in this country are not properly trained as to when to deploy this equipment, and when it's deployed, how to use it.

Just as spreading weapons of war to all parts of the middle east hasn't exactly lead to peace there, spreading weapons of war all across America will hardly lead to an orderly society. It seems like if you give men weapons, they itch to use them, or the power it gives them.

alxndr• 12 years ago

Yeah,

As long as Joe doesn't start getting delusions of terrorism.

...seems to be the real problem.

lifeisstillgood2• 12 years ago

I presume this is an obvious leap, but in Ferguson the BBC reporter just commented on the tensions between the Black population and "the security forces".

I used to scoff at the "militarisation of the police" stories on HN. I assume this is the same mental leap everyone else is having?

partacus2• 12 years ago

Sorted from most greedy to least, wow, look at Florida:

State Amount FL $252,801,365.49 AL $117,321,970.75 TX $93,960,116.66 CA $92,199,594.55 TN $88,036,810.05 GA $74,258,127.54 IL $63,856,929.46 SC $49,446,894.25 MI $43,557,552.29 IN $43,153,872.84 OH $41,066,993.83 KY $38,672,666.63 PR $37,712,132.94 AZ $37,173,754.35 OK $26,113,459.50 NM $25,077,905.94 NY $24,920,353.44 VA $24,257,646.82 NJ $24,075,459.47 WA $23,543,976.61 AR $21,806,380.46 DC $21,741,478.09 LA $20,327,539.50 CO $17,701,285.74 MO $17,481,149.80 NC $17,296,016.90 DE $12,483,178.35 ME $12,048,389.87 MA $11,878,711.88 WI $10,122,693.61 WV $8,970,650.95 MT $8,943,135.43 NH $8,806,115.87 MN $8,580,400.72 CT $7,920,540.33 IA $7,493,026.26 ID $7,491,468.74 MD $7,049,130.61 OR $6,891,336.22 PA $5,942,289.17 NV $5,836,317.88 GU $5,466,524.59 NE $5,430,787.55 RI $4,812,144.16 WY $4,575,149.39 KS $4,009,658.36 ND $3,871,164.59 UT $2,264,747.07 MS $1,791,294.53 SD $1,771,105.98 VT $1,609,630.11 AK $706,554.76 HI $521,054.41 VI $228,504.00

munificent• 12 years ago

That's a meaningless comparison unless you account for population differences. Per capita would be useful to show.

partacus2• 12 years ago

Do you really think per capita works in this situation? It's not like your average citizen can pick up a surplus grenade launcher. Breaking down geo-politically would be more useful.

But grabbing per capita values from bjterry and going by party...

Senator Avg per Capita

Democrat $4.14 External (VI, PR, ..) $33.19 Independent $9.07 Republican $7.21

bjterry• 12 years ago

Here it is per capita: https://gist.github.com/BJTerry/d9392dc1d494cad052ef

eitally• 12 years ago

I think what might make more sense is to account for the location of large military bases.

orf2• 12 years ago

AL,CHEROKEE,4930-01-442-9106,"DRUM,HOSE REEL",1,Each,1630.97,2013-06-20

AL,CHEROKEE,5120-00-222-2220,"MALLET,RAWHIDE ",3,Each,50.31,20120518

CA,LOS ANGELES,3530-DS-SEW-ING0,"INDUSTRIAL SEWING MACH, MOBILE TEXTILE",2,Each,0,2014-02-19

Stop this madness.

dandelany• 12 years ago

Really? The dataset contains a record of tiny towns receiving MRAPs, but for some reason your biggest concern is LA getting a sewing machine? Why is this a problem?

orf2• 12 years ago

It was sarcastic, I thought that was pretty obvious?

jaredstenquist• 12 years ago

The DOD gives this shit away (to our rural towns and cities) so they can consider it in use and order more from contractors, which gives more cash to the lobbyists and contractors. Rinse and repeat. It's very straightforward.

Game_Ender• 12 years ago

On MRAP front the military is getting rid of them for a good reason. They ordered the initial batch very quickly and ended up several different platforms [0]. None of them fit the bill perfectly and having several variants made logistics and vehicle operations difficult. They then did a real procurement program and ended up purchasing several thousand M-ATVs. The surplus MRAPs went into this law enforcement give away system [1].

0 - http://en.wikipedia.org/wiki/MRAP#MRAP_deployment

1 - http://en.wikipedia.org/wiki/MRAP#Post-war_reductions

wcummings• 12 years ago

Their Github has some other repos w/ data, and a few w/ code (to process the data etc)

dmourati• 12 years ago

NPR covered this military surplus program. The most frightening fact was the typical local law enforcement agencies must agree to "use it or lose it" with the acceptance of the gear. This sets up a perverse incentive and encourages local PD to misuse military hardware or forfeit it back to the federal government.

NY Times releases military surplus gear data on GitHub

Related Stories