U.S. military to analyze 350B social media posts to understand popular movements

https://www.bloomberg.com/news/articles/2019-05-25/u-s-military-to-trawl-through-350-billion-social-media-messages

by pseudolus • 7 years ago

135 80 7 years ago

skilled• 7 years ago

In a perfect world people would use this as an opportunity to tell the government to shove it. But, sadly, it's not a perfect world.

westmeal• 7 years ago

People are just too afraid or they don't give a damn.

sambull• 7 years ago

Probably just increase your shadow threat score.

JumpCrisscross4• 7 years ago

In a perfect world people would use this as an opportunity to tell the government to shove it

In your vision of a perfect world. Personally, I’m fine with this. They aren’t using call logs or other private data. Information on social media is public or semi-public. Fair game, and I’m curious about the results.

(Agree that a more productive public debate is needed. But that shouldn’t start by presuming values.)

keiferski3• 7 years ago

So you're okay with someone following you 24/7/365 every moment that you step outside your house? By definition, this type of surveillance data would still be public information.

abdullahkhalids• 7 years ago

The problem in this case is not that private data might have been used. The problem is that the outcome of this research will be the US military developing a weapon that can be used to create or stop or influence political movements across the world (including inside the US). The question is whether it is a good idea to create such a weapon. War is inevitable but ever since time immemorial people have respected certain rules of engagement. Its difficult to draw the line in the sand to decide what type of propaganda is unacceptable, but it should be obvious that propaganda created using every dark pattern in the book, trained on 350B social media posts is probably not going to bode well for Earth's future. Its a lose-lose situation.

BurningFrog• 7 years ago

I think you just did.

Perfect world achieved!

lettergram• 7 years ago

If they are using public posts, this seems fine. It’s more of a general study and could have a real impact on our understanding of how stories spread (at least via this medium, today).

tty2300• 7 years ago

What is not fine is the fact that social media companies have spent years using dark patterns and tricks to get us to publish more and more of our info publicly. It used to be common knowledge that you never post any of your real life details on the internet but now the most popular websites will ban you for not using your real name.

austincheney4• 7 years ago

I remember a decade ago when http://pleaserobme.com/ came out and how my coworkers were shocked. I honestly thought over-sharing was a common sense bad idea, but it was not as common sense as I had thought. They did not require any dark patterns to willfully give up everything to social media.

isostatic• 7 years ago

It used to be common knowledge that you never post any of your real life details on the internet

But before then it used to be common knowledge that you always post your real life details on the internet, at the very least your name and email, often your phone and office address too

Nasrudith• 7 years ago

The real concern isn't the privacy given that it is publically available but what it says about their motivations and respect for civilian control of the military.

Monitoring pacifist groups based on publicly available information may respect rights technically but it is not a good look. It indicates either major bad intentions to subvert democracy or sets the stage for any who plan to abuse it while being peripheral to their nominal core functionality.

To give a crude analogy it may be technically legal to note the comings and goings of my neighbor and every aspect of their home security but it would be very understandable if they think I plan to burgle them.

jmartrican• 7 years ago

I just assumed this was going on already. Maybe because they are working with a professor on this effort it had to be revealed. I wonder if behind the scenes the Navy or the military had already done this multiple times and this is just another attempt.

Zenst• 7 years ago

Or by announcing it, they give the impression that it is something they have not done previously. Indeed, they might want to see how some people react to this and a sudden stream of deletions of social media posts by individuals/groups after finding out about this - would be of interest too them.

Another possibility is that the military like a large company has many arms and legs and highly possible that it is something that is already done, but by some seperate part of that collective that the rest are not aware of. I've seen a company the size of 100 have two people do the same project for nearly 6 months before they realised about the other and the duplication of work. More people, more separate divisions/locations, more chance duplication on some levels happens.

For me, I suspect the later, but equally mindful that as a tactic - doing something and then after you have done it, say your going to do it to gauge reactions - is a tactic that would not be out of place in any military intelligence collective.

mlb_hn3• 7 years ago

US Government's been great at collecting tons of data. Analyzing it... not so much.

squarefoot2• 7 years ago

"Analyzing it... not so much."

Or maybe they have learned a few things by reading Sun Tzu.

AndrewKemendo2• 7 years ago

This is a scary headline for a run of the mill research project at Naval Postgraduate School. Here's the solicitation: https://www.neco.navy.mil/synopsis/detail.aspx?id=537809

Our research aims to provide enhanced understanding of fundamental social dynamics, to model the evolution of linguistic communities, and emerging modes of collective expression, over time and across countries ...
As a central requirement for this research, we seek to acquire a large-scale global historical archive of social media data, providing the full text of all public social media posts, across all countries and languages covered by the social media platform. We aim to use this research to advance knowledge through scientific publications. This data will also be used for pedagogical purposes in the classroom, giving students new opportunities for thesis research and the development of “big data” analytic skills.

How can the Department of Defense do a better job at interfacing with the general public to better explain intentions, motives etc... with the work that we do?

soulofmischief10• 7 years ago

They could start with being honest that those aren't the DoD's only motive behind wanting such a corpus and analytical model.

There are many positive and negative things that could result from something like this, and it isn't the fact that the Navy is doing it that scares people. It's the fact that the US government is doing it.

austincheney4• 7 years ago

What does that mean? Would you rather it were Cambridge Analytica?

AndrewKemendo2• 7 years ago

They could start with being honest that those aren't the DoD's only motive behind wanting such a corpus and analytical model

That would be akin to asking [insert company] if it will possibly ever use data collected from one thing on a different project at some later date. That is a complete unknown, so it wouldn't even be appropriate to put into a proposal, as they have specific formats.

NPS is a graduate study program, with a research component and long history of academic publishing. Whether that data could possibly, someday be used beyond that is completely out of scope for this solicitation. Further, if other groups in the military wanted similar data, they would likely not go to NPS for it, for a variety of reasons.

I think at the end of the day though, if someone has an assumption that everything the DoD does or messages to the world is somehow subterfuge, there's no amount of discussion that would reverse that feeling.

techrich• 7 years ago

If you post it publicly, then its fair game what it can be used for.

luckylion5• 7 years ago

Should you expect & accept for everything you speak in public to be recorded and monitored? Is it "fair game" to do facial recognition on everybody that shows their face "in public"?

soulofmischief10• 7 years ago

Walking around the park and speaking privately with a friend is not the same as making a permanent, public post on Twiter.

mtgx2• 7 years ago

Why exactly is the military doing this? What's the business of the military what "popular movements" do inside the country?

atemerev• 7 years ago

It is no secret that foreign actors (including Russian and Chinese military intelligence) are employing “active measures” to influence political changes in Western countries. Additionally, information operations are now a part or modern warfare, in much of the current conflicts.

The US military is interested in mapping these activities and developing countermeasures.

maxheadroom• 7 years ago

...information operations are now a part or modern warfare...

Information/disinformation operations have been a part of warfare since... ...forever it seems. It is - certainly - nothing new. The medium has merely changed and that's what important to the staging of the front[s], as it were.

You should pick up a copy of Eric Frank Russell's WASP[0], if you haven't read it before. It's a very good read in this area.

[0] https://en.wikipedia.org/wiki/Wasp_(novel)

faissaloo• 7 years ago

They want an excuse to get paid to look at dank memes

rjf722• 7 years ago

As always, propaganda. For instance we recently made made major efforts to try to overthrow the government of Venezuela that included, directly or indirectly, digital propaganda efforts. These efforts are ongoing but seem to have failed. Why? It can also be used internally. For instance in the 60s see things such as COINTELPRO [1] or the FBI's efforts to attempts to blackmail and push MLK to suicide. [2]. COINTELPRO is probably where you can draw the most analogs relevant to today - planting fake stories in the media, subverting groups through internal infiltration, compelling 'private' media to prohibit access to targeted groups, character assassination, etc. What are the effects of these actions on digital media, and how can it be used to pursue our interests? The same is true of MLK. The FBI's primary goal was simply to destroy him as a leader. What causes leaders of online movements to disappear in ways that help splinter groups rather than turning them into martyrs?

Bigger question is why in the world this would be made public. I think it's a 100% safe bet to say that the military/intelligence has been engaged in this research for many years already. I see no positive benefit to making this information public aside from preempting 'leaks' / creating preemptive PR.

[1] https://en.wikipedia.org/wiki/COINTELPRO

[2] https://en.wikipedia.org/wiki/FBI%E2%80%93King_suicide_lette...

itronitron2• 7 years ago

The US military engages in a lot of aid/relief work, this may help them better understand where that work will best be received and its benefit.

rjf722• 7 years ago

Is your comment meant to be sarcastic? If not, can you elaborate on why - of all possible options for the department who's primary purview in practice (in contemporary times) is the destabilization and/or destruction of other nations, you think this would be the most probable explanation for a mass scale analysis of 350 billion social media posts?

If that message comes off abrasive it's only because I am genuinely confused, but would like to know where you're coming from. There's always a risk of us living in our little bubbles and so I make every effort to try to see things through the eyes of others, especially when I simply cannot otherwise empathize with their views on any level.

2Ccltvcm2• 7 years ago

Government aid is a proxy for soft power through population pacification, political subversion, intelligence gathering operations, and proselytization. They know exactly who they want to target at all times.

hevi_jos• 7 years ago

It is their main business. It has always been the first step before war against other country to support revolts and popular movements.

Sometimes, countries protect themselves using revolution in other countries. For example Lenin wanted to make the revolution in Germany, as they believed Russia was not prepared for revolution. Lenin was captured in Germany and released with the condition of not trying it again on Germany(or he would be shooted and killed) and sent to Russia, where he started the Soviet revolution.

For example, United States wanted to control the Panama channel, so they supported an "independence" movement against Colombia, so a new "free" country was created, that by "coincidence" gave the US the total control of it for a long time. Even today, the US is in control of it.

Before that, when the US felt the Spanish Empire was weak, they supported the "Cuba(and Philippines) libre" movement. People in Cuba and Philippines were very happy for the help until they realized the helpers army did not go away after the "independence". In Philippines the US exterminated over a million people, forcing everybody to speak English.

After Panama, the US continued supporting coups in other America's countries and around the world.

One of the most important movements the US supported was the popular movement in Persia, now Iran, because they wanted the British Empire to leave(so their own companies could enter and control the oil).

In the end, this movement converted Iran from secular country to a radical Islam one, and the Americans were ousted.

For the US it is routine creating havoc around the world, as the hegemonic power in the world that they are. All empires before had done that too.

They would love to be able to support popular movements in China or Russia as it is a threat for US hegemonic power in the future. Getting united China to disintegrate in several smaller countries would be a great victory.

But Chinese people already play the game.

tehjoker• 7 years ago

It's fine, they just would like to create their own Salvador Allende in the United States if things go left.

2Ccltvcm2• 7 years ago

Social media is at the very core of 4th generation warfare. Social media is more powerful than nukes because he who controls the discourse of the public is more powerful than he who destroys the public. He who controls the social network can make people believe whatever he wants them to believe to achieve his goals. Vastly more powerful, invisible, and humane than nuclear weapons, especially in terms of reach. What we are already witnessing online is an information war which hijacks and disrupts our social connections and thought processes to replace them with highly engineered stimuli which steer the people to think and act in a certain way. Neural networks are in development which will provide a positive feedback loop to continually evolve puppet account discourse in a way which maximally achieves the goals of a behavioral campaign. Phrases which maximize psychological impact for each campaign will be identified and automatically evolved based on replies and related discourse to always maintain maximum engagement as the overton windows shifts. These experiments are already well underway.

Anon84• 7 years ago

This seems to be a bit of too much ado over nothing. This type of projects have been going on for over a decade now (just search for Twitter or Facebook in Google Scholar). Even the government has been involved in financing some of this research (For example: https://www.iarpa.gov/images/press/osi/24_open_source_indica...).

Many interesting things can be done with this kind of data (and it's what originally got me interested in data science https://www.springer.com/us/book/9783319140100). That being said, the specific solicitation (https://www.neco.navy.mil/synopsis/detail.aspx?id=537809) doesn't seem particularly exciting or novel. Just an overgrown grad school project to keep the students busy. With some Python experience and patience to collect the data, pretty much anyone can do something similar.

You can find the slides for one of my old (PyData 2015) tutorials on social media mining here: https://www.slideshare.net/bgoncalves/mining-georeferenced-d...

uncoder0• 7 years ago

Yep this is nothing new. Was working on similar and it's all public domain back in 2013-2014.

femiagbabiaka• 7 years ago

Certainly much ado, but not necessarily about nothing. Military sanctioned experiments, no matter how simple, can be used to justify action later on.

coldtea• 7 years ago

This seems to be a bit of too much ado over nothing. This type of projects have been going on for over a decade now

Yeah, the Overton window has now moved, fascism is now acceptable. /s

jcims• 7 years ago

Is there something recent that brought ‘Overton window’ into common use? I honestly believe the first time I’ve ever seen the term was within the past few months and now I see it regularly.

cjslep• 7 years ago

I remember being a uni student at a career fair nearly a decade ago and the folks at the CIA booth were wanting to hire to "predict the next Arab Spring" (which at that time was fully underway). When I asked a little bit, they didn't divulge more and weren't interested in me (my degree is not comp sci).

I've just been assuming this sort of work has been going on since then.

chippy• 7 years ago

Right, thats about when I also heard pitches about the same. Perhaps social media has changed now. I certainly use it differently. Or perhaps the US Military takes a decade to get research like this going?

dmoy• 7 years ago

Probably it's ongoing in defense research, just usually not much comes of it and it's not published a lot. I did something tangentially like this if you squint at it the right way, maybe 10-12 years ago, for some defense research grant. It would shock me if they didn't have a dozen other parallel grants going with other companies at the same time, and more since then.

dictum• 7 years ago

predict the next Arab Spring

The idea of the military arm of the U.S. government predicting the next Arab Spring reminds me of that classic mind-bender of a quote, which I think is often misunderstood as a defense of lying to the public, when it is about something deeper:

The aide said that guys like me were "in what we call the reality-based community," which he defined as people who "believe that solutions emerge from your judicious study of discernible reality." I nodded and murmured something about enlightenment principles and empiricism. He cut me off. "That's not the way the world really works anymore," he continued. "We're an empire now, and when we act, we create our own reality. And while you're studying that reality -- judiciously, as you will -- we'll act again, creating other new realities, which you can study too, and that's how things will sort out. We're history's actors . . . and you, all of you, will be left to just study what we do."

https://www.nytimes.com/2004/10/17/magazine/faith-certainty-...

GreedCtrl• 7 years ago

Have you heard of the free energy principle? The two ideas seem related.

From Wikipedia:

The free energy principle is that systems—those that are defined by their enclosure in a Markov blanket—try to minimize the difference between their model of the world and their sense and associated perception. This difference can be described as "surprise" and is minimized by continuous correction of the world model of the system. As such, the principle is based on the Bayesian idea of the brain as an “inference engine.” [Karl] Friston added a second way to minimization: action. By actively changing the world into the expected state, systems can also minimize the free energy of the system. Friston assumes this to be the principle of all biological reaction.

zozbot123• 7 years ago

Maybe we could all settle on a common insult towards the government then?

freeflight• 7 years ago

I would be very surprised if this hasn't been going on for years already, back in 2014 they were already as far as using Facebook users as test-groups [0].

Then there was also the case of a Pentagon contractor leaving their AWS buckets open exposing billions of scraped social media posts. [1]

As much as people can reach for benign reasons for "research" like this, the reality is that it's not solely done to "understand" movements but rather to better influence them [2].

[0] https://news.vice.com/en_us/article/zm5k4a/the-troubling-lin...

[1] https://www.upguard.com/breaches/cloud-leak-centcom

[2] https://www.theguardian.com/technology/2011/mar/17/us-spy-op...

Gpetrium• 7 years ago

The reality is that private entities and external actors are already using data to figure out better offensive and defensive maneuvers. If the entities of your nation do not take similar steps, they are likely to be ill prepared to "serve & protect" you and your interests, which may include a fair and functional democracy.

Some may ask "why is the military involved" the answer relates to the fact that today's military have realized that foreign actors will weaponize technology (e.g. internet, social media) to serve their goals. To ensure that the goals of your nation are protected, the military has moved more and more towards the cyberspace.

Here are a few benefits that can come from such endeavor:

* Relief efforts - How do people react in social media before, at the time and after a disaster happens? How can social media be used to connect to the ones in need? How can X entity use social media to advise groups in a risky area (e.g. landslide) to diminish damage and risk of life?

* Trolls - Are there cues that can be used to determine when someone is a troll? Can that information be used to help society become more resilient in the future?

* Training - Can this research help dozens/hundreds of personnel to gain valuable know-hows and capabilities in the cyber arena? Will that knowledge translate in a more capable entity?

Another question to ask is whether X entity follows your ideological view and whether there is enough oversight. If the answer is no to either/or, then as part of a democracy, you should raise your concern with representatives & the people around you.

chiefalchemist• 7 years ago

"...and individual users won’t be identified..."

Perhaps not as individuals, with enough personal meta data your identity can be reverse engineered.

That aside, anyone you believes there is no hidden agenda here is naive. Any understanding will have a second benefit for doing the opposite (i.e., preventing movements).

jolfdb• 7 years ago

That's a boring complaint. Any tech can be used for good or evil. When is a movement and movement, and not a guerilla terrorist cell network?

SturgeonsLaw• 7 years ago

When is a movement and movement, and not a guerilla terrorist cell network?

That depends on who wins the war, and thus writes the history books. From the point of view of the British, the American Revolution was an act of terrorism and treason.

Attempts by the military to understand (and, one would assume, prevent) popular uprisings would nip a second American Revolution in the bud, for better or worse.

The linked article refers to an attempt to entrench the status quo, and not everyone is pleased about that.

U.S. military to analyze 350B social media posts to understand popular movements

Related Stories