The Follower: Using open cameras and AI to find how an Instagram photo is taken

https://driesdepoorter.be/thefollower/

by maciejgryka • 4 years ago

499 108 4 years ago

driesdepoorter.be

londons_explore2• 4 years ago

Down without a mirror...

proto_lambda• 4 years ago

https://archive.ph/YjOfv

Rias• 4 years ago

Looks like the website is down, here's the YouTube video about the project: https://www.youtube.com/watch?v=sd_P0LxIBGI

BuildTheRobots• 4 years ago

Archive.is managed to archive it: https://archive.ph/YjOfv

allenu• 4 years ago

Seeing the context in which the photos are taken (busy streets, kind of grimy looking) makes you realize how powerful a camera is at creating an illusion. That is, taking a good photo means composing your shot so that you don't include elements you don't want in it. What you end up with is a photo which makes it appear you're the only one at a location, or that a location is more beautiful or exciting than it actually is. Your brain fills in all the details of what's outside of the frame, which often isn't what's really there.

leobg2• 4 years ago

Exactly. “The photographer’s eye”. Photography isn’t much less art than, say, painting. Not much different than writing a novel, reporting on a court case or sharing your version of events to your marriage counselor.

hackpert• 4 years ago

That is actually pretty freaking cool. Not hard to do by any means, just cool.

londons_explore2• 4 years ago

The machine learning element isn't really needed if the instagram posts are tagged by date/time...

FrenchTouch42• 4 years ago

You could have taken the picture a day before or am I missing something? The post date/time itself does not mean much.

fortran773• 4 years ago

I think it's harder than it looks to do it reliably and automatically.

matt-attack2• 4 years ago

We have no idea as to the reliability of the system. We don't know how many false positives were found. We don't know if these were just hand selected from date/time/location.

jsdwarf• 4 years ago

What does "open camera" mean? Public webcam? Hacked CCTV? How did the author find those cameras?

netsharc2• 4 years ago

Considering the open camera "feeds" have the bounding boxes around people, I guess they're CCTVs mistakenly put on the open Internet, findable through some services (can't remember the name of a famous one).

Well actually he probably added the bounding boxes. DDG result for "Dublin Temple Bar livestream": https://www.earthcam.com/world/ireland/dublin/?cam=templebar

Ftuuky• 4 years ago

Shodan is the name IIRC

htrp• 4 years ago

The Temple Bar (Dublin) actually makes this publicly available. They also have a feed of the inside of the bar so that you can virtually experience Ireland.

https://thetemplebarpub.com/

aaaaaaaaaaab2• 4 years ago

https://www.tfljamcams.net/

Traffic cameras are publicly available in London. The first thing I used to do when coming to the office each morning was to look myself up in the traffic cameras along my journey (historic footage is also publicly available).

ElCheapo• 4 years ago

It's not surprising, but I find very cute the fact that the further you are from the center the older the security cameras are.

penneyd• 4 years ago

Do you know why it's just London?

Frost1x• 4 years ago

Many state department of transportation groups in the US have camera on roadways all alone their respective state online. They're usually fairly low res, probably to reduce costs and data as they only need to check traffic flow, but they're often public. Check your states DOT website and look for traffic, there could be a treasure trove of cameras.

doodlesdev• 4 years ago

What the actual fuck. This is actually frightening. I can watch anyone, anywhere, anytime in London, anonymously, without them knowing? I guess I have to add yet another city to my never-visit-again list. No wonder schizophrenia is more common in developed countries, you ARE being watched.

Remember that this is still 2022. People think there's no privacy anymore, but in a few decades we are going to look back and wish we could go back. Of course these are public spaces where privacy has never _truly_ existed, but having cameras transmitting your every step to anyone with internet access is chilling.

bochoh• 4 years ago

A really famous spot in Dublin is the Temple Bar (Red bar in examples). They have live cameras in HD available inside and out https://worldcams.tv/ireland/dublin/temple-bar

bla3• 4 years ago

There are a whole bunch here: https://www.earthcam.com/mapsearch/

MonkeyMalarky• 4 years ago

This reminds me of an old 4chan "prank". There was an open camera somewhere in New York(?) and in the view was a shop with various display stands. Users would post something like "Im gonna do it, watch!", everyone would start watching the video feed and like half an hour or so later a kid would run up and knock over one of displays. They'd just fuck with the poor shop owner whenever they were bored.

Edit: https://knowyourmeme.com/memes/events/nyc-cardstand-earthcam...

naillo3• 4 years ago

Not creepy at all

dylan6042• 4 years ago

How is this any more creepy than any of the other people in those public locations videoing the same thing? Every day, you are recorded by hundreds of cameras, and a lot of those cameras are privately owned/operated. Specifically, I'm talking about people's smart phones. No matter where you go, just take a look around you, and you will probably be in someone else's camera frame.

There is no expectation of privacy in public. This just drives it home.

naillo3• 4 years ago

Both are really creepy. It becomes more creepy when you go from a recorded video that no one really watches to one systematically analyzed with AI though.

drewbeck• 4 years ago

It's creepier because it's connecting it to real life people's accounts, meaning anyone can then trace that person in the video and find out so much more about their life or stalk them etc.

vitiral• 4 years ago

Scale

jdelman• 4 years ago

"Enemy of the State" (1998) tech becoming reality.

ahofmann• 4 years ago

Well, some of the things shown in the film were already reality at that time. Only that one was called a conspiracy theorist, if one has noted this at that time.

jcims• 4 years ago

Reminds me of this RadioLab episode about a company that loiters over areas with a high resolution imaging platform that allows them to 'rewind' from the scene of a crime to see where the suspects came from:

https://radiolab.org/episodes/update-eye-sky

The company in question:

https://www.pss-1.com/

jstrieb• 4 years ago

If you're looking for more info, there is a whole book that goes beyond the content of this episode. I found it to be an informative, and eye-opening (albeit disconcerting) read.

https://www.amazon.com/Eyes-Sky-Secret-Gorgon-Stare-ebook/dp...

tomatocracy• 4 years ago

It also made me think of this: https://www.bbc.co.uk/news/science-environment-20629671

All of these are good illustrations of the idea that by putting multiple data sources together, even "anonymous" data sets can be de-anonymized surprisingly easily.

Breza• 4 years ago

I'm working on a project to study Reddit data to see how many users have unique footprints. I'm only looking at the subreddits where people participate. If you participate in at least five subs, you're probably the only person with that unique combination of subreddits.

woevdbz• 4 years ago

All these could be submissions to https://instagram.com/influencersinthewild

altacc2• 4 years ago

Some people put a lot more effort into their photos than I do! Although as they selected people with 100k+ followers I guess taking good photos is basically work for them (or rather their patient partners).

matt-attack2• 4 years ago

There's something about it that's so undignified, and cringe inducing to me. To see someone pose, repose, inspect, repose, etc. I just could never want to put that much effort into how I'm appearing. The faux spontaneity is just so off-putting.

Shacklz• 4 years ago

The faux spontaneity is just so off-putting.

I think it would be really good educational material. Showing the silly dance that is performed removes all the glamour from the result, and might help alleviating the bad psychological effects that glamour is causing

bil7• 4 years ago

capturing a vibe can be very lucrative if you have the follower count. Authenticity doesn't matter if your post still gets likes and shares.

sangnoir2• 4 years ago

Seeing how the sausage is made almost always results in disillusionment regardless of the industry.

Some of the magic was taken away when I learned that certain scenes a movie I liked required dozens of takes: it felt too mechanical/industrial and went against my romantic notions of how art is made. This is not far off from the pose-repose-inspect cycle you noted, but has a much larger supporting crew (and budget) and far longer hours.

HHC-Hunter• 4 years ago

Agreed, Instagram is the epitome of civilian posting.

mjwhansen2• 4 years ago

very thought-provoking - people think only a portion of what they’re doing is shared, but government sees all

secretsatan2• 4 years ago

These are probably privately owned cameras, which are far more widespread, i don’t think governments have what meta has on you.

I’d go so far as saying, hand wringing over government surveillance has served as a smokescreen over the proliferation of private surveillance.

Not that i want to advocate for government surveillance, just that, in my experience, people who worry so much about it seemed to have entirely missed how much private companies have expanded into public space.

anony23• 4 years ago

Private surveillance IS government surveillance at the end of the day. Government doesnt need to build an all seeing eye when they can just buy the data.

mjwhansen2• 4 years ago

These are traffic cameras

bolasanibk• 4 years ago

I wonder what the storage requirements for video from multiple traffic cameras for weeks is? A few TB per day?

jewel• 4 years ago

It depends on the bitrate. Let's assume 10 mbps for five cameras. That'd be 16.4 TB for a month's footage.

Depending on the application, you could lower the bitrate significantly during times of low motion or low detail (such as at night). You could also drop the framerate.

I've found that when the scene is static, without panning, and no wind blowing leaves on trees, you can compress really well. I have a Wyze outdoor cam looking at the front yard that does 1080p at a little less than 1 mbps.

fortran773• 4 years ago

Some of the photographers and subjects seem to know exactly the right angles to appear more "fit."

yieldcrv3• 4 years ago

Yes, this is common

To detect faux-lean people you can understand that the angle shot has evolved to being from a low angle to the ground

And also wearing black should immediately be a flag for further scrutiny

It is the first suggestion for slimming attire and it is for the optical illusion for their own confidence and having other people want to engage with them

As if there is some awareness that everyone stopped saying anything about it but still aren’t personally interested

fortran773• 4 years ago

I've heard them called "MySpace Angles!" (See https://knowyourmeme.com/memes/myspace-angles )

NIL8• 4 years ago

This is a mixed bag of both good and bad. Imagine a domestic violence victim being tracked/stalked by their abusive partner. Conversely, allowing citizens to monitor these cameras could potentially help prevent/solve crimes. In the end, it's probably a Constitutional dilemma that will find it's way into the high courts someday soon.

MisterTea2• 4 years ago

This is why I have no social media accounts. No one knows what I am up to. No one knows where I have been. No one can find me.

Do not track is now a lift style.

dvngnt_• 4 years ago

you can be found easily if someone really wanted to. social media is just one way you can be tracked

JohnFen3• 4 years ago

Do you use a credit/debit card? Do you pay utility bills, rent or a mortgage? Do you carry a cell phone? Do you drive a car? If you said yes to any of these (and many more!) then you are not invisible at all.

MisterTea2• 4 years ago

This is in response to broadcasting everything you do on social media. This is not hiding from determined actors targeting you.

yieldcrv3• 4 years ago

This involves actually going out of your way to posting and tag the location

But you dont have social media so you wouldnt know

JohnFen3• 4 years ago

Man, that is creepy as hell. Perhaps it can be useful to really bring home to people how they're being spied on every day.

bosswipe• 4 years ago

Why is it creepy? The influencers already publicly posted their location and photo.

rexpop• 4 years ago

What makes an "influencer" less deserving of privacy in the dimensions they haven't shared? Furthermore, what makes someone an influencer, when this technique can be used on smaller accounts, too.

JohnFen3• 4 years ago

It's not about the influencers. It's about the rest of us, and how pretty much anybody can do this sort of thing with anyone for any other purpose.

It's just another piece of evidence that we really are living in dystopian times.

mikotodomo2• 4 years ago

Not everyone on insta publishes where they took the photo or wants it to be known. But if someone with an AI watches every camera in the world, they could easily find lots of people.

lilsoso2• 4 years ago

Great project.

Is the code for this project or similar projects available on Github?

The unstated implications of all this are unsettling.

lifeisstillgood• 4 years ago

I realised what they were doing on the second photo/video.

OMG that's the most terrifying thing I have seen in ages.

Amazing, innovative, tipping-point-sign, but still, terrifying

Cyberdog• 4 years ago

I was scared at first too until I realized we're talking about people already taking photos for the purpose of posting to Instagram anyway. It's involuntary surveillance while participating in voluntary surveillance.

somebodynew• 4 years ago

It's only following people who posted their own photos voluntarily in this particular implementation, but consider the next incremental step for this same approach: a website where you upload a photo of anyone's face and get a map/trail of every public photo/video posted by someone else where that face appears in the background for the last month. Now you're not just finding the subject of a voluntarily portrait in security camera footage from a public place, but finding anywhere that any private person has gone in any populated area where anyone else is taking photos for social media.

How many involuntary social media photo and video backgrounds do you think someone living in NYC or SF is identifiable in every day? I would venture to guess, enough to track a lot of their life. The only thing I see stopping anyone from making this site today is the challenges in scraping large quantities of public data from social media sites. Once you have the data, the rest seems like a solved problem.

MauranKilom• 4 years ago

They certainly seem to have a hand for... uncanny projects.

Like a clock that shows you what percentage of your life is over.

thih9• 4 years ago

You might enjoy "the green leopard plague" [1], it's a sci fi novella and (mild spoilers ahead, pause reading if you want to avoid them) it describes a future where image processing like this is a bit more common and accessible.

[1] https://en.wikipedia.org/wiki/The_Green_Leopard_Plague

spoonjim• 4 years ago

Can someone explain to a novice how “AI” is used here? Is the person in the IG image being correlated to a security camera time stamp with face recognition or similar?

status200• 4 years ago

It is using computer vision to compare the instagram photo with the recorded 'open camera' footage to give the best guess at a timestamp. You can see the CV drawing boxes around the objects it is identifying in the footage on the right side, and it already knows that the photo was taken at the location (since the location tag was used that to scrape the photos), so it is basically just hunting for a person that best matches the subject in the instagram photo.

bmicraft• 4 years ago

The instagram photos are most likely not posted exactly when they were shot but some time after. Also, even if they post a rough location there is no guarantee that they would show up on any particular cam. I'd image the ai has to go through a very large amount pictures and cam footage to find one where the influencer shows up on a cam some time before it was posted.

secretsatan2• 4 years ago

The metadata of the image would reveal the actual time taken over when it was posted

willxinc• 4 years ago

The concept and visualization reminds me a lot of similar technology in the TV show Person of Interest. https://en.m.wikipedia.org/wiki/Person_of_Interest_(TV_serie...

It's quite interesting to think about how relevant the themes being discussed in that show are ever present today.

robk• 4 years ago

Underrated show and absolutely prescient!

cmeacham98• 4 years ago

It was a cool show, but a lot of the domains I am familiar with (hacking/cybersecurity, programming, compression) were very much made up for the storyline, so I suspect they didn't have much accuracy in domains I don't know much about (superintelligent AI, government surveillance, etc).

HereIGoAgain• 4 years ago

Lovely... in a sinister sort of way.

antegamisou• 4 years ago

Some buzzword clean-up

  "AI" --> OpenCV --> Histogram of oriented gradients (algorithm for pedestrian detection)

https://en.wikipedia.org/wiki/Histogram_of_oriented_gradient...

marktolson• 4 years ago

I highly doubt AI is being used here to match up the Open Camera footage with the instagram pic. Identifing humans as an object within a frame is trivial. Identifying individuals requires a considerable amount of custom training which is more than what is available from an instagram pic and a few video frames. I'm not saying it can't be done, it's just not that easy. I'd bet $100 bucks that most, if not all, of these matches were accomplished using good old stalking (matching post times to manually reviewing video data). Would be happily proven wrong but without any source code provided I stand firm with my opinion.

bastawhiz• 4 years ago

I suspect it's likely that there's a fair amount of effort involved, but I don't think you'd have to work too hard to substantially narrow your search with machine learning. Even just a small classifier that outputs clothing color and timestamps would make a remarkable difference. Identifying humans (which you can see in the videos) and checking how long they linger for could inform potential candidates for folks getting their photo taken.

leobg2• 4 years ago

Have a look at pimeyes.com. A single picture is enough, often even with sunglasses. Though that site requires input photos of slightly better quality than what you would be able to get from surveillance webcams. Then again, try it often enough, and use AI enhancement/upscale, and perhaps you can just use the site to replicate the project.

melony2• 4 years ago

Combine this with Grassland for a crowded mass surveillance system

https://news.ycombinator.com/item?id=19529921

melony2• 4 years ago

*crowd-sourced

mikotodomo2• 4 years ago

This is horrifying. Insta should require government ID to view so if you stalk / kill someone the police can quickly narrow down the set of possible perpetrators and find you.

python273• 4 years ago

OMG that's the most terrifying thing I have seen in ages.

Too late.

NTechLab scraped vk.com (Facebook-like Russian social network) profile photos and were running a public website "FindFace" that allowed to find profiles by photo of a face. They provide services to Russian government, e.g. it's used to find people from protests. Also FindFace was used to find and harass webcam models.

https://www.businessinsider.com/intel-spacex-philip-morris-l...

Intel, SpaceX, Philip Morris, and dozens of other US companies were in a leaked database of users for a Russian facial recognition company
The NTech Lab user list, which was shared with Insider by an anonymous source, includes more than 1,100 entries, with businesses and government agencies from more than 60 countries.

https://en.wikipedia.org/wiki/FindFace

In 2017, NtechLab face recognition algorithms were built into the Moscow city video surveillance system operated by the Moscow Department of Information Technology . The system uses the database of the Ministry of Internal Affairs to find correspondences to it on video. The alleged use of the system is the search for criminals and the fight against terrorism.
In 2022, NTechLab was accused of assisting the Russian and Belarus government by tracking thousands of political activists which led to their unconstitutional detentions and arrests.
In 2016, FindFace generated controversy because it was once used to deanonymize Russian pornography actresses and alleged prostitutes. These efforts were organized by users of a Russian imageboard Dvach who claimed that women in the sex industry are “corrupt and deceptive”, according to Global Voices. In addition, FindFace has been characterized as a major step in the erosion of anonymity.

personjerry• 4 years ago

Yeah Facebook was doing facerec for a WHILE with a LOT more data:

https://about.fb.com/news/2021/11/update-on-use-of-face-reco...

That's different than LOCATING random people through their photos though. This post is directly Mission Impossible kinda spy stuff.

mclightning• 4 years ago

Not gonna lie, I thought of this becoming possible with advent of AI & constant stream of social media posting.

You could also the reverse and find social media posts from the CCTV footage, to capture some passerby's face at a higher resolution.

We are living in dystopian times.

The Follower: Using open cameras and AI to find how an Instagram photo is taken

Related Stories