Running a fake power plant on the internet for a month

https://grimminck.medium.com/running-a-fake-power-plant-on-the-internet-for-a-month-4a624f685aaa

by tutfbhuf • 6 years ago

430 102 6 years ago

grimminck.medium.com

papaf• 6 years ago

It seems obvious not to connect a power plant to the internet but there have also been of attacks against "air gapped" systems: https://www.reuters.com/article/cybersecurity-powerplants-id...

Edit. There is also a more in depth Scientific American article. Search for "Hacking the Lights Out".

cmrx64• 6 years ago

i learned to my displeasure that thick manuals are sometimes distributed with products as USB sticks these days, i immediately thought of this when i opened up an inverter box and saw a USB stick sitting there

implements• 6 years ago

Some home broadband routers have a NAS-like ability to mount USB sticks as a SAMBA share - I use a spare one for iffy sticks on the assumption they’re unlikely to come ready to compromise some random non-PC embedded OS.

krisoft2• 6 years ago

I understand that not sticking USB sticks into sensitive systems is the prudent conservative security choice.

The “silly users picking up USB sticks dropped in the parking lot” is a basically a security trope nowadays. But I feel there should be some blame associated with our operating systems too. Like why is this an axiom that if you use an untrusted USB stick you are going to get eaten by the Grue?

If an Os would say “sorry bad people got into your network, your computer is now owned by them” that would be an unacceptable security vulnerability, why is the equivalent accepted as a fact of life with “bad usb sticks”?

I understand the OS cant do much with a usb device which burns out the motherboard with an electric shock. But there is a whole set of other things it should reasonably protect itself from.

netflixandkill• 6 years ago

Almost every power plant is effectively internet connected even if it has old control equipment that predates IP as the vast majority of SCADA systems have IP connected HMIs or other core components.

There may be steps involved in getting your RDP exploit to send commands over vendor proprietary RS-485 protocols, but except for certain nuclear plants that are truly air gapped, but it's fewer than you'd sleep soundly knowing about.

I once had a network admin at a major US transmission utility tell me with a straight face that all of their SCADA was pure serial as I was telnetting into the Zhone mux doing those serial channels via a WiFi connection.

akg_67• 6 years ago

In 1990s, When I worked with process control systems, primarily DCS, for petrochemicals, beverages, and other chemical plants, we had phone modems connected to our systems. Only precaution was that modems were not connected to phone port unless someone needed remote access and were disconnected after use.

Actually, we used to computer simulate operations of facility to test our DCS systems against.

weaksauce• 6 years ago

yeah, having actually worked with plcs in industrial control systems... the security is, ahem, lacking for the most part. not that my work was with high security process control but I'd say it's fundamentally lacking for the most part as the state of the art is not that great.

wyager• 6 years ago

I was once talking to an industrial automation engineer at a huge and strategically relevant US industrial group (they made aluminum parts or something, and their direct customers included aircraft manufacturers and military) and he was proudly telling me how they patched raspberry pis into their industrial control systems so they could administer things remotely. It wasn’t super confidence inspiring.

merlinscholz3• 6 years ago

The book Sandworm by Andy Greenberg also goes a little into depth about attacking powerplants and other industrial control systems. Can highly recommend!

hutzlibu3• 6 years ago

"attacking powerplants and other industrial control systems. Can highly recommend!"

Sounds fun, but please not in my neighbourhood ..

fbi-director• 6 years ago

Seconded!

secfirstmd3• 6 years ago

Yep it's great. Countdown to Zeroday by Kim Setter about Stuxnet is awesome also.

hutzlibu3• 6 years ago

Ok, but the attack on the air gapped system was simply via a infected USB stick.

There are more interesting ways, to do it:

https://hackaday.com/2017/02/02/hacking-the-aether/

zwog• 6 years ago

Stuxnet[0] for example. A highly sophisticated attack using several Windows 0-day exploits and infecting USB Flash drives to get to a air gapped Notebook that is used to program PLCs.

0: https://en.wikipedia.org/wiki/Stuxnet

deepstack2• 6 years ago

Digital system are just not mature and harden enough for security. Seems like we need to harden them on the software and hardware level before they can be trusted for driving car and other machinery stuff.

ktpsns• 6 years ago

Scanning the whole IPv4 Internet goes pretty fast (I think you can do it on a private subscriber line in a few days). What about IPv6? If I "hide" my service in my public /64 network, can I feel safe against IP scanning? That would be some security by obscurity based on the huge address space (similar to changing default TCP ports)

vbezhenar• 6 years ago

I think that you'll be safe for outside scans. But IPv6 devices usually talk via ICMPv6 inside their subnet, so if there are other guys on your subnet, they might find out your address. That was the case in some VPS hosters that I used, with tcpdump I was able to see addresses of other nodes.

nisa• 6 years ago

while at uni a colleague discovered zmap and scanned 0/0 from the 1gps line for the port of the intel amt management a day after a remote code exec bug was found there. he was just curious but watching the fallout and the angry forwarded mails these days was quite fun...

be careful :)

londons_explore• 6 years ago

Any time you connect to a website, you reveal your IPv6 address. I can bet there is a market for making lists of valid IPv6 addresses from web logs and selling those lists to people who want to port scan those hosts.

You might even find big carriers mine IP packets to find IP addresses they can sell.

e2le• 6 years ago

Given that a fair few devices will fill-in the "interface" bits of the IPv6 address with their MAC (SLAAC), it might be possible to reduce the number of addresses to scan in an IPv6 /64 prefix assuming you know what devices are likely to be used on that network.

It's also very likely your device will be syncing it's clock with an NTP server such as (pool.ntp.org) which can be scraped by running your own stratum 2/3 server and adding yourself to the pool.

https://www.ntppool.org/en/

binarysneaker• 6 years ago

Would be interested in a follow up article, especially if it baited more activity. The conclusion is woefully short, and I could speculate on some of the results, particularly the activity from the .tw domain. More investigation welcome.

Stierlitz• 6 years ago

"One of the concerns .. was the lack of insight into malicious digital (state sponsored) activity towards vital infrastructure."

Have these people ever given consideration to not connecting their vital infrastructure to the Internet. Instead using VPNs running on embedded hardware providing a .. virtual private network.

gorgoiler• 6 years ago

If you run a fake host (honeypot) to waste their resources, won’t they run a fake attack to waste your resources? Unless it is a substantial asset, all a honeypot will do is detect a honeypot detector?

When the attack comes from dhcp-XX-XX-XX-XX.rotation5.pool7.isptelecoms.co.abc, you can now determine to block all further attacks from that IP address, but to what positive effect? The next probe will come from somewhere else and just skip over your detector?

meibo• 6 years ago

The point of honeypots is not to block malicious IPs, but to become aware of places in your security concept where more hardening is needed, be it exploits, misconfigured firewalls, etc.

That can be a lot of things, blocking IP ranges can be one of those things if you e.g. only want to allow access to your assets from your building, but that's a general step and not reactionary to attacks.

waihtis11• 6 years ago

I run a honeypot software company, with our customers being mid-large enterprises. While the case presented here is certainly interesting, it's actually fairly uncommon for companies to run internet-facing honeypots, mostly due to it being a huge resource sink, and it's fairly difficult to learn anything truely useful from observing attacks coming from (mostly) bots.

The more interesting use case for most is planting these in your network internals, which gives an added benefit of early, high-fidelity threat detection in addition to the "threat intelligence" bonuses. It's not completely trivial to set up, but can be a reasonably quick way to build good detection capability into even very disparate environments.

A vast majority of organizations still lack good situational awareness of their infrastructures and this is one way of improving on that.

wwv25• 6 years ago

Did you plant internally to detect malicious actors from within the organization or as a way to definitively detect external actors who have presumably entered the network through an exploit?

faeyanpiraat• 6 years ago

Why not both?

waihtis11• 6 years ago

Both are viable. It depends a bit on industry; pharma companies, say, are quite concerned about internal threats whereas financial and retail are more likely targets of external actors.

You do need to consider the type of honeypot used - asking the question "what is the goal the adversary has" is a good question to ask and optimizing you honeypots based on that is a smart thing. An internal threat is going to look for specific types of assets, and you need to build honeypots (or decoys, as the modern lingo calls them) that look like those assets.

SCHiM3• 6 years ago

I'm wondering, given how threat actors normally behave when intruding into networks, do you also have honey user credentials in the network?

I don't imagine you're catching many serious attackers just by exposing fake servers vulnerable to MS17 for example.

Could you explain more about your strategies and which types of attackers you can say you'd catch with surety?

merlinscholz3• 6 years ago

Internal honeypots as early warning system are called Canaries. Usually they run on a just slightly outdated and vulnerable software version, so that an attacker does not need credentials.

One example would be emulating a forgotten file server with SMB1 enabled. The homepage of Thinkst Canary https://canary.tools/ gives a good overview.

waihtis11• 6 years ago

I'm wondering, given how threat actors normally behave when intruding into networks, do you also have honey user credentials in the network?

Yes, for some types of honeypots credentials can be useful. Although it bears to keep in mind that the more accessible you make your honeypots, credentials, and other "detection elements", to more you open yourself to false alarms. Some vendors in our space plant credentials to every endpoint and I've heard mostly negative results (legitimate users clicking stuff out of curiousity, which completely destroys the key benefit of using honey/deception things.)

I don't imagine you're catching many serious attackers just by exposing fake servers vulnerable to MS17 for example.Could you explain more about your strategies and which types of attackers you can say you'd catch with surety?

One use case we suggest is tying your honeypot strategy with your existing internal vulnerability & threat management, in practice for example circulating the types of honeypots you have to reflect acute, high-value vulnerabilities - or for slightly more advanced orgs who are on top of their threat intelligence, they could actively plant honeypots that match asset types threat actors in their industry space have been known to target.

Tie that with creds, integrations and such where applicable and you have a pretty decent palette for catching events that also bear useful and timely security telemetry.

justinclift2• 6 years ago

Hopefully one of your honeypots simulates NetBackup. eg the Veritas (was Symantec) software

That should be reasonably easy to simulate, and (I'm guessing) Netbackup infrastructure would be significantly interesting to any hacker once they've popped an org.

waihtis11• 6 years ago

To be frank I haven't heard of such scenarios from any direct sources. If you're thinking ransomware here then it's usually fairly automated and takes a shotgun approach for propagation, and trying to extend your breach to also manipulate any potential backup software would significantly increase the cost of the attack.

That being said - I could easily see this as a future trend (targeting backups) and it is not remotely a bad idea.

TazeTSchnitzel• 6 years ago

A fun example of a honeypot was that placed by Cliff Stoll in 1986 on a computer system at Lawrence Berkeley when trying to get a repeated attacker to stay on the line long enough to be traced, as described in The Cuckoo's Egg and portrayed in this PBS NOVA recreation: https://www.youtube.com/watch?v=hTx9h3Sm29I

e12e2• 6 years ago

Wait, "stay on the line long enough" was an actual thing? I guess that means shorter calls were free, as they could not be documented/invoiced?

davidyliuliu• 6 years ago

hahahah the intro of the video is hilarious

voicedYoda• 6 years ago

I read this book last year. What a fun read! Well thought out, clever solutions, and a very focused amount of tenacity.

hcrisp• 6 years ago

There was a recent WSJ article I submitted recently that details how companies such as Land O' Lakes are doing just that -- honeypots on internal networks.

https://news.ycombinator.com/item?id=25741589

waihtis11• 6 years ago

Seems there's a bit of love going around for deception lately, always good to see some implementation stories going mainstream.

GordonS• 6 years ago

Nice, your software looks pretty polished! Have you published any kind of best practices guides for using decoys, or alternatively are you aware of any other good guides?

I've seen decoys mentioned in infosec Twitter quite a few times recently (I think more about OSS ones), and I'd like to learn more about how they are generally actually used, what the expectations are etc.

waihtis11• 6 years ago

We haven't so far..

Mitre SHIELD (shield.mitre.org) lays out the different types of capabilities there are in this domain but doesn't have a lot of practical how-to advice which is what I think you're more after.

If you think it would be helpful I can send you our "product guide" which touches a bit on the decoy best practices. If this sounds useful message me at simo@[the domain in my profile] and I'll send it over.

I might actually write something about it online also, sounds like it could be useful knowledge for many.

varispeed• 6 years ago

Do all employees know about honeypots or just a select few?

waihtis11• 6 years ago

Just a select few usually, from within the security team. of course this applies for boilerplate big company deployment.

I could see use cases where you would involve others like IT or dev in the deception process..

AdrianB14• 6 years ago

At work I have access to servers controlling dozens of manufacturing plants around the world. All the PLCs and the control equipment are behind a local firewall with very strict controlled inbound addresses; that means I cannot interact with it from my computer, only from the server that is collecting and archiving data. This setup is approximately the standard in the industry, there are a small number of solutions everyone is using and the implementation guidelines are very clear and easy to follow. That makes any honeypot an obvious honeypot because there is no such thing as a PLC accessible from Internet in a real production site.

mercora2• 6 years ago

that will just move the target of those attacks to these servers (or now yourself who stated to have access to these) with apparently a lot more rewarding outcomes too.

AdrianB14• 6 years ago

Correct, but the servers are not published to Internet either, there is no reason to ever do that. Cracking the VPN to get to the company intranet, that is a different story.

rm445• 6 years ago

there is no such thing as a PLC accessible from Internet in a real production site.

That is so massively optimistic. I don't doubt you know your stuff, but manufacturing is a huge field, widely distributed, it is done by small companies as well as large ones, and specifying and purchasing a PLC system can be done to satisfy operational needs without necessarily having suitable network infrastructure and security expertise. The number of PLCs "accessible from Internet in a real production site" is probably in the thousands.

AdrianB14• 6 years ago

Smaller companies take shortcuts, that is true, but not a nuclear power plant like the bait used in the article. Smaller companies also do less damage when they are hacked.

acct776• 6 years ago

Would you post an email in your profile? I'd love to get in touch!

AdrianB14• 6 years ago

Sorry, no (I just put myself a virtual target on the forehead). If you have a specific question I can try to answer, but that is it.

secfirstmd3• 6 years ago

For anyone looking to experiment with honeypots TPOT is awesome. They've really done a great job of building a simple, well presented and fairly stable tool that pulls together a lot of honeypots into the one place. A sophisticated attacker can usually detect it but it's very good and easy to setup all the same. You really need to run it on 16gb RAM if you are using the full thing.

It's very interesting the results you see depending on where you put it (internal/external etc). Pretty quickly you get a decent sense of the pulse of the internet - XYZ is spreading, ABC range is compromised etc. Though you also get heaps of data so you need to find ways to really drill down also.

https://github.com/telekom-security/tpotce

ourlordcaffeine3• 6 years ago

Several power plants in the uk have their control systems entirely open on the net. No authentication or encryption, you just need to know the IP address

eli• 6 years ago

I don’t believe it

berkes• 6 years ago

Having worked at a power-plant and a pumping station, I can believe it.

Such places will be left open now and again. Mistakes happen. Anything from "ABB needs the telemetry but cannot visit the site in person due to covid, can you open the firewall on port 1337 when they call you". Sure. (and then forgets). Some engineer left a dongle in a controller, during an emergency a laptop logs on on the network that is not properly secured etc.

What I don't believe is that OP never reported it. Because such places will have protocols and will fix it the moment someone calls. And if they don't have protocols, they will have them by the end of the week. Edit: so what I also don't believe is that OP is certain they are still wide open.

hyper_reality• 6 years ago

How do you know these aren't honeypots?

ourlordcaffeine3• 6 years ago

I work in the industry. I know what power plants they are

mercora2• 6 years ago

why wont you report them? is nobody interested in securing these?

some_random• 6 years ago

This is interesting, but like most honeypot discussion kinda melodramatic. They got scanned by a bunch of known internet scanners, and a few mysteries including a "store that sells weaponry" (which turns out to just be scopes and red dots -_-). It's always good to people reporting that they discovered nothing, but it is a bit boring.

butwhywhyoh• 6 years ago

This is likely just a way to subtly show off the authors talents. But I completely agree. Absolutely nothing of interest.

stefan_• 6 years ago

So all the traffic is various indexing companies and none of them made an effort to contact the author? What if, you know, the guys pretending to be good are actually just baddies selling out?

jakespracher• 6 years ago

FWIW I interned at a US nuclear plant. They intentionally don't upgrade to digital systems for fear of being hacked, everything in the plant was analog when I was there 6 years ago.

beckingz• 6 years ago

Getting replacements for some of those older parts is hard!

deepstack2• 6 years ago

Good for them! These vary from simple “smart” light switches, to machinery used in industrial plants

This line makes me cringe. Current IT infrastructure are just NOT secure. Until the major IT tech companies and nation state can prove otherwise, important machineries, especially nuclear ought to be kept off internet and digital system.

collsni• 6 years ago

Yes because a nuclear power plant would have a plc Nat'd to the internet. Multi-million dollar budget, we'll just NAT the plc so joe can work from home.

lambda_obrien• 6 years ago

There are plenty of real power plants connected to the internet today in the form of virtual aggregations of large batteries. In a decade there will be gigawatts of them online, so let's hope those companies take security seriously because it'd be simple to break things with the instant availability of several gigawatts on some circuit.

SMAAART• 6 years ago

I feel like I have worked for fake companies in the past.

throwawaysea• 6 years ago

How do hackers get access to these devices and learn about how they're deployed in order to be able to exploit them? Can someone ELI5 this?

Moody_10001• 6 years ago

Instead of using reverse DNS to see where traffic is from (e.g. the optisan.com.tw fqdn) you might be better to do a whois lookup on the source IP addresses. Reverse DNS can point anywhere, but you can't make IP addresses in TCP connections.

stjohnswarts• 6 years ago

This seems like a really really obvious honeypot. I mean a nuclear power plant with an internet facing PLC??? I mean probably even haX0r bois aren't that stupid? Maybe they are it didn't seem like he got any real hits from a malevolent hacker.

Running a fake power plant on the internet for a month

Related Stories