Socat – A utility for data transfer between two addresses (2018)

https://copyconstruct.medium.com/socat-29453e9fc8a6

by brigandish • 5 years ago

230 65 5 years ago

copyconstruct.medium.com

vesinisa4• 5 years ago

This tool is like netcat on steroids. Definitely part of my stable toolbelt.

rwmj2• 5 years ago

Whenever someone mentions netcat or nc I always ask "which one?". There are at last count 3 different implementations, all subtly incompatible. Which is another reason why socat is better since there's only one to deal with.

cogburnd022• 5 years ago

How do you refer to each implementation & what are some of the subtle incompatibilities?

tyingq• 5 years ago

Indeed. There are examples for making local/remote shells. It can make a pty, use setsid() to be a daemon, reset the terminal state, then listen(), etc, all in a one liner. Or route a serial port over ip via ptys in raw mode, again, with a one-liner.

Datagenerator• 5 years ago

Shameless plug [1], socat combined with WireGuard

[1] https://github.com/lrvl/tunnel-wireguard-udp2tcp

maddyboo• 5 years ago

Excellent article!

I’ve tried to grok socat a few times but gave up quickly upon being faced with its screenfuls of dense help text.

This post sheds light on the core concepts behind the tool in a way that the man page and help text just didn’t do for me.

Now that I ‘get’ it, I’m actually excited to find an opportunity to use socat! I can think of so many interesting use cases.

je42• 5 years ago

very versatile - socat PTY,Link is awesome.

prussian• 5 years ago

very underrated. I've used this trick before to mock serial ports on some Linux-running industrial devices so I could debug them.

inivekin• 5 years ago

I made a small utility that'll replay socat log files captured by the -lf -x -v flags with the timing it collects if used in this way. It is very simple and is basically only there if you can't be bothered to roll your own: https://gitlab.com/inivekin/socatplayer/-/releases

So if you're commonly doing a socat process like: socat -x OPEN:/dev/ttyS0,b115200,echo=0,icanon=0 PTY,link=/dev/ttyO0,rawer 2> socat_log.log

then it may be of interest to you.

oofnik• 5 years ago

I've used socat in the past to debug all kinds of odd network behavior. It helped me discover a networking issue with some code by comparing performance of piping a Unix socket over TCP using socat vs. the native TCP implementation, which turned out to be caused by an overly conservative default send buffer limit.

tssva4• 5 years ago

I use socat along with npiperelay to allow the ssh client in my WSL sessions to access the Windows ssh-agent.

strictfp• 5 years ago

Interesting! Do you mind elaborating a bit? I ssh into WSL for this reason.

nullify88• 5 years ago

Likely something like this https://github.com/rupor-github/wsl-ssh-agent#wsl-2-compatib...

Similarly, you can also do this with KeepassXC

dataflow• 5 years ago

I haven't tried on WSL2, but on WSL1 this script also lets you integrate with KeePass's KeeAgent:

https://gist.github.com/duebbert/4298b5f4eb7cc064b09e9d865dd...

shawnz3• 5 years ago

Just to throw another one of these bridge applications into the mix, here is one that allows Pageant apps to communicate with Windows' native ssh-agent.

https://github.com/ndbeals/winssh-pageant

I use both this and rupor's wsl-ssh-agent from the other thread extensively.

jonenst• 5 years ago

We use socat to buffer logs sent from containers (docker-compose) during the short window at startup where logstash and elasticsearch are not ready yet. Works great!

nwjsmith• 5 years ago

Have you happened to blog about this? I'd love to be able to fix our setups with this trick!

mrspuratic• 5 years ago

Similar: using it to get syslog out of self-chrooting ssh with minimal trauma: socat -u UNIX-RECV:/home/sftponly/user/dev/log,mode=666 UNIX-SENDTO:/dev/log

It's also an CLI IMAPS client, with history: socat READLINE,history=$HOME/.imaps_history EXEC:'"openssl s_client -connect mailserver:993"'

or a CLI web browser with history: socat -d -d READLINE,history=$HOME/.http_history TCP4:www.domain.com:www,crnl

or, if you have to interact with something that has no readline, say sendmail: socat READLINE EXEC:"sendmail -bt"

sedachv• 5 years ago

if you have to interact with something that has no readline

lukax• 5 years ago

Socat is the default implementation for Kubernetes port-forwarding [1] when using dockershim for the kubelet runtime. Similar to how tar is used as the default Kubernetes cp command implementation.

[1] https://github.com/kubernetes/kubernetes/blob/770d3f181c5d7e...

Bluecobra• 5 years ago

Great article! I use socat all the time to test multicast subscriptions in Linux.

xrd• 5 years ago

Do you have any links/writeups to how you do this?

justin_oaks2• 5 years ago

I often use a socat docker container to proxy network traffic to docker containers. It effectively allows creating dynamic docker port mappings.

For example, I have a database with no docker port mappings (only a web application container in the same docker network accesses it directly). I can use a socat container with port mappings to proxy network traffic to the database.

barbazoo• 5 years ago

Interesting, what's the benefit over exposing the port through Docker?

throwaway888abc• 5 years ago

Security

https://en.wikipedia.org/wiki/DMZ_(computing)

justin_oaks2• 5 years ago

It is exposed through docker, just via a separate container. The reason for doing so is that a container can't have new port mappings added after it is created.

Using the socat container allows me to access a port on my original container that I wouldn't be able to access unless I recreated that container.

As for why the database didn't have port mappings originally, it is best not to allow network access to things unless it's necessary. It's the principle of least privilege.

fguerraz• 5 years ago

I use socat a lot, especially when wanting to transfer files without incurring the ssh encryption overhead or rsync daemon setup. Just tar | scocat and voilà, Superfast whole directory tree transfer. Sprinkle a bit of mbuffer or openssl on top and you can do a lot!

ldiracdelta• 5 years ago

I use these two socat commands for MITM all the time on my own stuff so I can see the traffic:

  socat  -v TCP-LISTEN:666,forever,reuseaddr,fork TCP:example.com:8080

  socat -v TCP-LISTEN:8080,forever,reuseaddr,fork ssl:example.com:443

jorl17• 5 years ago

socat is an invaluable tool. I first came about it at my first job, where we (just two clueless dudes) had to migrate the mostly undocumented infrastructure of about 15 machines at Digital Ocean to Linode, adding replication and failover on top of that (making it ~35 or so machines).

During that migration, we tried to minimize downtime, so we made sure that requests made to the old infrastructure were aptly forwarded to the new infrastructure wherever possible, until DNS properly propagated. I found out about socat and we went for it. It worked great, and I marveled at the simplicity of the solution!

Ever since then, I've used socat both as an emergency saviour and as an actual service. I seem to recall using it once to go around some docker bug or shenanigans.

bombcar• 5 years ago

This is a brilliant idea and I’ve never even considered doing that form of port forwarding.

I’ll have to remember it for the next time I migrate.

cowmix• 5 years ago

Finally socat is getting the luv it deserves.

prhn• 5 years ago

I use socat daily.

Nearly all of the microservices we develop have a way to be run locally with all dependent services using docker-compose.

The problem is most teams don't provide a way to do efficient local development for their particular service. Some do remote debugging into the built/running container (slow and requires rebuilding docker image for every change). Some inspect logs to debug (yikes!).

When I want to debug a particular service in IntelliJ, I simply remove it from the docker network and replace it with a container running socat, forwarding the appropriate port to host.docker.internal (the host machine). This is on a MAC, of course, where Docker runs in a VM.

This allows me to do in-IDE debugging of any service that runs in the docker network without having to rebuild the image every time I make a change.

I know there are IntelliJ plugins that provide a similar workflow, but I believe they all build an image and deploy a container to the network for every change. I could be wrong.

tomerbd• 5 years ago

Interesting .. Can you provide an example?

robbles• 5 years ago

How do you swap out the container in docker-compose?

Do you just have a service defined for each, and only start one of them?

e12e• 5 years ago

remove it from the docker network and replace it with a container running socat, forwarding the appropriate port to host.docker.internal (the host machine

Would you happen to have a fully worked example of this? We to have some services that we can and do run via docker compose - but it's often a cumbersome process to move one and one service out of the docker compose setup for debug/devel.

cperciva3• 5 years ago

A similar tool: spiped[0]. Connects two arbitrary TCP4, TCP6, or UNIX socket addresses, encrypting + authenticating data using a preshared key. For many applications it's basically socat + security.

[0] https://www.tarsnap.com/spiped.html

qbasic_forever2• 5 years ago

Have they done any audits or had critical analysis of their security code? These days I'm super hesitant to consider anything that hasn't had a lot of eyes on it. It's just too easy for a tiny screwup in protocol design/implementation, random number generation, etc. that puts everything at risk. Why risk it when I can just use wireguard.

boulos• 5 years ago

cperciva is the author of spiped and a security expert [0].

But your question about an audit is still a good one! Tarsnap has bug bounties, but the top hit for “spiped security audit” led to a sort of amusing flame war on HN [1].

[0] https://www.tarsnap.com/about.html

[1] https://news.ycombinator.com/item?id=7539903

cperciva3• 5 years ago

Lots of people have looked at the spiped source code; it has been around since 2011 and I've even handed out a few bug bounties.

It's also considerably simpler than wireguard, which reduces the potential for vulnerabilities; and operates on a per-connection basis, which means you can do things like establishing a secure connection to another host but have a socket endpoint which can only be accessed by certain users.

AnotherGoodName• 5 years ago

It talks serial too. I've used socat to send out data to the network from a set of serial connected devices for environmental modelling. Ended up being one of those jobs I got done from end to end in an afternoon with the socat part taking 10mins

megous• 5 years ago

Oh! I might as well use it for line editing on serial line with the readline local and serial remote addresses, for example when talking AT to a modem. Great! :)

And with history and all that stuff, too! I guess it can be used to forward serial port over network too.

rgovostes• 5 years ago

I'll join the chorus praising socat, but also share my gratitude for Gerhard, the developer / maintainer.

We exchanged emails back and forth about a use case I had, which was to split a serial device into two PTYs, allowing a process to control one while I monitor the other with picocom. We couldn't get it to work, so I gave up, but then Gerhard wrote back a few months later saying he added it as a feature.

It can be frustrating to figure out the right incantation for some advanced use cases, but this is more a complaint about esoteric corners of the operating system than socat which exposes the full complexity to you.

One of the invocations I recently worked out was to wrap an interactive process with a PTY, so that I could make a mock serial device that emulates a piece of hardware.

    socat PTY,link=/tmp/dev,rawer,wait-slave \
      EXEC:"./emulator",pty,setsid,ctty,echo=0

iJohnDoe• 5 years ago

Is it possible to use Socat (or any utility) to redirect a UDP connection on a local Windows machine to TCP port 443/80 to a remote system listening on 443/80 that will connect to the original intended UDP connection?

I can also do port redirects with iptables that can assist with solving the problem.

I need to host a microservice that uses UDP which won’t connect out on most corporate networks due to firewalls. I need to get the connection out via a standard TCP port.

Nothing malicious. It’s a basic corporate client/server app, but it’s bound to a UDP port that I can’t change.

cyberpunk3• 5 years ago

I think your best bet is to setup a simple WireGuard for this, also udp but put the server on 53 or something it’s unlikely to be blocked.

cyberpunk3• 5 years ago

Oh or skip that abd just use pen ( http://siag.nu/pen/ ) to get the traffic from 53->whatever port (can’t remember if haproxy supports udp)

axiolite3• 5 years ago

Socat is terrible.

When used to establish a persistent connection to a remote listening (TCP) port, it has NO WAY to detect that the remote end has dropped the connect, and automatically reconnect. It just does not work.

Yes, I know all about all the keep-alive and retry options in the manual. Try it yourself.

It's one of those horrible tools that you can get up and running quickly, but can never make work reliably.

bandie91• 5 years ago

i doubt that socat did not handle TCP RST. is not it the network environtment which eats TCP RST in some cases (like AWS NLB).

axiolite3• 5 years ago

That's why I said try it yourself... It's no good for serious use. Here's the first search result:

https://stackoverflow.com/questions/9341254/reconnect-socat-...

sundbry• 5 years ago

I don't think you understand how TCP works. A TCP connection can stay alive through an unplugged Ethernet cable if it hasn't sent any packets.

axiolite3• 5 years ago

I've worked as a senior network admin for some very large organizations. Safe to say I understand TCP.

Socat – A utility for data transfer between two addresses (2018)

Related Stories