Brim: Open-source desktop app to analyze large pcaps through the lens of Zeek

https://github.com/brimsec/brim

by siskojr • 6 years ago

104 16 6 years ago

github.com

WrtCdEvrydy• 6 years ago

This looks pretty interesting, I wonder how it would do with the CTF dump from Defcon 19 and 20.

ghostpepper• 6 years ago

Can anyone explain how this is better than analyzing wireshark captures within wireshark?

What is zeek and can I use it to analyze wireshark captures?

tptacek2• 6 years ago

Wireshark dissects protocols and gives debugging traces. Zeek (FKA "Bro") is an intrusion detection system that analyzes debugging traces (which it generates itself, not through Wireshark dissectors) to generate usefully semantic data, ranging from "which hosts are fetching this URL" to "where are there signs of this vulnerability being exploited".

Both Wireshark and Zeek use libpcap, which is the de facto standard for raw packet captures, so yes, the same data you'd be feeding to Wireshark to browse, you can also feed to Zeek to analyze.

e12e• 6 years ago

Hadn't heard of zeek before - actually find this submarine article useful - it hints better at where and how, than the official zeek documentation. I still don't know what storage space I should budget for a gbps-month of zeek monitoring logs, though (I understand they're typically filtered... But also text?).

https://www.csoonline.com/article/3313050/zeek-a-free-powerf...

As to the where - I guess I need to find some port that can give me a mirror of traffic.... But not that easy with heterogeneous lan+cloud+SaaS setup to figure out where/how to source "all" network traffic :/

geraldcombs• 6 years ago

It might be more useful to think of Brim in terms of "a different zoom level" than "better than". Wireshark focuses on showing as much detail as it can about every packet that you see. If you have a lot of packets this will burn a corresponding amount of CPU time and memory. Brim, Packet Analyzer (mentioned in another comment), and other tools zoom out a bit and let you process a larger number of packets, but with less detail. If needed you can pass the interesting and relevant ones to Wireshark for more detailed inspection and analysis.

Disclosures: Wireshark pays for my mortgage. My employer makes Packet Analyzer. I'm friends with people at Brim.

Macphisto• 6 years ago

The long and short of it is Zeek(used to be call Bro) provides session based analysis whereas wireshark is primarily packet based. With a few exceptions, they both generally do the same thing but in different ways. Zeek is at the bones, an event-based engine with a scripting language (think node-js but from the 80s!) - the scripting language is actually what provides the logs generated by Zeek.

One key aspect of Zeek is that it can be deployed within a network to passively generate logs. As an incident response consultant, the few times I've worked with a client with Zeek logs, our ability to answer some critical questions in short order was increased dramatically! Back in my Sysadmin days, I used to run Zeek (when it was called Bro) to provide network logs for security review but also for general network analysis.

You can definitely run a pcap capture by wireshark through Zeek. you'd run `zeek -r <yourpcap>` and you'll end up with some lovely TSV separated logs in your current working directory!

Full transparency: I'm not part of the Zeek team, but I did author the original Zeek scripting guide for them back in 2012 or so. For my money, the Zeek team is building some of the better network appliances and detection/logging capabilities available.

LoveKebab• 6 years ago

Zeek is actually super powerful.

It’s more of a Network Security Monitoring tool (NSM) than a Wireshark equivalent

It’s primary use case is to have a server(s) ingesting all traffic in a network and forwarding the traffic metadata to something like an Elastic Stack.

What makes it so powerful though, is that it actually has its own scripting language that allows you create automations based off what type of traffic is observed and these can be super intricate

tannerbrockwell3• 6 years ago

When you get to the level of analyzing packet data, the first obstacle is actually capturing the conversations that you may find relevant. While a capture filter can help on the acquisition, it is possible that an error in the filter could exclude relevant hosts, or ports, or protocols. In general wide open captures will then lead to very large capture files.

Wireshark performs poorly with opening such a large trace. Since Engineer's are costing well north of $150 per hour, having them twiddle their thumbs waiting is not economical or efficient.

Enabling a survey and filtering of the data as soon as possible from the pcap file will allow interactive examination. With Brim that identified data can be exported as a filtered, and thus much smaller file. This is similar to what tools such as Riverbed's Packet Analyzer do. PA is a .net front end to tshark type filtering and reporting. It ships with a number of useful filters to survey for common problems.[1]

Brim and Zeek appear to fill the need between Packet Analyzer on one hand and a regular Wireshark workflow on the other.

[1] https://www.riverbed.com/products/steelcentral/steelcentral-...

dang• 6 years ago

We changed the URL from https://github.com/brimsec/brim to the home page it points to.

Edit: then we changed it back because the Github page seems to have a shorter time-to-understanding. Preferences?

Etheryte• 6 years ago

Definitely prefer the Github link, the short intro video gives you a very quick understanding of a good part of the value proposition, what this tool does in general, etc.

arminiusreturns• 6 years ago

I've been meaning to dig into bpf as a packet tracing/sniffing tool for the same reasons... pcap files can get enormous quickly enough to make space and parsing a pita. Cool work.

tannerbrockwell3• 6 years ago

Agreed. using bpf can eliminate some of the requirements for packet captures. Computing the metrics on the host, also eliminates the network traffic of exporting logs or pcaps to for instance splunk. [1]

Brendan Gregg wrote a book that digs really deep into the bpf system. [2] I recommend you check it out if you need to do real work fast.

[1] https://docs.splunk.com/Documentation/StreamApp/7.2.0/Deploy... [2]: http://www.brendangregg.com/bpf-performance-tools-book.html

tptacek2• 6 years ago

BPF for packet capture predates BPF for system instrumentation, and libpcap itself is under the hood an elaborate, optimizing BPF compiler.

GekkePrutser• 6 years ago

Nice, I like this!!

It should be standard in Kali :)

rixed• 6 years ago

Since everyone who are into pcap analyses are looking, I cannot help but drop references to two other open source DPI products that are worth considering for some use cases.

The first one I authored myself many years ago and has been dormant for a long time although I'd certainly enjoy working on it again if an opportunity arises: https://github.com/rixed/junkie

The other one seems to stand in the same niche of highly specific, optimised programmable DPI, but I'm obviously much less familiar with it: https://github.com/DanieleDeSensi/peafowl

Brim: Open-source desktop app to analyze large pcaps through the lens of Zeek

Related Stories