AOL Moloch: open-source, large scale, packet-capturing, indexing database system

by bottle2 • 6 years ago

239 120 6 years ago

ignoramous3• 6 years ago

Not the greatest choice of name: Moloch[a] (also Molech, Mollok, Milcom, or Malcam) is the biblical name of a Canaanite god associated with child sacrifice, through fire or war.

From: https://en.wikipedia.org/wiki/Moloch

pnathan2• 6 years ago

That was my immediate thought as well.

Profoundly and deeply unsavory.

simlevesque4• 6 years ago

You are offended by the name of an open source project ?

Your life must be easy.

striking• 6 years ago

Perhaps they were going for an association with Allen Ginsburg's Howl (https://www.poetryfoundation.org/poems/49303/howl) instead?

lsb• 6 years ago

Reminds me of when someone saw that the NSA's public key (http://www.cypherspace.org/adam/hacks/lotus-nsa-key.html) to be included in Lotus Notes had an organizational name of "MiniTruth", and a common name of "Big Brother".

Moloch isn't the good guy in Howl.

Moloch! Solitude! Filth! Ugliness! Ashcans and unobtainable dollars! Children screaming under the stairways! Boys sobbing in armies! Old men weeping in the parks!

Moloch! Moloch! Nightmare of Moloch! Moloch the loveless! Mental Moloch! Moloch the heavy judger of men!

Moloch the incomprehensible prison! Moloch the crossbone soulless jailhouse and Congress of sorrows! Moloch whose buildings are judgment! Moloch the vast stone of war! Moloch the stunned governments!

hprotagonist2• 6 years ago

which is the same referent, and begs the same question: "What sphinx of cement and aluminum bashed open their skulls and ate up their brains and imagination?"

theprotocol• 6 years ago

Wasn't Allen Ginsburg also in favor of child abuse?... He was a member of NAMBLA...

At least Moloch isn't real.

pas• 6 years ago

https://slatestarcodex.com/2014/07/30/meditations-on-moloch/

_jal• 6 years ago

Kind of fitting for AOL though. I can name three startups that went to die there off the top of my head.

chews• 6 years ago

It's an inside joke to the Deep State that actively needs captures of network data, AOL... is a Verizon company after all. In these discordian times, even Eris would blush.

mensetmanusman• 6 years ago

Funny you mentioned the deep state. I remember being perturbed that the email leak from the 2016 race had references to child-sacrifice moloch. Strange times

scandox• 6 years ago

I think this is a bit distracting. It's an ancient name. Probably a better paragraph to fixate on would be:

Moloch has been used figuratively in English literature from John Milton's Paradise Lost (1667) to Allen Ginsberg's "Howl" (1955), to refer to a person or thing demanding or requiring a very costly sacrifice.

I mean a project called Zeus would hardly founder on it's namesakes fetish for turning into a bull and assaulting human women.

HideousKojima• 6 years ago

Zeus is known a bit better and for more things than Moloch

derptron• 6 years ago

Am I the only one who thinks of Moloch aka Edgar Jacobi, the old villain in Watchmen dying of cancer?

smoe2• 6 years ago

Interesting, newer knew the word is of biblical origin. In German the word Moloch is in somewhat common use to refer to an abstract merciless, all-consuming or all devouring power.

E.g big cities might get referred to as molochs the way they eat into nature around it or suck people into its anonymity

woodpanel• 6 years ago

I was thinking the same. In German Moloch is mostly used for groteskly big/growing cities. But thinking about it, the "eating into nature"-theme aligns well with the biblical "eating into kids"-theme.

For me at least, because of the city-context, Moloch matches: The moloch could stand for the amount of activity in a system that is humanly insurmountable to get a better-than-superficial understanding of.

finnthehuman• 6 years ago

Why not? That's a pretty metal name. Don't tell me anyone still has appetite for yet another saccharine cutesy-wutesy "yay software squee" name.

Hackbraten• 6 years ago

There’s always room for more cute in software.

runawaybottle• 6 years ago

How about AOL PackagieWackigie?

Thaxll• 6 years ago

Anyone who works on Linux probably knows that sometime you need to sacrifice a child...

bigiain2• 6 years ago

Some problems just can't be fixed using goats...

cls59• 6 years ago

One of the first things I grep for in dmesg ;)

gyehuda9• 6 years ago

Naming open source projects is a challenging task. The team hears this all the time, but hey, open source is about code you can use, a community you can join to make the code better, and the pride that people around the world want to use your code. If you are on a blue-team, you'll want to look at this project. If you are making lists of names that distract attention this goes on it.

ignoramous3• 6 years ago

If someone names their CLI JDahmer which then requires me to use it jdahmer <command> <options>, it is going to cringe me out.

That said, I totally respect the project's decision to name it so. I'm sorry if it comes off as an outrage or an unnecessary rant. I understand it takes away the energy which is otherwise better spent on a constructive discussion of this excellent project, which is also surprisingly well-maintained. But, I hope you realise why the name is controversial, especially since everyone that has to use the project has to get used to seeing/reading/using/typing Moloch everywhere from the docs, to the CLI, to the FAQ pages, and what-not.

May be the project can explain their rationale in a separate webpage (if they want to) and that might help?

xenihn• 6 years ago

Honestly the first thing that came to my mind was Watchmen.

travbrack• 6 years ago

At least it's not trafficly

phs318u• 6 years ago

https://slatestarcodex.com/2014/07/30/meditations-on-moloch/

Makes more sense.

totetsu• 6 years ago

Sheesh, I hope no-one looks up my user name on wikipedia..

yingw7872• 6 years ago

This is cool, thanks for sharing! I'll be honest, I didn't think AOL did anything...interesting...but this is interesting to me! What are some open-source alternatives / analogues to this product?

gyehuda9• 6 years ago

Sorta like https://www.wireshark.org/. But Moloch is a very active project, used by many, and used internally at Verizon Media. Aol is part of Verizon Media (which brought AOL and Yahoo together). Open source is very active here. ;-)

Compare also to https://enterprise.verizon.com/products/security/advanced-th... (formerly known as Protectwise), which is not open source.

solarkraft• 6 years ago

Ignorant question for you: What does AOL do these days (I can't view the website for the same reason I don't read TechCrunch articles)?

Is it fully about online publications now?

yingw7872• 6 years ago

Oh woah cool! I love the Verizon dashboard, looks very polished for an internal tool! I have a friend who did some network security work for Comcast, interesting stuff it is.

donio• 6 years ago

I'll be honest, I didn't think AOL did anything...interesting...

Well, there is also https://github.com/aol/ExtJS4-Fart

peterwwillis3• 6 years ago

AOL bought one of the first web server start-ups in 1995 and open-sourced the tech in 1999. It was pretty advanced for the time. http://philip.greenspun.com/wtr/aolserver/introduction-1.htm...

mindcrime• 6 years ago

Seems like there would be at least some overlap with Metron[1] and/or Spot[2].

Of the two, I'm more familiar with Metron (I actually did a small amount of work on it back before it was an Apache project). The core "thing" of Metron was always a large-scale, high-speed packet capture mechanism that would allow you to apply real-time streaming analytics / ML to packet streams, as well as supporting indexing the packets with ElasticSearch for post-hoc retrieval / analysis.

Spot seems to employ some similar ideas, but I haven't dug into it as deeply.

[1] https://metron.apache.org/

[2] https://spot.apache.org/

seemslegit• 6 years ago

This should work nicely with Google Behemoth and Facebook Beelzebub

burpsnard6• 6 years ago

awsanswers2• 6 years ago

AOL github account's location is Dulles, VA. Thinking of what this does, the name's meaning and its location in 'the corridor' is giving me the creeps

cfeduke• 6 years ago

AOL's location has been there for a very long time.

gyehuda9• 6 years ago

I fixed that for you https://github.com/aol

awsanswers2• 6 years ago

Yeah much less creepy

nerdbaggy• 6 years ago

It’s been there since before the Mapquest days.

dguido5• 6 years ago

Fun fact, Moloch was initially created with funding from DARPA's famous Cyber Fast Track program! It's great to see that Moloch is still going strong since ~2013.

There is some previous discussion of Moloch when it was released in this older thread: https://news.ycombinator.com/item?id=20586005

awick• 6 years ago

Not sure if you are joking, but Moloch was never part of DARPA's short lived Cyber Fast Track program. :) We do welcome contributions from everyone, and lots of different folks use Moloch. If interested join us over at https://molo.ch

dguido5• 6 years ago

That's odd. Are you one of the original authors? The CFT project list had Moloch on it. I'll try and dig it up, it's probably floating around my Google Drive. You may want to speak with Eoin Miller, as I believe he was the point of contact for the project in the document I'm thinking of.

whyhead• 6 years ago

Thanks, John. I tried to do something a while back with wireshark but I couldn't get the thing to scale. Gonna give this one a hard look.

dguido5• 6 years ago

I seem to be mistaken :-x. Moloch was never provided with early funding from CFT. I confused a few interactions I had with their project's original authors in 2013. It _feels_ like something CFT would have funded, but it was started on its own. See more here: https://news.ycombinator.com/item?id=22951925

closeparen2• 6 years ago

What can you do with a packet capture on a modern high security network besides go, "yup, that's TLS"?

May DoD turns off the forward secrecy stuff and escrows keys?

tptacek• 6 years ago

In large part you're not looking at TLS application-data with this stuff; you're monitoring internal networks and all the protocols they run, in part so you can retroactively see if exploits, once revealed, have been run. For that kind of stuff you often care a lot more about, say, SMB dissection than you do about what stupid websites people are looking at.

The longstanding existence of tools like these --- and there are "better" ones that aren't open source, and have been for decades --- is one reason that "vulnerability equities processes" don't make sense; if the DoD uses an exploit against a foreign target, it can't just reveal it a few months later without compromising sources and methods.

(That doesn't mean you should care about that problem; I'm just reporting).

dang2• 6 years ago

A thread from last year: https://news.ycombinator.com/item?id=20586005

dang2• 6 years ago

Url changed from https://github.com/aol/moloch, which points to this.

hprotagonist2• 6 years ago

This gets posted once in a while and I always, without fail, think of Ginsburg.

Moloch whose mind is pure machinery! Moloch whose blood is running money! Moloch whose fingers are ten armies! Moloch whose breast is a cannibal dynamo! Moloch whose ear is a smoking tomb!

caconym_• 6 years ago

Yeah, I literally heard that line in my head immediately after reading the title of this post.

What else are we supposed to think of?

burpsnard6• 6 years ago

The game

bigiain2• 6 years ago

And whenever anything here reminds me of Ginsberg, I always, without fail, remember:

“The best minds of my generation are thinking about how to make people click ads.” –Jeff Hammerbacher

mdonahoe• 6 years ago

For the uninitiated (like myself)

Allen Ginsberg's poem "Howl" starts with the line:

"I saw the best minds of my generation destroyed by madness"

gyehuda9• 6 years ago

According to my sources, DARPA wasn't involved in any of the child sacrifices, but they might use Moloch to help secure their networks.

dguido5• 6 years ago

Yep! As far as I'm aware, a number of military services (I think the USAF) have been public about using Moloch on their networks. It was money well spent by DARPA.

gyehuda9• 6 years ago

Moloch is free. :-)

kfrzcode7• 6 years ago

The amount of professional-grade hand-wringing virtue-signaling in this comment section makes me feel ill.

Why get bent out of shape over the name of a software project? It's virtually a meaningless factor in day-to-day life.

What about hearing phrases "Sacrificing a Chicken to Moloch," the "spirit cooking" culture and related symbolism rampant in elite political circles? Shouldn't we be more interested in that?

Lots of low-hanging fruit to pick, I guess. Someday I might unlock the "downvote" ability on this platform. Until then my opinions don't carry weight here. Also, uhh who decides the "threshold" for downvoting? Hint: nobody knows. [0]

This platform has become more of an echo chamber than a host of rational discussion based on merit. I suppose that's a problem with growth.

There's a lot of conversation around "Moloch" as a name and the subsequent emotional responses... but not a lot of discussion about the tech at-hand. And it's a repost.

Where's the value?

[0] https://news.ycombinator.com/newsfaq.html

Edit: while this is high-ranking comment, I'd point out that if I had the ability I would have just downvoted the comments I didn't like. Take that for whatever it's worth.

gyehuda9• 6 years ago

Preach Brother!

Moloch is a serious open source project, run by serious people who care about network security. They published their code under an open source license, showing off just how confident they are that it is solid. You can use this project to inspect packets on your network, you can learn how they built it and become a valuable network security engineer with a job somewhere finding people trying to hack in to your site. You can propose modifications to make this even better (and if your code is good enough, it will get accepted and used by security teams around the world). Or you could focus on the name they chose and the name of the company at the time this was published.

I'd seize the opportunity to talk tech and focus on network security. Infosec jobs pay better than brand marketing jobs.

kfrzcode7• 6 years ago

The fact we have access to tools like this (and many, many others) is so damn motivating. I am working through some structured course material in preparation for net-sec baseline `$vendor-$certification` exams, and there are so many complexities and rabbit holes that it's easy to be overwhelmed. Some nights are harder than others to see forest for the trees when it feels I'm trying to understand how to pick up a single pine cone to study.

Then I see projects like this, or Jaeger/OpenZipkin, or Chaos Monkey, and I'm simply inspired. Much like listening to a killer record.

I don't spend time feeling sorry for myself because an ancient, somewhat esoteric proper noun was used to describe the project.

ignoramous3• 6 years ago

Why get bent out of shape over the name of a software project? It's virtually a meaningless factor in day-to-day life.

A lot of people do care about what things (living or not) are named and even how, sometimes and attach a lot of emotional value to it for a good reason.

Quoting Wikipedia: https://en.wikipedia.org/wiki/Product_naming#Principles

- They strategically distinguish the product from its competitors by conveying its unique positioning.

- They hold appeal for the product’s target audience.

- They imply or evoke a salient brand attribute, quality or benefit.

- They allow companies to bond with their customers to create loyalty.

- They have a symbolic association that fortifies the image of a company or a product to the consumers.

- They help motivate customers to buy the product.

I'm sorry but claiming that an uncomfortable discussion on a project's name is not the response one wants to see is exactly what turns this into an echo chamber? To be noted how discussion on a product's technical merits isn't mutually exclusive to discussion on its naming. Both could easily be had, and both have their places, because I don't particularly think either is off-topic. In fact, a previous news.yc thread on this product brought up a similar line of discussion. I guess one could argue the discussion is counter-productive in a technical forum, but I don't think one should avoid it just because... unless the community rules explicitly state so. Technical people aren't robots, at least not yet.

kfrzcode7• 6 years ago

You have good points. Clearly the name of a thing matters.

I'm arguing that the naming of a thing matters less than the efficacy of the thing itself. Doubly so in a hacker-oriented forum.

As for the echo-chamber, that's a classic human fallacy which I think is magnified by the centralization of content aggregation into heavily personalized "newsfeed" style design choices perpetuated by major players.

The epidemic of "outrage culture," or "offense culture" is something comedians and free thinkers have been discussing for awhile, and I think is underpinning of my personal frustrations.

-----

"This idea of "I'm offended". I've got news for you - I'm offended by a lot of things too. Where do I send my list? Life is offensive. Get in touch with your outer adult, and grow up, and move on."

-----

Post, comment, discuss as y'all like, right? That's the beauty of the format. I'm simply one of ∞.

The folks in the community here are brilliant, and the reasonable ones are off doing something meaningful, whereas I'm here arguing for practice, basically.

maxerickson• 6 years ago

Unless a bunch of people deleted their comments, at the time you posted, the comments actually criticizing the name had about as many words as the first half of your comment, with replies mostly pushing back instead of agreeing.

None of the critical comments were particularly strident ('bent out of shape').

Your comment might as well be about itself.

untog• 6 years ago

Not to mention the OP participated in the debate about the name themselves. This being the top rated comment is a self-fulfilling prophecy, it’s getting in the way of a discussion about the tech!

Oh but now I’ve contributed to it as well. Uh oh...

burpsnard6• 6 years ago

What is the term of art ?

To be anti-grandfathered ?

TimTheTinker2• 6 years ago

If you have a strong, negative emotional response to the name of a project, I think letting the project author know can be helpful.

I’m a rational guy to the best of my ability, but FWIW I would avoid using this project solely because of its name (just the thought of that name evokes a sick feeling to me - that’s how strong the negative association is... I won’t explain why as this isn’t an appropriate venue for that).

We are human, and sometimes visceral responses can’t be ignored regardless of their irrationality.

I could invoke Godwin’s law to give a more universal example of words with negative associations, but I’ll refrain.

simlevesque4• 6 years ago

Do you think your reaction is professional ?

jstewartmobile2• 6 years ago

If they named it something like “Pinochet” or PIZZAGATE, I’m sure we’d be getting completely different responses from the same people who think naming a project after a child sacrifice idol is “funny” or “cool“.

akhilcacharya• 6 years ago

What about hearing phrases "Sacrificing a Chicken to Moloch," the "spirit cooking" culture and related symbolism rampant in elite political circles? Shouldn't we be more interested in that?

excuse me...w-what?

frequentnapper• 6 years ago

interesting. didn't realize till now that .ch was a TLD for switzerland!

simlevesque4• 6 years ago

It means Confœderatio Helvetica

voz_• 6 years ago

Pardon my ignorance - Is this the same AOL as America Online?

kfrzcode7• 6 years ago

It certainly is!

gyehuda9• 6 years ago

When this project was first published, the name of the company that published it was AOL. Before that, the company was called America Online. Subsequently it was named Oath, and now it's Verizon Media.

Names change. The common theme: names are not easy, sometimes they are beloved brands, sometimes they fall out of favor. Sometimes they were just bad ideas from the start, but happened anyway.

This gives rise to an interesting challenge for open source projects when you have an open source project in a github org, and the name of your company changes (or your company gets acquired), should you move the project? The problem is real since you don't want to lose your community, but you don't want to be stuck in the past.

narrator• 6 years ago

You'd think with a name like that that this would be a daemon that would take as input the cores of killed child processes.

X6S1x6Okd1st• 6 years ago

Is there anyone that uses something like this in a home/personal setting?

jlgaddis2• 6 years ago

The Filesystem Hierarchy Standard [0] has been around for ~25 years but it still took quite a long time before most Linux distros decided to adhere to it (for the most part; some still do "non-standard" things at times).

Now that we're to that point, please stop screwing it up and coming up wih your own locations for application binaries, data, etc.

To be clear, the "/data" directory -- under which Moloch's pre-built packages apparently install to, according to the README [1] -- is not part of the FHS.

---

[0] https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard

[1] https://raw.githubusercontent.com/aol/moloch/master/release/...

traden210• 6 years ago

This is probably intended to be run in a container, but as that's not specified, it's certainly extremely weird.

deathanatos• 6 years ago

Even inside a container, you want stuff in the standard places, for minimal surprises.

peterwwillis3• 6 years ago

If you look at the source code of most open source software, the devs typically install by default to arbitrary directories. They either don't know about, or care about, operating system standards. When Linux distributions package that software, they choose to change how that software works to align with standards. Usually their own standards, but those can often align with distro-independent standards too. But as a packager, you expect that you'll be modifying paths and making patches.

So really it doesn't matter where a random dev decides to install their software to by default, since either A) it's proprietary and it's non-standard anyway, or B) it's up to us to package it the way we want it.

brigandish• 6 years ago

Not everyone is using a package manager for every install, and even then it's not a magic bullet. I'm a Mac user and I've no idea why devs insist on installing to places like /usr/local or putting configuration in a dot file in my home directory when there's a standard place to put those (~/Library/Application Support), among other well thought out directory structuring[1].

Further to this, the moment you want to install somewhere else or have a dependency elsewhere (e.g. /opt/anything) builds break because of hard coded paths and poor assumptions. To top it all, I've no idea why a package manager like Homebrew then contributes to all of this by acting like no of this matters because a Mac is just like Linux, apparently.

If they don't like Apple's way then why not XDG[2]? Reinventing the wheel badly is more than just hubris, it breaks things and it's annoying.

[1] https://developer.apple.com/library/archive/documentation/Ma... [2] https://specifications.freedesktop.org/basedir-spec/latest/

zwaps• 6 years ago

This post 'gonna cost me, but what you write is endogenous to the FHS making little sense to most people who don't have a unix background.

"Where ARE my programs / settings that I installed, I always have to google it!"

is interrelated with someone else asking

"Where should I PUT this program / setting / data? I guess I'll chose one of those three or so locations where it may fit, or several"

For example: Why isn't there a data directory to be found? Where does the data go? Why not replace nondescript and unhelpful names like "opt" and "usr" (which is btw. not where the user data is!)?

geocar3• 6 years ago

/usr is absolutely where user data was. That's what it stands for "users".

First when / ran out of space, new programs were put on the other disk /usr in /usr/bin so everyone could assume programs were in /bin except ones that were newly installed so they would be in /usr/bin

When /usr ran out of space, a new disk was added as /home and things that were easier to move (user directories) were moved first leaving things that everyone's script was depending on (#!/usr/bin...) where they were.

This was done out of necessity, not out of good taste.

geocar3• 6 years ago

If the use of /data is the reason why you're not using this software, I'm pretty sure you wouldn't find any use of it if it stored data someplace in /var either.

Personally, I think the FHS is stupid marketing wankery that barely made sense in 1995 when the "preciousness" of the root partition was relevant, and has in the annals of time, caused more problems than it ever could have solved. Hey, what's the path to python? Is it:

    /usr/bin/python
    /usr/bin/python2
    /usr/bin/python3
    /usr/local/bin/python
    /usr/local/bin/python2
    /usr/local/bin/python3
    /usr/local/bin/python2.7
    /usr/local/bin/python3.7

This is perhaps the only question I need a "standard" to solve, and I've given up[†]. I don't need to know mail is "sometimes" in /var and sometimes in /var/local. I don't need to know that some of my configuration files are in /etc and others are in /usr/local/etc whilst still others are in /lib someplace. Where are my libraries? maybe /lib, /usr/lib /usr/local/lib /usr/lib64 /usr/local/lib64 and who knows where else. Fuck it, I'll just use find /.

I saw this great quote recently -- I know it was about something else, but I still wish Dan and the other FSSTND/FHS people could have seen and understood it all those years ago:

I suppose people blindly following suggestions on the Internet will eventually learn a hard lesson -- don't.

Because that is what the FHS was: A bunch of suggestions on a mailing list, with a smattering of armchair philosophy lacking any real analysis. But I mean, that's just my opinion. What if I'm wrong? ¯\_(ツ)_/¯

[†]: The answer is whatever /usr/bin/env python says.

t0astbread• 6 years ago

FHS may be a mess but I don't understand how that can be translated into a justification for making it even worse.

jzer0cool• 6 years ago

I read the instructions but was not clear. Is this just a pcap viewer or something more? The way I read it, it appears to be a running daemon which listens on all port which saves the pcap file, which then exposes API's for accessing such data.

If you had an application http server running, is traffic sent to Moloch first, and forwarded to the http server like a proxy?

jlgaddis2• 6 years ago

When running something like this on a large scale to capture all traffic going across a network, you'd typically use a "network packet broker" (cf. Google) that sends a copy of all traffic to the machine(s) running this software.

Your hypothetical application server would not even be aware that this was taking place.

yters• 6 years ago

There should be a competition for most useful software with most offensive name. Then we can make all the software stacks of top companies likely to result in jail time when discussed in public. That would be funny.

nodaemon• 6 years ago

Seriously, AOL named this after the god of child sacrifice, you might want to put a new marketeer on this project, I won't be touching anything with a name like that.

AOL Moloch: open-source, large scale, packet-capturing, indexing database system

Related Stories