Vectorscan – portable fork of hyperscan – high-performance regex matching lib

https://github.com/VectorCamp/vectorscan

by pabs3 • 4 years ago

87 9 4 years ago

philprx2• 4 years ago

Congrats Intel, failure to play nice with community granted you a more up to date community fork.

Now let's play the internal discussion at Intel:

Developer: see !! I told you so, we had to integrate the patch. Community leadership is more important than these corporate plays.

Manager: err... Uhhh.... Now what do we do? How to regain community drive?

Developer: i don't know now.... Invest tons of time/money into hyperscan, have new chips that tap hyperscan natively, or re merge the fork?

...

FridgeSeal• 4 years ago

“Wait, so you’re telling me we could sell a chip that only we could use to tap into for specialised performance?”

Is how I imagine that conversation going, but maybe I’m just cynical.

philprx2• 4 years ago

Cost of new die > market size, probably

NPU was exactly that...

And given scarcity, it's probable that foundries prioritize higher paying processors.

mmaunder2• 4 years ago

Why the fork? Originally, the ARM porting was supposed to be merged into Intel's own Hyperscan, and 2 Pull Requests had been made to the project for this reason (1, 2). Unfortunately, the PRs were rejected for now and the forseeable future, thus we have created Vectorscan for our own multi-architectural and opensource collaborative needs.

Karma's a bitch.

ktpsns• 4 years ago

Vectorscan is typically used in a DPI library stack, just like Hyperscan.
Hyperscan is typically used in a DPI library stack. [https://www.hyperscan.io/about/]

Can anybody tell me what is actually DPI?

I only found the term "Data Processing Interface" and this paper http://cidrdb.org/cidr2019/papers/p11-alonso-cidr19.pdf

But this looks very theoretic, compared to MPI (message passing interface), which actually has a reference implementation and thelike.

ulber• 4 years ago

DPI is deep packet inspection in this context. HyperScan did some groundbreaking work for DPI, where you (1) have to match a huge number of regexes at the same time, (2) need high and consistent performance, and (3) don't need the starting points of matches.

mmaunder2• 4 years ago

Groundbreaking in a cybersecurity context when you're writing regexes to match bad things in a fat network pipe.

rurban• 4 years ago

Thanks, will switch over to this with my perl bindings (replacing the original perl core regex with vectorscan)

Vectorscan – portable fork of hyperscan – high-performance regex matching lib

Related Stories