benwills

I'm assuming you're talking about the featured image for the post.

What you're seeing is not uncommon in America, especially in the mountains. Roads for traffic in the opposite direction can be separated from your road.

It's also often the case that the other road might be on the other side of a hill/mountain. I've seen that in multiple states across the US.

This specific instance, where two separate roads are going in the same direction can occur where the left road doesn't have exits/entrances, and can move faster than the right road when there's traffic. The left road may also be switched to change traffic directions. I most recently saw this in some mountainous terrain in Arizona.

It was added in the most recent release: https://manticoresearch.com/blog/manticore-search-27-1-5/

This was annouced a few weeks ago, and goes into effect today.

As someone who has done a lot of downloading/parsing, this is so awesome and impressive to see.

One thing to think about, which I also struggle with when it comes to large and complicated datasets, is the UI. Even being in the search industry for a long time, it's difficult for me to concretely see how I would use this.

I'd suggest taking a small sample of the dataset that might be reflective of how people would use it, then make that segment public and immediately searchable without registering. eg: One year of articles related to the Olympics.

What I've found is that it's hard for a lot of people to imagine how they would use something without actually using it. So giving people the actual experience of searching the archive and interacting with the results would go a long way.

Again, congrats on the work. This is really impressive work.

Yesterday, Opus 4.6 cost three credits. You can no longer use 4.6 or 4.5.

Opus 4.7 is available today for 7.5 credits per prompt.

They have also suspended new signups.

After testing all of the major IDEs/tools that integrate with LLMs over the last four weeks, I was happy to settle on Copilot. I, and others, seem to be a lot confident in that decision. Especially since there seems to be no refund path for people who prepaid for a year.

In my 30+ years online, I've never seen an industry change so much in terms of pricing, service levels, etc, as I have the last two months.

I'm really curious where all of this lands, and if AI coding tools will be something that only a small percentage can genuinely afford at a competitive level.

I'm also not seeing it. Did you have the "Auto-reload" on or off?

Both. Just sent an email. Thanks!

I have been attempting this exact sort of clustering solution for a few years now (on and off as a side project). Do you have source code available, or more detailed explanations/resources of how to approach this?

Edit: I just looked around for your YOShInOn RSS reader code and couldn't find it. I did find a number of references it looks like you've made to it on various forums, etc over the years.

In the photo of him on his last day [0], there's a cassette deck on his desk.

That could be something mundane, but I'd like to believe something crazy happens if you yell at it [1]...

[0] https://www.brendangregg.com/blog/images/2025/brendanoffice2...

[1] https://www.youtube.com/watch?v=tDacjrSCeq4

URL shorteners significantly increased in popularity as Twitter did. There was originally a 140 character limit that became quite squeezed when adding most URLs, especially to blog posts where the title is part of the URL.

Later, adding things like analytics and tracking (eg: not just in social media, but also in email campaigns) became another reason to use them, especially for those less tech inclined.

For those interested, there's a new set of hour-long videos on the PBS site that has more about the recent Pompeii excavations.

There are four so far. Not sure if there will be more: https://www.pbs.org/show/pompeii-the-new-dig/

I've heard 6 at Fordow, and 30 or so Tomahawks across Natanz and Isfahan.

I don't know you're experience with hash functions, so you may already know what I'm about to say.

This is a minor example, but since you asked...

https://github.com/Cyan4973/xxHash/blob/dev/xxhash.h#L6432

That's an example of a fair number of accumulators that are stored as XXHash goes through its input buffer.

Many modern hash functions store more state/accumulators than they used to. Previous generations of hash functions would often just have one or two accumulators and run through the data. Many modern hash functions might even store multiple wider SIMD variables for better mixing.

And if you're storing enough state that it doesn't fit in your registers, the CPU will put it into the data cache.

Sure. Any worthwhile hash function will fit in the instruction cache. But there are ways to make more or less efficient use of the data cache.

I think they may be asking about the CPU cache.

I haven't used Claude Code, but I have used Windsurf, Cursor, and Continue. They all do well with their own "rules" files. I essentially understand that as something similar to a System prompt sent before a chat session. I even have pretty specific rules on styling that are unique to me, and it generally follows those.

It's also worth asking what rule it would need in order to follow the rule. On occasion, a rule I've added isn't quite followed. So I'll respond immediately pointing out what it did, that the rule is in the file, and then will ask it to tell me how I should modify, or add to, the rule in order for it to be easier to follow.

I'd imagine Claude Code has something similar that might be worth looking into.

I do that. And it's not bad. But it still doesn't treat the input/interface strictly as plain text.

I can't remember all the little things that happen, which wouldn't happen in a plain text editor, but if you type hyphen-space, then hit enter, the line is deleted and your cursor stays on that line instead of advancing to the next.

It's a trivial example, but things like that happen.

I have used Joplin daily for years. The only thing I _REALLY_ wish they did was allow you to just store and display notes as plain text.

Instead, there is always either markdown or rich text formatting involved. And there's no ability to disable that.

That always seemed odd to me to force that kind of decision on users.

This is true in the US legal system. This is far less true in the general social system.

For people interested in this sort of thing, I recently published a blog post looking at counts of HTML tags and their attribute values from a 2.9B page Common Crawl dataset. [1]

There's also a SQLite DB available to download of the top 1k tag+attr+value combinations. [2]

[1] https://webparsing.io/blog/hidden-in-html-parsing-page-layou... [2] https://webparsing.io/data/commoncrawl-2024-11-html-tags-att...

If anyone is interested in an example of how ZSTD's dictionary compression performs against standard gzip, a number of years ago I put together an example using some Common Crawl data.

"I was able to achive a random WARC file compression size of 793,764,785 bytes vs Gzip's compressed size of 959,016,011" [0]

In hindsight, I could have written that up and tested it better, but it's at least something.

[0] https://github.com/benwills/proposal-warc-to-zstandard?tab=r...

This is an analysis I put together of the November 2024 Common Crawl HTML/Warc dataset. I counted HTML tag attribute values to identify the most common values per tag+attribute combination. I've done this analysis several times over the years and have found it to be invaluable when it comes to writing parsers.

The post is interactive, allowing you to search on the 500 most common values per tag+attribute. There is also a free SQLite database available for download of the top 1,000 values per tag+attribute.

This is the first post of an 8-part series that builds toward writing an article parser, the lessons from which can be transferred to writing any other kind of parser you might want.

This is my first time to publish content like this and I'd love any feedback you might have.

If I understand correctly, this is the SVN repository for all public WordPress.org plugins:

https://plugins.svn.wordpress.org/

If so, I'd imagine creating a mirror of the registry would start there.

I've heard "fast" as a contranym to "moving quickly," referenced as "to fast" or "fasting." ie: not eating at all.

Edited to add: Sticking with the context of food, "fasting from food" contradicts someone being a "fast eater."

Also, if you get stuck with it or want to configure it differently (eg: using a struct definition or values defined elsewhere), feel free to reach out to my username at gmail.

Just make sure to use a dot between my first and last name.

Here's a sample gperf config file I just recently used/auto-generated (with the data labels modified):

  struct SomeData_Gperf_st {
    const char* strKey;
    SomeData_et enumVal;
  };
  %compare-lengths
  %compare-strncmp
  %enum
  %ignore-case
  %includes
  %readonly-tables
  %struct-type
  %define slot-name            strKey
  %define lookup-function-name SomeData_Find
  %%
  val 1, SOME_DATA__VAL_1
  val 2, SOME_DATA__VAL_2

The lookup table is defined after the two percent signs; %%. You simply use comma separated values. And then you just have to define the const char* to use for the lookup string, as defined by `%define slot-name`

So, gperf will then generate a function, SomeData_Find(), which returns a pointer to a SomeData_Gperf_st object. Or NULL if it's not found.

There are a number of other options available, but what you're seeing above are the options I use most often.

And, finally, I execute a command like this to run it with that config:

  gperf SomeData.gperf.confg --output-file=SomeData_Find.c --multiple-iterations=100

If the author sees this, there's a comment on your HTTP Verb parsing post that says:

"The major drawback of gperf is that it generates function "exists", while we need a "lookup"."

gperf allows for returning actual values. I use it in a whole lot of ways (enough that I've written code that imports data to then generate enums, execute gperf and import generated code, etc), including for generating lookups for HTTP Verbs.

Let me know if you want to know how that's done and I can give more details.

As someone who has written a fair amount of networking code in C, I'd assert that without an understanding of what's in Beej's guide, it will be nearly impossible to competently use io_uring in a networking context.

Thanks for that link. I've done a bit of work with the Common Crawl data (and proposed moving to ZSTD with a proof of concept and performance metrics in C a few years ago).

I'll send you an email later this weekend to connect.

In a very different way, I'm also involved in a search-related project. (edited to add: also going solo on my project as well) If you ever want to bounce ideas around, I'd totally be up for that.

Related: you mention other sources than Common Crawl for WARC data. Is there a list of those somewhere?