HN user
You're likely right here, combining two trust systems does add complexity without solving the core problem. While browsers requiring CT was a great step forward, it's surprisingly under-utilized by orgs. I wonder if this is due to limited tooling for log interaction, or just general lack of awareness?
I know this is an oversimplification, but if the main issue is the single point of failure (centralized trust), wouldn’t a potential solution be to layer independent verification mechanisms on top of the current system?
For example, a secondary DNS-based verification layer where a site’s public key is published as a DNS record (though that would likely need DNSSEC to be effective). It seems like it could complement the existing CA structure without replacing it entirely.
Baader-Meinhof strikes again - checked this out in the morning and just caught your Citibike tweet. You're on a roll today!
Ahhhh, that tracks, cheers mate.
Exhaustive/Robust is the way for sure.
Minimizing storage was a priority for me since it's just a small side-project/automation.
I've looked for information on what the hell the `flowers-to-the-world` entries are that pop and have found nothing, curious what's going on there.
I also noticed you are ingesting/storing flowers-to-the-world.com certs, not sure what stage of optimization you are at but blacklisting/ignoring these certs in my ingestion pipeline helped with avoiding storing unnecessary data
I'm not sure but I believe that's used by Google internally for testing purposes.
For example if you search google, it returns 120k+ results, and these useless results are at the front.
Awesome, I will keep my eye on this for sure, I've spent the past few months tinkering with ingesting CT logs for bug bounty automation.
Curious if you're running your own CertStream server, or just continuously polling known CT logs with your own implementation.
Have you considered adding a monitoring feature where a user can enter a domain to be monitored and then be notified if a "similar" domain comes across the ingestion pipeline.
This would be useful for early detection of potential impersonations/typo-squatting domains typically used for phishing/scams.
Something as simple as a configurable levenshtein distance/jaro-winkler similarity check across CN and SAN of all new certs maybe? (user can configure with threshold to control how "noisy" they want their feed).
I'd imagine it's a combination of
- CT log monitoring (https://github.com/CaliDog/CertStream-Server)
- Mass-Scanning across ipv4 on 80/443 at the least?
- Brute-forcing subdomains on wildcards with large DNS wordlist (like something from assetnote: https://wordlists-cdn.assetnote.io/data/manual/best-dns-word...)
- Scraping/extracting subdomains/domains from JS
But I've never attempted to enumerate subdomains on this scale before, so I could be missing something obvious