Do LLMs identify fonts?

https://maxhalford.github.io/blog/llm-font-identification/

by alexmolas • 12 months ago

64 30 12 months ago

maxhalford.github.io

Doohickey-d• 12 months ago

I'd be curious how much better a more expensive LLM would do - gpt-4o-mini and gemini-2.5-flash-preview-05-20 are definitely not the most capable LLMs one could have chosen.

double051• 12 months ago

Maybe they're just cheap and fast enough for the author to perform an affordable analysis?

I agree that using the frontier models would be much more interesting.

Workaccount2• 12 months ago

Maybe I simply don't know how advertising works, but wouldn't it be totally possible that these fonts are just one-off drawn in text?

rubyn00bie• 12 months ago

I would say there’s a good chance they could be one-offs created by whoever was doing the ad. If you’re paying an artist, having them do the lettering could certainly be cheaper than licensing a font for the purpose (or developing a font that’ll never be used outside of one, or a series, of ads).

k3liutZu• 12 months ago

My fellow designer friends would often do this. But they would start from actual fonts and do slight (or more than slight) adjustments to them to match what they wanted as an outcome.

b112• 12 months ago

Yes but if you mess with fonts too much, then this can happen:

https://www.youtube.com/watch?v=snjCj0ntG8E

smallerize• 12 months ago

The benchmark only evaluates responses once the community has identified the font.

E.g. https://www.dafont.com/forum/read/569491/taylor-swift-font-p...

empath75• 12 months ago

That isn't the font, you can look at it yourself, it doesn't match.

micromacrofoot• 12 months ago

How close are the wrong guesses? Fonts are fairly incestuous because the shapes of the characters themselves can't be copyrighted (only the code), so there are sometimes dozens of clones of very similar fonts... especially on a free site like dafont

gdudeman• 12 months ago

Missing from the methodology: - was thinking on or off? (At least for Gemini) - was web search allowed? - was tool use allowed?

It’s quite likely LLMs don’t “know” the fonts in the dataset, but they could figure many of them out.

Rastonbury• 12 months ago

I've asked LLMs to suggest fonts/similar fonts for me from screenshots and seems like they are close enough to my untrained eye

HocusLocus• 12 months ago

Why don't you just ask the document creator?

Every time I turn around these days I encounter someone ready to use an infinite amount of energy that is being paid for by other people, to 'simulate' some analog process by temporarily taking the reins of some data center that is burning megawatts of energy. We are being given the reins for 0.5 seconds but very soon the horse will gallop away unless we have a lot of money to spend.

qezz• 12 months ago

It's not always possible to ask the creator, especially for the old pieces

mjburgess• 12 months ago

In this case, it's seems it is highly likely to be possible: https://www.studioheavenly.com/our-work/mira-wellness (1 google)

she46BiOmUerPVj• 12 months ago

I would have never thought to not use "what the font"

https://www.myfonts.com/pages/whatthefont

pwython• 12 months ago

Yea, I just used WTF to help that guy who was waiting 2 years to find a font.

https://www.dafont.com/forum/read/522670/font-identification

squigs25• 12 months ago

WTF is often wrong, and actually, I don't think your answer in the 2 year old thread is correct

Lemaxoxo• 12 months ago

Op here. I tried what the font a bit but didn't mention it in the article. I didn't get good results with it. Although it's probably a good idea to ask it for a guess, and feed that to the LLM too.

cormullion• 12 months ago

I suspect that few professional (paid for) adverts use any fonts from dafont.com, and many fonts would anyway be unavailable to ordinary users. The current font recogniser programs are usually trained on commercially available fonts

StellarScience2• 12 months ago

With the latest Microsoft Word, if you open a PDF that is a scanned image of a document and convert it to Word format, it does a pretty decent job of not only OCR (optical character recognition) but also picking matching fonts for various sections.

I just tested this with my internet connection disabled and it still worked. Since it's doing local processing, I suspect it uses traditional OCR algorithms rather than LLMs.

As the article concludes, LLMs aren't magic, they're just one useful tool to include in your toolbox.

aaroninsf• 12 months ago

It's pretty easy to imagine an evolved mess of an open ad hoc but broadly adopted ecosystem where LLM are surrounded by a bewildering array of Node-like domain-specific extensions.

Security concerns aside (...) that sounds pretty useful.

StellarScience2• 12 months ago

Right, for example early LLMs were notoriously bad at math, as they had been trained on language. They'd get simple math right, likely due to "rote memorization", but couldn't do basic arithmetic with 3-digit numbers. The common AI agents seem much better now. I suspect they added separate math processing logic and trained the LLMs to recognize when and how to delegate to it, though I'm not certain of that.

Similarly coding-focused LLMs can access backend engines that actually run the code and get feedback, either to show the user or to internally iterate.

Having a whole host of such backend processors would be great. Users still only ever have to interact using natural language, but get the power of all these specialized tools in the backend. There are some tasks LLMs can do, but special-purpose algorithms may do better, faster, and/or with less energy usage.

mopsi• 12 months ago

I recently tried to identify a font from a screenshot of an ad and used everything I could find, from WhatTheFont to LLMs. The LLMs were hopeless at identifying the font from the screenshot, but ChatGPT eventually led me to the correct result after I threw away the image and started describing the font in plain text: monospacing, a dot in the middle of the 0, and (presumably) wide usage. It turned out to be Ubuntu Mono. It was surprising that so many obscure fonts were suggested, none of which were even a reasonably close match, while Ubuntu Mono was completely overlooked.

elicash2• 12 months ago

Results here are bad, obviously, but it'll be interesting when LLMs can not just identify fonts but unredact pieces of documents in places where just a few words are removed by analyzing the length of redaction, combos of letters that fit into it, and the context.

lblume• 12 months ago

Why would you even need LLMs for that? Notwithstanding context, finding text that fits into a given bounding box is already perfectly doable via a classical algorithm (in this case e.g. based on dynamic programming).

elicash2• 12 months ago

Notwithstanding context

This is actually quite important! Especially when you're not talking about a single word/name but a group of several words/names.

larodi• 12 months ago

What makes us think font information made it into the traning set atll, rather than something more along the line of "all chars that look like this one are to be interpreted as 'a'". doesnt need to provide font name for it.

Do LLMs identify fonts?

Related Stories