HN user

thenickdude

282 karma
Posts1
Comments165
View on HN

The biggest trap is the hallucinated citation

The "biggest problem" being the one thing that is trivial to verify against concrete databases is a bit convenient don't you think?

I think it's more likely that it makes mistakes evenly but the one thing that you are able to check with certainty is the only place you discover the errors.

I was trying out an IDE plugin for local LLM integration, but it would just make totally insane edits instead of what I asked for.

It turned out that the LLM-runner's default setting for handling queries that were too long was to silently delete the middle of the query. So the LLM saw the beginning of my ask, and the tail end of the code context, and nothing else.

Interestingly when I asked GPT-4o (at least that's what it said it was):

According to the instructions, which UnicodeCIDFont fonts should be used when generating PDFs?

It replies:

When generating PDFs using reportlab for East Asian languages, you must use specific UnicodeCIDFont fonts depending on the language. According to the instructions, use the following:

Korean: HeiseiMin-W3 or HeiseiKakuGo-W5 or HYSMyeongJo-Medium

Simplified Chinese: STSong-Light

Traditional Chinese: MSung-Light

These fonts must be registered using pdfmetrics.registerFont(UnicodeCIDFont(font_name)) and applied to all text elements in the PDF when outputting those languages.

This list also has the Japanese fonts merged with the Korean list.

https://chatgpt.com/share/6895a4e6-03dc-8002-99d6-e18cb4b3d8...

The free tier now supports connecting to local AI models running on LM Studio or Ollama, but it still doesn't actually function without an internet connection.

If you block access to the internet or to their AI API servers [1], it refuses to start a new chat invocation. If you block access halfway through a conversation, the conversation continues just fine, so there's no technical barrier to them actually running offline, they just don't allow it.

Their settings page also says that they can't even guarantee that they implemented the offline toggle properly, a flag that should be the easiest thing in the world to enforce:

Prevents most remote calls, prioritizing local models. Despite these safeguards, rare instances of cloud usage may still occur.

So you can't even block access to the very servers that they say their faulty offline toggle would leak data to.

[1] https://www.jetbrains.com/help/ai-assistant/disable-ai-assis...

Surely they would at least be protected by Database Rights in the EU (not the US):

The TRIPS Agreement requires that copyright protection extends to databases and other compilations if they constitute intellectual creation by virtue of the selection or arrangement of their contents, even if some or all of the contents do not themselves constitute materials protected by copyright

https://en.wikipedia.org/wiki/Database_right