HN user

netsec_burn

1,169 karma
Posts7
Comments272
View on HN

Speaking from experience, the LLM agents adapt fairly well to these contexts too. It's not at all FUD, you're at a significant disadvantage if you don't compete with AI now. I went to a CTF recently against teams I have won against every year, and within 10 minutes of the event starting they had solved every challenge. They have an agent loop and it solves everything immediately, so they won. Anyone attempting to solve the challenges on their own has no chance, even if you think "maybe this is too out of the box for LLMs". Furthermore, the DEFCON CTF you're referring to has quals, and if you don't qualify you don't get those challenges in the finals. Quals has mainly binary exploitation challenges which Opus (and others) solve as long as you hold the gas pedal down on your API bill. I don't believe it's hyperbole to say CTF is dead, as a competitor.

MacBook Air with M5 5 months ago

I prefer the Dell Rugged line or Thinkpads, since a single water droplet on the keyboard is enough to kill this laptop.

Claude 3.5 Sonnet 2 years ago

Meta, OpenAI, and Anthropic come to mind.

In TFA they name OpenAI, Google DeepMind and Anthropic.

Claude 3.5 Sonnet 2 years ago

Hit or miss, with significantly more hits than misses. I was able to build a useful cybersecurity service in about 6 hours leveraging Claude to write most of the code. It has certainly made me more productive.

Claude 3.5 Sonnet 2 years ago

Have you tried 3 Opus or 3.5 Sonnet? Are you using it for programming, or something else?

Claude 3.5 Sonnet 2 years ago

Opus remained better than GPT for me, even after the release of GPT-4o. VERY happy to see an even further improvement beyond that, Claude is a terrific product and given the news that GPT-5 only began its training several weeks ago I don't see any situation where Anthropic is dethroned in the near term. There are only two parts of Anthropic's offering I'm not a fan of:

- Lack of conversation sharing: I had a conversation with Claude where I asked it to reverse engineer some assembly code and it did it perfectly on the first try. I was stunned, GPT had failed for days. I wanted to share the conversation with others but there's no way provided like GPT, and no way to even print the conversation because it cuts off on the browser (tested on Firefox).

- No Android app. They're working on this but for now, there's only an iOS app. No expected ETA shared, I've been on the waitlist.

I feel like both of these are relatively basic feature requests for a company of Anthropic's size, yet it has been months with no solution in sight. I love the models, please give me a better way of accessing them.

Claude 3 Opus by a wide margin. I'm a regular GPT-4 user who had tried Claude 2, and went into Claude 3 with muted expectations. I was shocked with how much more capable Claude 3 Opus was compared to GPT-4, it's not even close for my work (optimization, programming/algorithms). I asked 20 questions and Claude 3 solved all of them where GPT-4 failed on all of them. What's more surprising to me is that I don't remember GPT-4 being this bad, I was similarly impressed when GPT-4 was released. The disparity was significant enough for me to reconsider my subscription to OpenAI, but the Browsing capability as well as the Android app kept me. I use Opus for my work now though, by default, and fall back on GPT.

Tools to evaluate LLMs to make it harder to generate malicious code or aid in carrying out cyberattacks.

As a security researcher I'm both delighted and disappointed by this statement. Disappointed because cybersecurity research is a legitimate purpose for using LLMs, and part of that involves generating "malicious" code for practice or to demonstrate issues to the responsible parties. However, I'm delighted to know that I have job security as long as every LLM doesn't aid users in cybersecurity related requests.