HN user
If I am being honest, the value came from doing evals and testing against different models.
Essentially all I needed was a way to upload a data set, run tests against that data set and spit out a percentage of pass fail.
Braintrust makes this pretty easy, but If I was to do it again I would vibecode the same functionality.
I highly rate Braintrust.
It wouldn’t be too difficult to build something like that for your own usage, but I found it pretty easy to get datasets set up.
Essentially a game changer in understanding if your prompts are working. Especially if you’re doing something which requires high levels of consistency.
In our case we would use LLM for classification which fits in perfectly with evals.
“I admit that I still disagreed with him after the exchange, but I had a new respect for him as a designer because he was able to articulate a rationale for his decision.”
Any competent designer gets really good at justifying their decisions. Everyone has an opinion about design and thinks that their taste is correct.
I’m glad I don’t have to deal with that on the software side.
Aren’t people researching the companies they are applying for?
Also, I don’t think I have ever applied to a fake job.
I’m sure the welfare of the Iranian people is a top priority for Trump.
Marginally related, I feel the same way about honesty, especially in a work context.
I’ve always prided myself in being an honest but considerate person.
A recent experience with a colleague who weaponised my honesty in an attempt to manipulate me has left a foul taste in my mouth. Luckily their contract ended and the problem resolved itself.
But I remember distinctly feeling that I will be professional and polite but I do not automatically owe anyone my honesty.
Talk to people.
There are an infinite amount of problems to solve.
Deciding whether they’re worth solving is the hard part.
We analyse thousands of lines from a csv using an LLM. The only thing that worked for us was to send each individual line and analyse it one by one.
I’m not sure if that would work in your use case, but you could classify each line into a value using an LLM then hard code the trends you are looking for.
For example if you’re analysing something like support tickets. Use an LLM to classify the sentiment, and you can plot the sentiment on a graph and see if it’s trending up or down.
I’m really not sure I follow this argument.
A lot of software has friction to get to the value. This is often because of constraints not choice.
To give a concrete example of this, in my company we had users upload files for analysis. To get the export for the file, it took many steps. Not hard, but a lot to get done.
We switched it to an integration and now it’s 3 clicks. We’ve gone from 10% of users onboarding to 100%.
It doesn’t mean we get people to stay, but the barrier to understanding if our tool provides value to them has completely disappeared.
I’m very curious though, what value did you strip away when trying to make your product easier to use?
This also goes the other way too, you don’t assume the original is incorrect.
I see this a lot with developers who come in and start to criticise before understanding.
There is always a reason for why something is as it is, and it’s unlikely that the people before you were just idiots.
How do you introduce any tool/change to a team of people?
You get buy in, start having conversations see what AI people have explored. Have they tried claude? Do they prefer other tools? If so why? What are the objections. Actually listen. I’d also showcase what you can do. I love to present what codex has found when debugging something, or a prototype I’ve put together.
If you have the budget pay for subscriptions so they can play around.
Also, you say that development velocity is a big problem, but I would dive into why that is. You may be disappointed when velocity remains the same with AI tools.
My time in the military has made me hate acronyms with a fury.
If you focus on the differences that’s all you’ll see.
The opposite applies too, if you try hard enough you’ll find plenty of similarities with an Eskimo. We’re all just people in the end.
I didn’t even realise that was a possibility with iOS. Thanks!
I’m curious have you ever been burnt or seen anyone burnt by copyright infringements in code?
Sometimes it’s super obvious because a game company steals code from a previous employer, but I have never seen this play out in entreprise software.
What do the markdown files look like?
Curious to hear more about this, do you have any examples?
I think we misunderstand each other.
Accounting is quite broad and there are many examples of where LLMs can help.
For example, what is tax deductible is subjective to local laws. This is essentially a classification problem that LLMs are particularly good at.
I understand your pain, but this is very country dependent.
For example, most of what you describe can be solved using Xero in Australia.
Why is AI unethical in its current form and what would make it ethical for you?
I find this take so strange, do you find no value in AI?
I don’t even want AI to replace us, but it’s a great tool with many use cases.
To be clear, I agree with you.
Our target market is accountants. I want to help them not replace them.
In the same way security audit tools aren’t replacing professionals, but that can help on initial scan.
I disagree on this, there are plenty of problems in accounting that an LLM can help with.
I’ve built some software[0] that analyses general ledgers and uses LLMs to call out any compliance issues by looking at transaction and account descriptions.
Is it perfect, nope. But it’s a hell of a lot better than sifting through thousands of transactions manually which accountants do and get wrong all the time.
This is key, I’ve had great work environments where we’ve had lots of fun solving problems together.
At my last work place everyone talked about how great therapy was. Curious I decided to try it myself.
Turns out therapy was great, it made me realise that my work place was toxic and since leaving I haven’t needed a therapist again!
I really like co-working spaces, but not enough to pay for them when I have an office I can go to.
Funnily, I do a lot of those things. I have a bunch of hobbies that are very social.
But for me it’s not really that. It’s about walking in a place and knowing everyone, having small talk and feeling comfortable chatting.
Work takes up a lot of time, so I prefer to have that along side it. Rather than only in my spare time. I
I don’t do well with 100% working from home.
My preference is 3 days in the office, I find anything less than that and I struggle mentally. My home starts to feel like a prison and I lose connection with people.
I really value human connection and I just don’t get the same thing online.
I feel this pain.
We have an intern that is finishing a four year computer science degree that has no clue what git is, never used a log and all he presents is AI garbage.
I find it profoundly depressing to try and teach someone who has no interest in the craft.
I’ve seen push back on this idea multiple times on HN.
Humans review their own work, just like LLMs can.