Tell HN: Stop using LLMs to apply to jobs

https://news.ycombinator.com/item?id=39229186

by breckenedge • 2 years ago

11 36 2 years ago

news.ycombinator.com

It’s really easy to eliminate them because they all answer our hiring questions the same way.

andy99• 2 years ago

It’s really easy to eliminate them

Isn't that a good thing?

samstave2• 2 years ago

Not if you're on the opposite end of a shitty job market attempting to find any way to get a foot in.

The market sucks, and cynicism is growing among all parties.

nonrandomstring2• 2 years ago

Not if you're on the opposite end of a shitty job market attempting to find any way to get a foot in. The market sucks, and cynicism is growing among all parties.

What you say is interesting, because I am watching this "market" carefully as an educator. The disconnection you speak of is real and worrying.

Each day we read stories about layoffs and talented people desperately seeking a job. Each day we read an equal number of stories about how tech firms cannot hire anyone. We hear of "massive skills shortage". In cybersecurity we have 50 jobs to fill for every candidate we can train.

How is this mismatch possible? Someone, somewhere is lying.

In Little Britain we have Dafydd Thomas (Matt Lucas) the controversial "Only gay in the village". The running joke is that his gaydar is so broken (because he is so insular) that he misses every chance. Guys are coming on to him all the time, but he can't see it [0].

That is "industry". It's so misunderstood, so delicate, and special, and hard-done-to.

So called "industry", insofar as it's a collection of HR people with completely made up job-descriptions based on a drunken game of buzzword bingo, hasn't the first clue what it really wants.

Likewise we have universities that churn out tepid, conformist graduates who nobody would ever want to employ. If they're lucky most of them have been taught by adjuncts and temps using materials that are 10 years out of date. If they're unlucky they were taught remotely by some Microsoft product, and would certainly have learned more from just picking up and reading the first chapter of any decent textbook.

I'm talking about the UK now, but I'm sure the same disconnect applies to similar cultures.

A dating game in which both parties are motivated to lie, is just round after round of disappointment.

I wish you luck. My advice is steer clear of any company big enough to have an HR department.

[0] https://www.bbc.co.uk/comedy/littlebritain/characters/daffyd...

breckenedge8• 2 years ago

I figured folks that use HN deserved a heads up. At the very least, turn up the damn temperature so that the responses are varied.

spidersenses• 2 years ago

Stop generating your hiring questions with LLMs first.

breckenedge8• 2 years ago

We don’t.

jrflowers2• 2 years ago

This is useful since you have told us what company you’re hiring for

dsattt• 2 years ago

That’s funny, because I’m going to start using them to apply for jobs.

breckenedge8• 2 years ago

There’s gotta be a service or library people are using for this because we’re getting a ton of duplicate responses to our questions. At least turn up the temperature if you do this.

clusterhacks• 2 years ago

Share some numbers - I'm totally curious how many applicants you are getting for roles total/applicants that you easily identify as using LLMs. Even ballpark would be ok, "We got 40+ applicants for a senior software engineer, at least 50% had identical answers to our questions."

I have some theories about the duplicate answers that go beyond LLM usage . . .

zerr2• 2 years ago

Why would you eliminate someone for using LLMs?

Assuming Lycosing is allowed as well as browsing Experts Exchange... LLMs are new search engines.

solardev2• 2 years ago

Lycosing

What's that? Karaoke with old search engines?

zerr2• 2 years ago

I mean, in case browsing Internet and researching is allowed, so should be LLMs.

OJFord2• 2 years ago

'Googling' and 'browsing Stack Overflow' said in an obtuse way.

I also dislike the former/avoid it myself, but I just say 'searching'...

g4zj• 2 years ago

If candidates are using LLMs to generate answers to your hiring questions, perhaps it's your questions that are the problem. Care to share some of them?

paulcole• 2 years ago

Where I work we ask some candidates in client-facing roles how they’d respond to a hypothetical email from a client about a situation they’re likely to face in the job.

I know an LLM can easily generate a pretty OK and boring reply. But I want to get to know how the person we’re considering would answer it so we can talk about their approach in their next (and final) interview.

We ask people nicely to not use an LLM and explain our reasoning. They can still use one if they want of course.

OJFord2• 2 years ago

Well.. I guess they'd use an LLM? And if the result is 'a pretty OK and boring reply' maybe that's fine or even good?

mattbrewsbytes2• 2 years ago

I'm tempted to make snarky comments about how its a red flag if a company is asking open ended questions with a textbox as part of an initial screen but I'll be open minded and ask:

What hiring questions are you asking?

solardev2• 2 years ago

"Your boss asks you to travel back in time to kill baby John Connor, but you've fallen in love with his granddaughter. What do you do?"

breckenedge8• 2 years ago

One is a basic technical question that I’ve been asked a handful of times in interviews.

We could eliminate the question, but then we’d need to have hundreds of phone screens and literally ask the same question. That’s not feasible. So this is how we screen.

We’ve received over 250 applications in ~1 day.

mattbrewsbytes2• 2 years ago

Is that a quality way to screen though? Anyone can look up answers to basic technical questions.

Also have you considered that maybe ChatGPT itself is applying? Its gotta feed all those babyGpts when it gets home from work. Might be a good hire, LOL.

markus_zhang• 2 years ago

I mean people can still modify the answers. Let me say that in the era of LLM it's probably a lot better just to change the interview format -- for example, put questions and coding tests in zoom/in person instead of online.

breckenedge8• 2 years ago

That’s infeasible based on the number of candidates who have applied.

Sateeshm• 2 years ago

You should probably read the resumes than asking the candidates to answer your questions before an interview.

gngoo• 2 years ago

Sounds like your hiring is the problem. Not people applying with LLMs.

JamesLeonis• 2 years ago

Maybe slip in a poison string [0] in your questions that trip up the LLMs?

[0] https://twitter.com/goodside/status/1745511940351287394

vlod• 2 years ago

TELL Employers: Stop using LLMs/AI to filter job applications.

I'm convinced that 99% of my applications are not getting through to a human.

breckenedge8• 2 years ago

That’s not happening in this case.

ecf• 2 years ago

Unfortunately this ship has sailed. The job market is broken right now for software engineers and it’s far easier to use AI to apply to a hundred jobs in the of hopes one gets backs to you instead of 5 handcrafted applications per day that have tailored responses.

jrflowers2• 2 years ago

Attn applicants across the job market: stop using ChatGPT to write answers! This advice does not apply to people doing hiring though, as I personally do not use it for that purpose for the company that I decline to mention

Tell HN: Stop using LLMs to apply to jobs

Related Stories