Engage with an unemployment algorithm

104 77 4 years ago

filt• 4 years ago

I tried it from my phone. A lot of the times, trying to sexpand the select list moved me to the previous question.

After answering all of the questions, I came to a black and empty section, scrolling down from there gave me an option to do the test again.

kkarimi• 4 years ago

Same, seems like not working on iPhone safari atleast

jonsen• 4 years ago

Same on iPad Safari.

kgeist• 4 years ago

My phone gave up and the browser crashed :)

kzrdude2• 4 years ago

This all seems a bit backwards but I guess we're just looking at a visualisation of the algorithms.

The "forwards" part would be to look at risk factors and how to mitigate or improve those points. Of course together with the unemployment office.

DocTomoe• 4 years ago

If you talk to unemployment offices, worldwide, that's pretty easy: accept any job, for any salary. There's always work in the sulphur mines.

It's better to try and improve yourself without involvement by biased and agenda-ed state actors. Maybe with a coach. But not with g-men.

kzrdude2• 4 years ago

The ideas behind Reagan's quote "I'm from the Government and I'm here to help." is the most damaging meme in american politics

lancebeet• 4 years ago

Unemployment sure doesn't seem to be a big issue for front-end developers in Denmark.

Izmaki4• 4 years ago

Anything that allows you to pocket a starting salary of 36k / month fresh from a B.Sc. does not have a problem with unemployment, lol...

tokai3• 4 years ago

The job marked for B.Sc.'s is actually quite small in Denmark. Everybody takes a master.

googlryas2• 4 years ago

Is that take home per month or pre-tax and all that?

PeterStuer2• 4 years ago

Even when you are over 56?

Olphs• 4 years ago

Is the "algorithm" that simple tree at the end? Really confusing

hampereddustbin6• 4 years ago

Maybe it is to bring about it's simplicity, or a sarcastic tounge-in-cheek jab at it. The word "algorithm" doesen't really distinguish it's complexity in any way. It's just a sequence of steps.

yobbo2• 4 years ago

It's most likely a trained decision tree with binary output.

laputan_machine• 4 years ago

Trained in what way? I'd have thought this was just a hard-coded DT, it's multiple-choice questions all the way down

danuker• 4 years ago

https://en.wikipedia.org/wiki/Decision_tree

bryanrasmussen2• 4 years ago

part of the reason I posted this is there has been some recent controversy as to whether or not the algorithms used are racist https://www.version2.dk/artikel/styrelse-frikendes-algoritmi...

translated to English

https://www-version2-dk.translate.goog/artikel/styrelse-frik...

on edit: changed where to are, not sure why I wrote where instead of were, but there we go.

jmeister2• 4 years ago

Very interesting. I'm working on visualization and explainability of ML algorithms.

Thank you for posting this.

bryanrasmussen2• 4 years ago

I doubt it is a real ML algorithm, there is some discussion of the algorithm in the comments, so it seems the one currently in use is very downgraded from an earlier one that had more variables but not quite sure about that part.

notahacker3• 4 years ago

Certainly it's difficult to imagine someone motivated by accuracy rather than making political points about migrants (and education and desire to work) concluding that what industry people have worked in, the years of experience they have, the level of seniority they've obtained, how much of their previous work is in Denmark, previous salaries they've earned and whether they're willing to take a lower paid job etc are all less important than whether they're non-Western and what their highest level of education is...

Admittedly, it's also the type of model you'd expect someone working back from very limited available data to come up with, but I struggle to believe it's a particularly accurate model

stevesimmons• 4 years ago

They could well be the few factors that, in aggregate at a population level, give the greatest explanatory power. And which have then been roughly calibrated.

The resulting model can be well be accurate in aggregate for informing policy decision tradeoffs. Just totally irrelevant for an individual's particular circumstances...

ClumsyPilot• 4 years ago

If this algorythm was a fancy neural net, we wouldn't even be able to inspect it and have this discussion

timvdalen2• 4 years ago

Turns out I'm at high risk of long term unemployment if I were to move to Denmark...

smcl• 4 years ago

Me too, not sure if I should be worried or if the algorithm is a bit wonky. I am in my mid-30s, graduated with university degree in a STEM field, and have been continuously employed since I left university but filled it out as if I had just quit. The summary after I entered everything showed the following decision tree:

How quickly do you believe you will get a job: "Don't know" (I've enough savings to carry me a couple of years, no rush)

Origin: "Western immigrant"

Verdict: high risk of long-term unemployment.

If the little summary is to be believed, there are no other important factors. Changing my "Don't know" to 3 months reduced me to "low risk of long-term unemployment". So if I’m not mistaken, the main factor for their estimate for how long you will be unemployed … is your estimate on how long you’ll be unemployed?

tokai3• 4 years ago

A bit wonky is a very mild way of putting it. Absolutely garbage seems more precise.

timvdalen2• 4 years ago

I think we entered the exact same data!

mrweasel3• 4 years ago

It might be a case of the algorithm not being designed for your case. If you work in IT, you most likely only have to deal with the unemployment office is you've been hit by stress and is trying to recover. The unemployment office almost exclusively deal with people who are very reluctant to seek employment or have other issues that might prevent them from functioning in the job market (lack of education, mental issues, health issues, language barriers and so on). That does leave them with some funny interactions when someone outside those categories show up.

A friend of my took a spent month, after getting hit masters degree in CS, finding the right job. That meant that he need to show up at the unemployment office. They where only able to offer him a forklift certification, which he didn't need or want. In the end they just left him alone for a month and he found the job he wanted.

Overall the Danish unemployment office is remarkably bad at finding jobs for people or even help them get better qualifications. There's an entire book written about it (based on one journalists experiences). In the book the only person, that particular office found a job for is one they hired themself.

YetAnotherNick• 4 years ago

It missed the "How good are you at leetcode" question

hampereddustbin6• 4 years ago

Instead of a specific question that most of the working sector is unfamiliar with, the form goes only by the field of study to guess where you're working

martin_a2• 4 years ago

Seems like this joke got over your head. :-)

m00dy• 4 years ago

I think Banks should also expose the internals of their credit rating algorithms. Openness and transparency should be part of new world order.

hampereddustbin6• 4 years ago

They might be afraid their algorithms would be too easy to manipulate if you knew all of it's factors. Since testing the all of the possible factors would be cost-prohibitive for most, it's not exactly 1:1 with software development

justinclift2• 4 years ago

They might be afraid ...

It's not really relevant what they're afraid of. They should be made to be transparent, and it should be enforced when they don't meet the mark.

GistNoesis• 4 years ago

Is this the real life or is it just fantasy ?

How far is this demo from the real algorithm ? In the demo it seems they use a very simple decision tree just based on very few criterion (In order : Self-perceived ability to get a job, Origin, Age, Recent employment rate).

Does this simplistic (racist and ageist) model have been derived from real data and is in use, or is it just some kind of propaganda (from CopenHagen University?) and the real model they use is much more complex.

Random forests can be made much bigger and match data quite well provided they are big enough, but the bigger they get the harder they are to explain.

Caught in a landslide, no escape from reality.

nonrandomstring• 4 years ago

The general problem with all this "algorithmic" (lazy-arse avoidant) jiggery-pokery is that it entrenches a bunch of extant values. An "expert system" - what this is - is nothing but the sum of historical expertise minus the human wisdom lost encoding it.

Thunderbolt and lightning very, very frightening.

PeterStuer2• 4 years ago

It seems to be the result of analyzing 152.000 unemployment case histories. While the outcome is rather coarse grained I have a feeling it wouldn't be too far of reality for a quick risk assessment.

You can say e.g. the resulting model surfaces 'ageist' rules, but isn't that just a reflection of reality? Jobs chances for the average >56,5yo are indeed pretty slim, and not just in Denmark. Should the model lie about that?

jmeister2• 4 years ago

Is it racist or ageist if it is accurate?

Aeolun2• 4 years ago

This website is quite terrible on mobile. I don’t know how many times we’ll have to tell people to stop hijacking the scroll position, but I suspect until the end of time…

Also, none of my questions mattered, by virtue of being a western immigrant and expecting to look for a job from 3-6 months I’m somehow at risk of long term unemployment.

hampereddustbin6• 4 years ago

It would be really great if any javascript binding could be turned off in about:config or it's equivalent in other browsers. It's not really a solution, but would bandaid a lot of bad design which I can't see any legitimate use for (like scroll-hijacking).

I guess you can already overwrite any existing function with your custom non-function with userscripts, but that seems quite hacky and non-portable, since most mobile browsers don't really support extensions (Firefox does, but only a small subsection)

dry_soup• 4 years ago

It's pretty terrible on a desktop browser as well

hahamrfunnyguy• 4 years ago

It's quite terrible on desktop too.

marcinzm• 4 years ago

Also, none of my questions mattered, by virtue of being a western immigrant and expecting to look for a job from 3-6 months I’m somehow at risk of long term unemployment.

I wonder if the underlying cause is "having savings and not needing a job to have a comfortable living." May also tie into how easily or not you can get government support. The more desperate you are the more likely you'll settle for any job as quickly as possible.

martin_a2• 4 years ago

Yeah, I got the same result, too. Really wonder if this algorithm is good for anything. I tried to play "Give me a break, then I'll look for a job and I am certain I'll find something" and it just tells me I've got a high risk for long term unemployment... Ok, whatever.

dusted• 4 years ago

Is it any comfort if I tell you it's also quite terrible on desktop? :P

Aeolun2• 4 years ago

A little bit. At least I don’t have to regret not looking at it in my desktop browser.

tgsovlerkhgsel2• 4 years ago

Based on the visualization the algorithm seems to be:

If you input that you expect to be unemployed for a short/long time, it returns that assumption. Only if you say you don't know...

Immigrant/descendant: High risk of long term unemployment, right away. (Edit: Apparently except non-western immigrants, based on the source code!)

Age is the next check that can classify you as high risk, I didn't look for the threshold (Edit: 56, according to the source). As long as you're below that, the deciding factor is...

How long were you employed out of the past 36 months? 12 months is apparently good. (Edit: <2 is the threshold).

It seems like most of the questions don't seem to be considered (if the visualization is to be believed) and it's a very simple "if x return foo" style algorithm, not some fancy scorer/ML.

Look for function setValues() in https://ledighedsalgoritmen.dk/p5vis.js

notahacker3• 4 years ago

Immigrant/descendant: High risk of long term unemployment, right away. (Edit: Apparently except non-western immigrants, based on the source code!)

The latter part is probably on the basis that non-Western immigrants mostly needed skilled worker visas, whereas descendents and 'Westerners' (which apparently maps to "EU members" in this case) don't. Though as with most of the rest of it I'd struggle to see this as a conclusion from advanced modelling and not just a stereotype kept relevant largely by not asking the descendent or 'Westerner' what type of work they'd been doing and how long they'd worked in Denmark

silvestrov• 4 years ago

There are (compared to SV) very few asians in Denmark. They also integrate well so the Danish "I don't have a job" system isn't focused on them.

It is a long-standing taboo to split "non-Western" into "middle east" and "asian". Avoiding the split makes the bad numbers less noticeable.

bjarneh• 4 years ago

Look for function setValues() in https://ledighedsalgoritmen.dk/p5vis.js

It seems you've found the algorithm right there. Not only is this algorithm the good old mega-nested-if-test-of-hard-coded-answers for a few of the questions asked. It basically checks if you think you are at risk, then puts you "atRisk" if you are older than 56/about to retire/an immigrant/pregnant.

Sadly; that is probably enough to report relatively correct values.

hampereddustbin6• 4 years ago

There is another way to look at it; a student was able to visualize what they wanted with minimal investment in learning secondary tools to their actual field.

Similar issue is often misattributed in other places like game engines aswell. For example Unity is often labeled as lower quality because of the correlation with bad code quality in many of the games made with it, while forgetting that this is precisely because it's so easy to get started with, more new developers choose it

kqr• 4 years ago

I don't know that it's so sad. If it replaced humans putting their fingers in the air and then pulling an answer out of their ass, it's probably a strong improvement.

wodenokoto• 4 years ago

"if x return foo" style algorithm, not some fancy scorer/ML

It's a decision tree. Decision trees falls under ML. The thresholds are learned from data.

I would imagine they had to use a decision tree in order to tell users and case workers _why_ they are at risk.

The algorithm has been heavily criticized for putting basically all non-western immigrants into the "high risc"-group, but if the job market historically hasn't been willing to hire high performing, non-western immigrants, isn't it the whole point of building an algorithm from data? To shed light on this?

kaoD• 4 years ago

Feels like a decision tree is not the kind of algorithm that can solve this problem due to its multidimensional nature. It's very vulnerable to Simpson's paradox.

It may be right most of the time if we lump everyone in the same bag, but that is not helpful at all.

Also, having the self-evaluated unemployment risk the first deciding factor seems to defeat the purpose.

googlryas2• 4 years ago

A fixed decision tree isn't machine learning. If the tree can be updated or generated by code, then sure, but a human just writing a decision tree has nothing to do with machine learning.

tgsovlerkhgsel2• 4 years ago

The interesting thing is that at least the version on this website does NOT put non-Western immigrants into the high risk group. It puts Western immigrants (and descendants) there, and non-Western descendants, but not non-Western immigrants themselves!

larsnystrom• 4 years ago

One of the named authors in the footer also wrote an article about the STAR algorithm, which I think might provide some more context: https://www.zetland.dk/historie/sOMVZ7qG-m8aLzmJK-94d1e

hawkeye424• 4 years ago

Wonder if the algorithm is based on this

http://www.youtube.com/watch?v=YvsKIBSPfWw

movie ?

Dave3of5• 4 years ago

I'm also high risk of unemployment if I ever moved to Denmark. Oh well ...

londons_explore• 4 years ago

Soooo.... What is the use of the output of this algorithm?

If it's for the government to predict how many 'help finding employment' leaflets the government should print next month, that seems fine.

If it's to decide who is eligible for state handouts, it is unsuitable.

arghwhat2• 4 years ago

Dane here.

I know nothing about this particular "algorithm" or how it is used, other than that it was to my knowledge meant for use in job centers - i.e. it applies to someone who is already unemployed and unable to find work on their own for whatever reason (we should not judge, every life is different). In some cases, those on social benefits are also forced to job centers despite their condition, which might negatively affect this crude "algorithm" as well.

In other words, it does not represent the job market or life in Denmark as a whole. With a 2.5% unemployment rate that should be more or less obvious.

Of course our job market has a degree of ageism, sexism and racism (with the rather racist nationalist parties being popular in some regions), unfortunate as it may be. A male, white Dane in their 20s is likely going to have an easier time getting jobs than a female foreigner in their early 30s, where the language barrier, additional paperwork for work visa and non-zero risk of maternity leave (can easily be 50 weeks, which is costly in industries where the standard is full rather than partial salary) may make a small company uncomfortable.

But while I cannot speak for industries I have never worked in, industries that we are familiar with here on HN are fully internationalized and would hire a foreigner in the blink of an eye, and such a person would is unlikely to ever discover a job centers.

TL;DR: don't worry about this crappy "algorithm".

mrweasel3• 4 years ago

industries that we are familiar with here on HN are fully internationalized and would hire a foreigner in the blink of an eye

Small correction: Foreigners from within the EU. Many of the Danish software companies have contracts with e.g. banks and government organisations, that makes them nervous about taking in people from outside the EU.

It depends very much on the company. It should be easy enough to find someone willing to hire you, but many won't.

arghwhat2• 4 years ago

Many of the Danish software companies have contracts with e.g. banks and government organisations, that makes them nervous about taking in people from outside the EU.

I actually meant outside EU. I do not know of any places that would exclude foreigners due to government contracts. I see no lack of foreign hires within fintech and medical, and the usual government contract dumpsters (KMD, Netcompany, that sort) also hire foreigners. I myself work in a place that employs a lot of engineers from India on-site for example, and I know many companies like it.

Foreigners from within EU will have easier paperwork and living situation though.

recoveringeng• 4 years ago

Dane here. Disclaimer: I can't say I really know much about this, but here is a bit of background info:

STAR is acronym for Styrelsen for Arbejdsmarked og Rekrutering (English: Agency for Labor Market and Recruitment).

They have a note describing the system (in Danish) at [1]. I copy/pasted a few relevant sections to Google Translate at [2].

TL;DR: Algorithm is a decision tree based on questionnaires and outcomes of about 152000 cases. The note states a precision of 70% and a recall of 32%. There is no mention of a split between modelling and test data.

[1] https://star.dk/media/12514/2020_01_31_beskrivelse_-_profila...

[2] https://translate.google.com/?sl=da&tl=en&text=Modellen%0APr...

qwerty4561272• 4 years ago

Just a fun fact: ~90% people outside academia I ever knew had jobs unrelated to their formal education so the questions about education level and field always seemed pointless to me.

BlargMcLarg• 4 years ago

Most questions are pointless and have almost no evidence behind them. Doesn't stop hiring from believing they know better.

qwerty4561272• 4 years ago

Why isn't anybody trying to fix the system so it would be more efficient at directing people to the jobs they would be actually good and happy at? To me it seems this (using the human resources the optimal way) becomes more and more important nowadays. And I believe the formal education one has completed can only serve as a hint to what the best job for him is and not a direct definition of that.

Engage with an unemployment algorithm

Related Stories