YAML: The Norway Problem (2022)

https://www.bram.us/2022/01/11/yaml-the-norway-problem/

222 164 1 year ago

This problem occurs because pyyaml load() uses the full YAML 1.1 schema. There is another function BaseLoader that will interpret everything as a string which is the workaround that the article suggests. Just another way to achieve it.

It’s a bit of a sore spot in the YAML community as to why PyYAML can’t / won’t support YAML 1.2. It was in maintenance mode for a while. YAML 1.2 also introduced breaking changes.

From a SO comment: “ As long as you're okay with the YAML 1.1 standard, PyYAML is still perfectly fine, secure, etc. If you want to support the YAML 1.2 spec (released in 2009), you can use ruamel.yaml, which started out as a fork of PyYAML. – CrazyChucky Commented Mar 26, 2023 at 20:51”

- https://stackoverflow.com/q/75850232

rat872• 1 year ago

Yeah it's a problem I had to put up a PR on a tool I was using because I ran into the Norway problem on yaml I was getting from another team. I did ask them to add quotes just in case

firesteelrain2• 1 year ago

A supplier we contracted with and we gave requirements to asked me what format do we want the export/import of the data to be in and I said JSON. It’s simple, easy and can be converted into anything else very easily

gschizas• 1 year ago

I wish that ruamel.yaml had better documentation. I've had to dive into the code so many times to find out how to do something.

gnabgib• 1 year ago

The YAML document from hell (566 points, 2023, 353 comments) https://news.ycombinator.com/item?id=34351503

That's a Lot of YAML (429 points, 2023, 478 comments) https://news.ycombinator.com/item?id=37687060

No YAML (Same as above) (152 points, 2021, 149 comments) https://news.ycombinator.com/item?id=29019361

mdaniel10• 1 year ago

And some light commentary a few days ago: https://news.ycombinator.com/item?id=43648263 - Apr 2025 (51 comments)

dissent3• 1 year ago

I reckon if this is really a big concern for anybody, then they are probably writing way too much YAML to begin with. If you're being caught out by things like this and need to debug it, then it maps very cleanly to types in most high level languages and you can generate your YAML from that instead.

makeitdouble4• 1 year ago

Sadly you usually realize you've been writing too much YAML way past the turning point, and it will be a pain to move a single file to JSON for instance when you have a whole process and system that otherwise ingest YAML, including keeping track of why this specific part of JSON and not YAML.

So people work around the little paper cuts, while still hitting the traps from time to time as they forget them.

generate YAML

I've a hard time finding a situation where I'd want to do that. Usually YAML is chosen for human readability, but here we're already in a higher level language first. JSON sounds a more appropriate target most of the time ?

dissent3• 1 year ago

There are probably two use cases.

Configuration files for programs. These tend to be short.

DSLs which are large manifests for things like cloud infrastructure. These tend to be long, they grow over time.

My pet hypothesis is these DSLs exist mostly for neutrality - the vendor can't assume you have Python or something present. But as a user, you can assume just that and gain a lot by authoring in a proper language and generating YAML.

See https://github.com/cloudtools/troposphere for a great example for AWS CloudFormation.

charrondev3• 1 year ago

Isn’t yaml a strict superset of JSON? Any compliant YAML parser should be able to ingest a JSON document.

dev_l1x_be• 1 year ago

True. YAML is an intermediate representation between my intention expressed in Dhall and what runs in production.

https://github.com/dhall-lang/dhall-kubernetes

ashishb2• 1 year ago

How often do people even encounter this issue? I have been using YAML for 5+ years and have never had it before. Further, I use `yamllint` which points this out as a lint issue "truthy value should be one of [false, true]".

rat872• 1 year ago

I have when getting an openapi yaml file from someone else.

mongol• 1 year ago

Has been encountered where I work. A global website with lots of country-specific config.

hinkley3• 1 year ago

Fractions are discriminatory when they happen to one individual or group every time or even just the first time.

See also p95 but the same couple of users always see the p99 time, due to some bug.

ashishb2• 1 year ago

Indeed, based on the comments, it is a scissor-bug. Most people never encountered it while some encountered it a lot.

Y-bar• 1 year ago

Never experienced it for the past 10+ years since the bug was fixed in the spec.

speedgoose• 1 year ago

I have encountered it once, though I live in Norway and worked in IT there for a decade.

tetha2• 1 year ago

I don't recall encountering the norway problem in the wild.

Ansible has a pretty common issue with file permissions, because pretty much every numeric representation of a file mode is a valid number in YAML - and most of them are not what you want.

Sure, we can open up a whole 'nother can of worms if we should be programming infrastructure provisioning in YAML, but it's what we have. Chef with Ruby had much more severe issues once people started to abuse it.

Plus, ansible-lint flags that reliably.

peanut-walrus• 1 year ago

I for one did encounter exactly this problem when configuring a list of countries via ansible for geoip whitelisting.

jeltz2• 1 year ago

I have seen it twice but I work in Sweden where we often do things also for the Norwegian market.

Y_Y• 1 year ago

I don't think false is truthy.

thyrsus• 1 year ago

I do a lot of ansible which needs to run on multiple versions, and their yaml typing are not consistent - whenever I have a variable in a logic statement, I nearly always need to apply the "| bool" filter.

polski-g• 1 year ago

Yep. I just want strict yaml:

anything encased in quotes is a string, anything not is not a string (bool, int or float)

awestroke• 1 year ago

Including keys?

mdaniel10• 1 year ago

This is likely hair splitting, but you are far more likely getting bitten by the monster amount of variance in jinja2 versions/behaviors than by anything "yaml-y"

For example, yaml does not care about this whatsoever

  - name: skip on Tuesdays
    when: ansible_date_time.weekday != "Tuesday"

but different ansible versions are pretty yolo about whether one needs to additionally wrap those fields in jinja2 mustaches

  - name: skip on Tuesdays
    when: '{{ ansible_date_time.weekday != "Tuesday" }}'

and another common bug is when the user tries to pass in a boolean via "-e" because those are coerced into string key-value pairs as in

  $ ansible -e not_today=true -m debug -a var=not_today all
  localhost | SUCCESS => {
    "not_today": "true"
  }

but if one uses the jinja/python compatible flavor, it does the thing

  $ ansible -e not_today=True -m debug -a var=not_today all
  localhost | SUCCESS => {
    "not_today": true
  }

It may be more work than you care for, since sprinkling rampant |bool likely doesn't actively hurt anything, but the |type_debug filter[1] can help if it's behaving mysteriously

1: https://docs.ansible.com/ansible/11/collections/ansible/buil...

quechimba2• 1 year ago

We had this issue many years ago when people from Norway couldn't sign up. Took us a while to figure out

duxup• 1 year ago

Or were they from Noway ...

magicalhippo• 1 year ago

As a Norwegian I'm very curious, where in the pipeline were you using YAML? And why?

I've only seen it used for configuration.

tough• 1 year ago

usually locale's paths gone wrong

StableAlkyne• 1 year ago

I've seen teams use it as a replacement for JSON because it has the perception of being more "modern"

quechimba2• 1 year ago

We were using a fork of https://github.com/carmen-ruby/carmen/tree/master/iso_data/b... with our own translations. We used the data in the signup form.

TZubiri2• 1 year ago

I usually think of yaml for internal config files, would never think of yaml for user data.

Don't ask me why though, might have something to do with how it's written like a python file, no user would want to write their data in yaml format.

nurgasemetey• 1 year ago

Probably, OP didn't keep user data in YAML, but I think there was config that kept allowed countries to sign up.

dmckeon• 1 year ago

Narrow escape for people from Yemen (YE).

singpolyma3• 1 year ago

Quote your strings

pavel_lishin• 1 year ago

Empty the footgun before firing.

umanwizard4• 1 year ago

“Be liberal in what you accept” rears its ugly head once more.

eyelidlessness• 1 year ago

Being liberal in what you accept, also known as the “robustness principle”, doesn’t mean being ambiguous or surprising about how you accept it. If anything, robustness requires a great deal more precision and clarity (at least with your own reasoning, then with how you communicate what to expect from it).

kazinator6• 1 year ago

Postel's Law does not deserve to be nicknamed the Robustness Principle.

Robustness has a meaning and it refers to handling bad inputs gracefully. An example of a lack of robustness is allowing a malicious actor to execute arbitrary code by supplying a datum larger than some buffer limit.

Trying to make sense of invalid inputs and do something with them isn't robustness. It's just example of making an extension to a spec. The extension could be robust or not.

Postel's Law amounts to "have extensions and hacks to handle incorrectly formatted data, rather than rejecting them. So, OK, yes, that entails being robust to certain bad inputs that are outside of the spec, but which land into onto one of the extensions. It doesn't entail being robust to inputs that fall outside of the core spec and all hacks/extensions.

Cherry picking certain bad inputs and giving them a meaning isn't, by itself, bona fide robustness; robustness means handling all bad inputs without crashing or allowing security to be compromised.

inopinatus4• 1 year ago

This has little to do with the robustness principle, however mis-stated. It's just shitty design. But if someone was still hell-bent on invoking it, then if anything, it's a straight-up violation of the adjacent words "be conservative in what you do"¹, and further disregards the commentary in RFC1122²:

    ... assume that the network is
    filled with malevolent entities that will send in packets
    designed to have the worst possible effect ...

[1] https://datatracker.ietf.org/doc/html/rfc761#section-2.10

[2] https://datatracker.ietf.org/doc/html/rfc1122#page-12

senderista2• 1 year ago

That works only when everyone is trying in good faith to follow the standard, i.e. basically never. My version of Postel's Law:

If you accept crap, then eventually you will receive only crap.

kazinator6• 1 year ago

In Lisp, if you want to read text into symbols (e.g. file of words), you just switch to a dedicated package in which those symbols are interned. Then if NIL happens to come up, it will be a symbol named "NIL" in that package, unrelated to the special object.

TZubiri2• 1 year ago

That edge case sounds like a reasonable tradeoff you would make for such a simple and readable generic data format.

Escaped json probably hits that sweetspot by being a bit uglier than yaml, but 100 times simpler than xml, though.

tetha2• 1 year ago

Mh, since I just commented about ansible, you just made XML-based ansible flash in front of my eyes. I think I'm in a bit of pain now.

    <tasks>
        <ansible.builtin.copy notify="restart minio">
            <src> files/minio.service </src>
            <dest> /etc/systemd/system/minio.service </dest>
            <owner> root </owner>
            <group> root </group>
            <mode> 0x644 </mode>
        </ansible.builtin.copy>
    </tasks>

But you could use XSLT to generate documentation in XHTML from your playbooks about what files are deployed, what services are managed...

mdaniel10• 1 year ago

I will die on the hill than writing out ansible.builtin. are characters of my life I'll never get back, and refuse to. If it's built in why do I have to qualify it?!

Also, watch out: 0x644 != 0644 which is the mode you meant

thund• 1 year ago

I like using tags and avoid any doubt

!!boolean

https://dev.to/kalkwst/a-gentle-introduction-to-the-yaml-for...

nnurmanov• 1 year ago

Another solution is to change the country name:)

gunalx3• 1 year ago

No way.

rwoerz• 1 year ago

Neitherway

hinkley3• 1 year ago

Or in New Zealand: Nor way.

gunalx3• 1 year ago

Thoug we have renamed amino acidsvi think it was. Because microsoft excel switched the original names to months.

madcaptenor• 1 year ago

Genes, not amino acids.

https://www.theverge.com/2020/8/6/21355674/human-genes-renam...

riffraff• 1 year ago

Usual reminder that this is not a problem in YAML 1.2 released 15 years ago.

Sadly many libraries still don't support it.

lifthrasiir• 1 year ago

This effectively means that a new version of specification didn't solve the problem at all.

normie3000• 1 year ago

Google App Engine used to do this to environment variables defined in YAML. IIRC it would convert the string "true" to "Yes", which was a fun surprise when deploying Java And NodeJS apps.

raffraffraff• 1 year ago

Why not just use quotes all the time for strings?

mystifyingpoi2• 1 year ago

I like that in concept, but 1) literally no one does that (prime example - Kubernetes docs) and 2) it looks much more messy with quotes, when you know that they are unnecessary in 95% of cases.

zelphirkalt2• 1 year ago

Oh, I did that in Ansible stuff. Using quotes for all strings. Exactly because I know what a mess YAML is.

kinow2• 1 year ago

I guess sometikes it is out of your control. I work on a workflow manager where users specify their workflows with YAML. So there's little we can do to prevent them from writing things like no, n, t in a place it could cause some issue like ij the article.

zelphirkalt2• 1 year ago

Ah, the many places that choose to use YAML for no good reason...

kergonath• 1 year ago

Because that’s annoying. YAML is often written and read by humans. If you want a verbose and more regular way to do it, there is always JSON. But JSON is really annoying to deal with for humans, although it is much better than YAML for several applications.

hnlmorg4• 1 year ago

You don’t actually need quotes to define a string in YAML. Eg the following syntax

   User:
     Name: >-
       Bob
     Phone: >-
       01234 56789
     Description:>-
       This is a
       multi line
       description

That’s both readable and parses your records as strings.

Edit: This stack overflow like provides more details https://stackoverflow.com/questions/3790454/how-do-i-break-a...

stephenr• 1 year ago

It's not a coincidence that YAML is a perfect acronym for "yet another migraine looming".

I mean ok it is technically a coincidence but it definitely feels like the direct result of the "what could possibly go wrong" approach the spec writers apparently took

praptak• 1 year ago

Related: the YAML exponent problem[0]

TLDR: unquoted hex hash in YAML is fine until it happens to match \d+E\d+ when it gets interpreted as a float in scientific notation.

[0]https://www.brautaset.org/posts/yaml-exponent-problem.html

ajuc• 1 year ago

YAML is just doing too much and trying to be too clever.

xelxebar3• 1 year ago

This has been fixed since 2009 with YAML 1.2. The problem is that everyone uses libyaml (_e.g._ PyYAML _etc._) which is stuck on 1.1 for reasons.

The 1.2 spec just treats all scalar types as opaque strings, along with a configurable mechanism[0] for auto-converting non-quoted scalars if you so please.

As such, I really don't quite grok why upstream libraries haven't moved to YAML 1.2. Would love to hear details from anyone with more info.

[0] https://yaml.org/spec/1.2.2/#chapter-10-recommended-schemas

xigoi9• 1 year ago

I’m sad that the “fix” was to disallow “no” as a more readable alternative to “false”, rather than to disallow unquoted strings.

xelxebar3• 1 year ago

The fix is to make conversion user-controllable. If you want to disallow bare scalars except for booleans and numbers or whatever, it's just a little bit of configuration away.

mckn1ght2• 1 year ago

It’s silly to have so many keyword synonyms as specified in that earlier regex. I’m also glad we can’t specify numeric literals as roman numerals. KISS

pydry• 1 year ago

Yeah, you still get the same issue that 3 is an integer, 3.3 is a float and 3.3.3 is a string.

heavenlyblue• 1 year ago

Why do you need an alternative spelling of false?

maxloh2• 1 year ago

Why wasn't that a major version bump, like YAML 2.0?

That sounds like a breaking change that rendered old YAML documents to be parsed differently.

transfire• 1 year ago

Absolutely correct! Please correct me if I am wrong, but as far as I know, no one has implemented YAML completely according to spec.

The tag schema used is supposed to be modifiable folks!

And why anyone would still be using 1.1 at this point is just forehead palming foolishness.

xelxebar3• 1 year ago

AFAIK, libfyaml[0] (not to be confused with libyaml) passes the 1.2 test suite, and if I remember correctly, it's currently the only YAML loader with that claim at the moment.

The yaml.orf website also lists a bunch of implementations, with about 1/3 supporting 1.2. I'm guessing that the users of those libraries just happily hum along and we never hear from them!

The issue is that downstream consumers of popular languages with a vocal community here on HN tend to just pull in libyaml, PyYAML being the major offender in my mind.

[0] https://github.com/pantoniou/libfyaml

anvandare• 1 year ago

"The limits of my keyboard mean the limits of my programming language."

If only they had had ⊥ and ⊤ somewhere on their keys to work with Booleans directly while designing the languages. In another branch of history, perchance.[1]

[1] https://en.wikipedia.org/wiki/APL_(programming_language)#/me...

tossandthrow• 1 year ago

⊥ and ⊤ is not entirely congruent to false and true.

Boolean and propositional logic is not the same.

Q6T46nT668w6i3m• 1 year ago

For _ordinary_ two‑valued classical propositional logic, e.g., YAML, they are congruent.

rusk• 1 year ago

I have an emacs macro for this

hgomersall• 1 year ago

IMO the proposed solution of StrictYAML + schema is the right one here and what we use extensively for human readable configs. StrictYAML (linked to in the post) is essentially a string-type-only restriction of YAML, so you impose your type coercion on the parsed data structure.

vander_elst• 1 year ago

If you have a schema, why not using directly something like protobufs?

LelouBil• 1 year ago

The comment you replied to talks about human readable configs

weinzierl3• 1 year ago

Perl has a Poland Problem. The customary file extension for Perl files is *.pl. This worked well until Apache introduced content negotiation and the convention to add a language code as file extension. It had index.html.en, index.html.de, for example.

index.html.pl is where the problem started and the reason why the officially recommended file extension for Perl files used to be (still is?) *.plx.

I don't have the Camel book at hand, but Randal Schwartz's Learning Perl 5th edition says:

"Perl doesn't require any special kind of filename or extension, and it's better not to use an extension at all. But some systems may require an extension like plx (meaning PerL eXecutable); see your system's release notes for more information."

dtech2• 1 year ago

That sounds more like an Apache problem than a Perl problem. It's their mistake and it's not even relevant outside Apache context

weinzierl3• 1 year ago

It should have been an Apache problem, yes. Not only did it turn out that at least the language negotiation part of content negotiation wasn't the best idea but the way Apache handled it was problematic apart from the pl problem. In the end the Perl community took the issue upon them, so historically I'd say it was a Perl problem (of choice).

maxloh2• 1 year ago

That should be marked as a breaking change on Apache side IMO. It would be a security nightmare if server code were leaked to public.

ginko• 1 year ago

Also, Prolog has the Perl problem. :)

weinzierl3• 1 year ago

Yes, I think Prolog had pl even before Perl came around, but Perl snatched it away,

whacko_quacko3• 1 year ago

Pandas has a Nigeria problem, where NA -> NaN.

It's not that bad, because you can explicitly turn that behavior off, but ask me how I know =(

trueismywork• 1 year ago

How?

whacko_quacko3• 1 year ago

For example when reading a CSV. Try using `read_csv` on a file that contains two letter country codes including NA.

orangewindies• 1 year ago

That's a Namibia problem, Nigeria is NG.

whacko_quacko3• 1 year ago

Damn, you're right ^^' Thanks for pointing that out

pkkm• 1 year ago

Programming with string templates, in a highly complex and footgun-rich markup language, is one of the things I find most offputting about the DevOps ecosystem.

sph2• 1 year ago

I believe Satan itself decided to mix YAML, Jinja and Turing-complete logic when it created Ansible. It truly is the sendmail of the modern era.

senderista2• 1 year ago

Several years ago when I was writing a deployment system for a cloud distributed database, I tried to automate everything with Ansible playbooks and the Ansible "API" (LOL). I pretty quickly gave up on implementing anything but the most trivial logic in templated YAML and switched to Python (wrapping maximally-dumb Ansible playbooks) for everything nontrivial.

Fizzadar2• 1 year ago

YAML: The Norway Problem (2022)

Related Stories