Bugs in the YAML specification

http://pyyaml.org/wiki/BugsInTheYAMLSpecification

by networked • 10 years ago

102 77 10 years ago

lwis• 10 years ago

I've always felt that YAML was incomplete.

TillE• 10 years ago

If you want to use it for complex things, sure, there's no end of potential features. If you need some logic with your data, I like Lua.

But I think most uses of YAML are basically just a much more flexible JSON.

AstroJetson2• 10 years ago

     If you want to use it for complex things, sure, there's no end of potential features. If you need some logic with your data, I like Lua.

Say some more about this. I look at YAML as a way to do data exchange, like using JSON/SON. How does Lua fit into this?

andybak2• 10 years ago

I use YAML as human-editable JSON - especially for non-programmers (although it's surely more pleasant to write even for programmers).

I would like a clearly documented YAML subset that left out some of the more complex stuff and avoided a few 'more than one way to do it' features. That would go a long way to removing some of the criticism of YAML with very little cost in functionality.

crdoconnor10• 10 years ago

Despite being a nice readable language if you use it right, the spec is rife with disgusting features and nasty surprises.

This was my attempt to cut out 90% of the crap, leaving the nice, readable core (which maps on to JSON pretty neatly):

https://github.com/crdoconnor/strictyaml

jean-• 10 years ago

Very nice! Another saner alternative to YAML which I like is TOML: https://github.com/toml-lang/toml

voaie• 10 years ago

TOML don't even have a clear and definitive standard/test suits. It seems unmaintained but some people are trying to push it to 1.0 no matter what.

crdoconnor10• 10 years ago

I actually went over why I'd consider this is a better format than TOML:

https://github.com/crdoconnor/strictyaml/blob/master/FAQ.rst...

akkartik• 10 years ago

I love the example at https://github.com/crdoconnor/strictyaml/blob/master/FAQ.rst.... Convinced in 2 seconds that full YAML is idiotic.

guitarbill• 10 years ago

this whole idea of strictyaml just makes so much sense to me. plus having comments is a huge bonus. why am i only hearing about this now?

MasonOfWords• 10 years ago

The FUD here is a bit ridiculous.

Writing movie scripts in YAML using unquoted strings? That's pretty contrived. Using literal style is easy when it is potentially needed (e.g. programmatic output), and any decent editor can highlight inferred types in helpful ways. I've used YAML in a variety of contexts and never been bitten by this one, and I don't think that any of his examples are still problems in YAML 1.2 (from 2009).

The Ruby security problem they reference is also absurdly misattributed. The problem there is with trusting serialized data to mark its own types, and having no limits on what types can be deserialized into. That's a depressingly common security problem in many web frameworks, and YAML as an interchange format isn't a unique source of vulnerability. Any data format is dangerous on the web if you trust it to create arbitrary types.

sirn• 10 years ago

I read that and thought, hey, it couldn't be that bad, right? Then proceed to type:

    irb(main):002:0> YAML.load("on: yes")
    => {true=>true}

Oh.

jgalt212• 10 years ago

If you stick the language agnostic core, yaml is great.

We use it for:

1. test case specifications we use for unit tests suits that must run across multiple languages.

2. DSL for business processes

ThePhysicist• 10 years ago

What I find more interesting is that YAML allows you to define self-referential / circular data structures:

    foo: &foo
      bar: *foo

In PyYAML, this will give you a self-referntial dictionary. Powerful but pretty catastrophic if you use (naive) recursion to analyze a user-submitted data structure.

twelvechairs2• 10 years ago

Feature not a bug. YAML is what people turn to who need these kind of advanced feautures which arent provided in json or similar.

OskarS2• 10 years ago

I would imagine this feature is primarily useful if you want to serialize a whole bunch of objects that all reference each other, but it makes me feel a bit icky. It feels like it breaks an intuitive assumption you have about hierarchical formats, which is that there should be no cycles.

Maybe that's just my personal bias, but I feel like the relative simplicity of JSON is a strong feature in its favor. As a developer, I have very clear understanding of the data I'm reading, and with that I can more easily make safer and more stable code.

crdoconnor10• 10 years ago

YAML is what people turn to who need these kind of advanced feautures

Most people don't use these features.

techdragon• 10 years ago

In my experience, YAML is what people turn to when they want something like JSON, but don't want to have to type the extra braces and quotation marks.

ChoHag• 10 years ago

If you're doing naive recursion, you're doing it wrong.

jstimpfle4• 10 years ago

This is an opportunity to plug my private project "WSL" [1] which is a clean text serialization format for relational databases. The scope is somewhat different and it's not really released (but beginning to stabilize), but I'll be happy to hear what you think.

[1] http://jstimpfle.de/projects/wsl/main.html

nerdponx• 10 years ago

This is interesting! You should make a separate thread about it.

kiwidrew• 10 years ago

Interesting -- I like the concept.

But it's completely unusable for the Real World (tm) because strings cannot contain the '[' and ']' characters AND YET there is no mechanism for escape sequences. What if my data legitimately contains those two characters???

jstimpfle4• 10 years ago

Thanks for reading and the feedback!

That's just the default string type. I started out with escaping but noticed it's a lot of complexity that is rarely needed (not for my own use case, which is accounting, inventory, and some web apps which don't need it).

The advantage of not having escaping is easier seds and greps which don't miss the field boundaries.

The important concept though is that arbitrary datatypes elements can be added by the user of an API implementation (the python library already offers that). The datatypes define their lexical syntax, like in perl6. I will also declare more "default" datatypes and might include a C-like string after enough consideration.

eyelidlessness• 10 years ago

I have minimal experience with YAML (100% of which configuring for CI environments because apparently that's what all the modern ones prefer). I don't really know the ins and outs of the format, and try to stick to extremely simple representations.

Today I learned that what I perceived as somewhat complicated syntax is actually overwhelmingly complex.

I know that my preference for s-expressions is not shared by everyone, but the more complex a syntax I encounter, the more I wonder if simpler alternatives were even considered.

Genuine question: apart from inertia, and apart from recursive references (noted in comments before I wrote this), is there a use case for YAML that isn't solved by simpler and less ambiguous formats like EDN or Transit?

jaredklewis• 10 years ago

Just one data point, but I had never heard of transit or edn before today. Both look awesome.

On the other hand I picked up YAML in less than a day (many years ago) and despite its complexity can't remember that ever causing me any issues. Not 100% sure if missing from your suggested formats (which do support multi line strings) but I enjoy use of yaml's folded strings.

fibo• 10 years ago

YAML was the first human readable/writable format, so better t'han XML and JSON for us humans. Also is a superset of JSON.

Someone• 10 years ago

Human writable? I have little experience with it, but if what Amazon uses for its CodeDeploy configuration files is representative for the format, I disagree.

IMO, a human writable format shouldn't need a help page like this: http://docs.aws.amazon.com/codedeploy/latest/userguide/app-s...

Yes, part of that is CodeDeploy, but saying "tab characters must not be used in indentation" for me, disqualifies a standard as "human writable".

CodeDeploy makes that horrendous by requiring five or so clicks to figure out what causes an error (try forgetting adding an app spec.yml file or saving a file with a BOM)

stubish• 10 years ago

I think all the cited bugs I'd also fail in a code review. YAML is great, provided you pretend most of the spec doesn't exist and treat it as JSON with some syntactic sugar. I don't ever want to see !! even in output for machines.

The big problem is when the useless features that you should, never, ever use (and if you do you are doing it wrong) bite you on the arse. Because it tried to do too much, like XML, even when we knew that was a bad idea. But is is still the best 'JSON with better layout' out there, which is what we want when we want readable documents and maintainable code. EDN, Transit and TOML all fail in that somewhere, IMO, and of course popularity is actually a hugely important feature for any data interchange format.

mst3• 10 years ago

YAML is designed for machines to write, and only incidentally for humans to read.

If you want a configuration format, ingy and I designed JSONY for that.

burke2• 10 years ago

Piling on to the general theme of the rest of the comments here:

I really wish there was a more popular middle-ground between YAML and JSON. People use YAML because it's the next step up from JSON if you need comments, etc., but I think most purposes would be better served by the likes of JSON5, HJSON, or TOML (for example) if only any of those were as popular as JSON and YAML.

I implemented a YAML parser from spec last month. It (YAML) goes to great lengths to provide human-friendly features, trading off computer-friendliness to an fairly extreme extent.

Eliminating 'plain' scalars (unquoted strings-as-values), folded multiline literals, tags, anchors/aliases, and possibly directives, as a sort of reduced yaml would make the language a lot less silly for the kinds of things a lot of people end up using it for.

crdoconnor10• 10 years ago

JSON5, HJSON and TOML are not as readable or editable as YAML.

They may have a slight advantage over the default spec of YAML but they don't stack up well against YAML with the features you mentioned removed.

I really don't think an entirely new standard is warranted.

burke2• 10 years ago

Right. My hypothesis here is that, if there were, instead of JSON and YAML to choose from, three standards, one of which were something slightly less human-friendly but much simpler than YAML, I think it would be widely adopted.

I think YAML is more widely-used than it would otherwise be, in a broader variety of domains, simply because the only popular alternative doesn't have comments.

I'm not drawing from this any sort of conclusion that we should try to push one of these alternatives as viable, just lamenting the state of things.

mst3• 10 years ago

ingy and I already invented that. It's called JSONY.

Bugs in the YAML specification

Related Stories