mvdan

Indeed! This blog post is also relevant: https://cue.dev/blog/guardrailing-intuition-towards-reliable...

https://github.com/mvdan/sh/blob/master/syntax/canonical.sh is one small example.

For the first two caveats, I actually agree that we could and should handle ambiguous input. It just hasn't been a priority because doing that properly would be quite a bit of work, and such ambiguous syntax isn't particularly common. See https://github.com/mvdan/sh/issues/686 for my current thoughts on how to tackle it.

The third caveat concerns parsing `export` and `let` as keywords rather than as builtins. Like the README says, this is to properly build the syntax tree without leaving opaque strings as expressions, but also to support `declare foo=(bar)` which wouldn't work if `declare` was treated like any other builtin simple command.

How else would you have a static parser handle these two builtins? They are in a bit of an awkward middle ground between builtin and keyword. My instinct is that giving them special treatment in the parser to allow tokens like `(`, while at the same time representing them in the syntax tree with opaque strings as expressions, would be pretty underwhelming to any users of the parser.

That said, we already have that problem with `let "foo=123"` for example, where our parser currently represents the expression as the quoted string without going any deeper. https://github.com/mvdan/sh/issues/754#issuecomment-96329574... considers doing a second parse stage in the shell interpreter to fix cases like these, though always doing a second parse could get expensive.

We _could_ leave all arithmetic expressions as input strings in the parser, and do all the actual parsing when they are evaluated. That would be more compatible with Bash and more consistent. But it would also be less useful to any parser users who don't run into any of these weird edge cases, which aren't common at all, I think.

In short, I have some ideas, but I'm not sure at all what's best :) Doing a good job for 99% of users feels better than aiming for 100% compatibility with bash syntax, particularly where bash syntax is a bit weird.

gofumpt will never have formatting knobs, following gofmt's design. But if one of gofumpt's rules forbids a style which is reasonable even if it's not very popular, we might want to make the rule more conservative or remove it entirely.

You might be getting confused. gofmt does not warn about lacking comments, that was golint, which has since been deprecated.

gofmt does not divide or re-join imports into groups, but gofumpt does :)

Neither gofmt nor gofumpt enforce a character limit on lines. Some form of line length limit is the most common gofumpt feature request by far, but one that I haven't been brave enough to release in any form.

Perhaps read gofumpt's README, it has a lot more information.

I hadn't really considered that people might want to do this. From experience reading and writing Go code for ~8 years at multiple companies, the only times I've seen leading or trailing empty lines in blocks have always been either inconsistent or unintentional, and usually both.

Are there Go codebases that stick to the formatting you show, out of curiosity?

Either way, please file a bug. Perhaps others can chime in there if they also use the same style.

It's not quite as simple as that :) I think upstreaming half of gofumpt's additions to gofmt would be reasonable, but the person who wrote and maintains gofmt is Robert Griesemer, who continues to be quite busy with generics. They are still actively fixing typechecking bugs and performance issues, as far as I can see.

I think it's hard for anyone to justify pausing or distracting the generics work in favor of upstreaming parts of gofumpt. At the end of the day, gofumpt works today - it's just a bit awkward to have it as a third party tool.

That said, it is on my radar to talk to him and make a plan for upstreaming.

Edit: to clarify what I mean with the above: I of course could do the legwork to port my formatting changes to go/printer, the guts of gofmt. The reason I mention Robert is that he'd be the one to consider and approve each formatting change, and review the code changes and tests. That work is trickier than it sounds, because you have to think about the possible effect any formatting change would have on all kinds of existing Go code out there.

I agree that empty lines can help. I've been careful to only remove empty lines which, in my opinion, don't help when structuring the code or making it more readable.

If you have examples where gofumpt is being a bit too aggressive, I'd like to hear about them. I've corrected, and even removed, some of gofumpt's rules in the past thanks to user input.

I'm the author of that presentation, and I'm surprised to see the very similar content and structure, to say the least. I left a comment on the Reddit thread[0] to see if I'm missing something.

[0] https://www.reddit.com/r/golang/comments/ginp22/whats_coming...

It might lead to fewer allocations in complex programs, as the compiler is able to put more in the stack under some edge cases.

Simple Go binaries do tend to be statically linked, which is where I think the confusion comes from.

The most common source of dynamic linking is cgo, since CGO_ENABLED=1 is the default when not cross-compiling. For example, importing os/user will dynamically link against libc, unless you disable cgo or force statically linking with libc.

See https://github.com/golang/go/issues/26492 for more details. I think the purpose of the flag will be to force static linking in all edge cases, while not disabling cgo explicitly.

These are the slides from a talk; I didn't spend the extra time to change the format or write down what I explained.

After all, I didn't post them here, someone else did :)

I personally think it's best to just follow the links if the title sounds interesting to you.

See https://github.com/golang/go/wiki/Go-Release-Cycle. We're one month into the three-month freeze, so it's unlikely that new features are going to be merged in before the tree reopens.

Bugfixes are still welcome, though, as long as they're safe enough for 1.13.

If anyone wants the recording, this is the timestamp in the meetup recording before an edited version is available: https://youtu.be/RGeHsqj2RWo?t=2h16m11s

I can see that the slides alone aren't self-explanatory in some points, for example with the random byte reading or gofmt. I'll make sure to link to the issues/CLs in the slides next time :)

I'm going to go on assuming that you're not being sarcastic :)

As I mentioned in another comment, once Go's wasm support is shipped and stable, I hope to make the JS package smaller and better. Until then, this is the best I can do without manually rewriting the Go code in JS.

That's a good point. I am adding more and more examples to the Go documentation these days.

Ideally I'd just point the JS people at the Go docs and examples, but the translation is not exactly one-to-one. This is why the README file published with the JS package has a complete example. I'll try to add a few more.

It sounds like I should use streams then. After all, that's what the Go API uses, Readers and Writers. Those simply pass chunks of bytes around.

Thanks again for the help!

Yep, I've exchanged a few emails with Andy before, the oilshell author. I started the parser in early 2016, so we were both getting started around the same time.

If you look at my README's caveats section, you'll see a very similar story - only accepting programs that can be parsed statically in a simple way.

Most Go code out there is under permissive licenses, I assume because static linking is the norm.

I hope I don't get an angry email from RMS at some point :)

Point taken; yes, when I said they were still different, I meant the simplicity of the tool and how its output is higher level than machine code.

But as you all say, this is just a spectrum. Perhaps the future generation will call JavaScript a low-level language :)

Is there a more appropriate word for source-to-source transformation? I understand that compilers aren't that different, but generating machine code is quite different.

I'm not sure I understand what you're saying. If there was a Go->Bash transpiler (or a JS->Bash one), yes, we would theoretically end up with a Bash parser and interpreter in Bash.

I imagine it would be quite the monster, though.

I hope noone notices that most of my code was written by a Markov chain.

This is hacky, but it beats having to write a shell parser from scratch again :)

Once wasm support is shipped with Go, I hope to be able to accomplish the same without a transpiler. I'd also hope that the final package would be smaller and more performant.

Thanks for the suggestions! I'll need to read up on these js/node features.

One reason that the API is a bit clunky is that it tries to mimic the Go API closely, so that the documentation and examples are reusable.

However, your points on the string returns are very valid. I replaced all of Go's readers and writers (byte streams) with strings, simply because I didn't know of a better way.

It definitely sounds like the features you suggested would be better. If the transpiler (gopherjs) supports them, I'll definitely give them a try.

I'm not very familiar with JavaScript and npm - looking for suggestions on how to make the package easier to use.

Factmata (http://factmata.com/) | Full Stack NLP Engineer | 3-month Contract with potential for 30-40% co-founder equity | London, REMOTE

We are Factmata, a machine intelligence and AI startup for automated political fact checking. We were just funded by Google and its Digital News Initiative to build a prototype of our NLP research (https://www.theguardian.com/media/2016/nov/17/fake-news-goog...)

Our team consists of 3 NLP researchers, entrepreneurs and developers. 2 of our team (Andreas Vlachos (http://andreasvlachos.github.io/) and Sebastian Riedel (http://www.riedelcastro.org/)) are distinguished academics in the NLP space and have contributed in excess of 50+ papers combined in the field of machine learning and natural language processing, especially claim detection and fact checking.

The position will entail implementing a usable version of our existing research as a product for people to use (Vlachos and Riedel, EMNLP 2015, "Identification and Verification of Simple Claims about Statistical Properties), enhance the NLP behind it, and build a full big data, information extraction and semantic parsing pipeline around it.

Apply at https://angel.co/factmata/jobs/191410-full-stack-nlp-enginee... or, Contact dhruv.ghulati@factmata.com with your CV and Github repo

This might be relevant: https://github.com/mvdan/interfacer/issues/19

And thanks for the link! That should be useful to make a decision on what to do in that issue.

To add to what jerf already said - This is a linter, so it should be taken with a grain of salt.

Having said that though, I'm trying really hard to make it conservative and not have any false positives. I always discard functions that may be implementing interface methods or declared function types, for example.

Is this what you meant by a contract? If so, interfacer would skip the function, even if nowhere in the code is the function used to implement the interface. It's just lazy and assumes a conservative position.

If you mean it in a broader way that might not involve any interface or named function type, then we need a way for the developer to somehow express in the code that they really want that specific type. For example, I made it so that mentioned parameter types are ignored:

  func ProcessInputFile(f *os.File) error {
      // use as an io.Reader
  }

I really want to make this tool as conservative and less noisy as possible, so if you have any ideas or a false positive example they would be very welcome.