jq 1.7

https://github.com/jqlang/jq/releases/tag/jq-1.7

by wwader • 3 years ago

434 205 3 years ago

Woohoo!! Finally!

It's really awesome how the community pulled together and helped us recruit new maintainers to revive the project. Special thanks to, well, all involved, but especially @stedolan, @itchyny, and @owenthereal (all GitHub usernames).

saziba• 3 years ago

I can't stress enough how I love jq. It saved my life countless times as data engineer. Good job guys!

mtmail• 3 years ago

Related last month "First release of jq in 5 years" https://news.ycombinator.com/item?id=36951830

Huge fan, I use it all the time.

nikolay2• 3 years ago

I love jq, but I also use JMESPath (especially with AWS CLI), yq (bundled with tomlq and xq as well), and dasel [2]. I also wish hclq [3] wasn't so dead!

[0] https://jmespath.org/

[1] https://kislyuk.github.io/yq/

[2] https://github.com/TomWright/dasel

[3] https://hclq.sh/

rmwaite• 3 years ago

I really like the JMESPath interactive tutorial page (https://jmespath.org/tutorial.html). It helped me when I was first learning the syntax and I still go to it if I run into a particularly weird syntax that throws me off.

js22• 3 years ago

Make JSON greppable!

https://github.com/tomnomnom/gron

I've been using `jq` for years and I'm always able to cobble together what I need, but I have yet to find it intuitive and I'm rarely able to arrive at a solution of any complexity without spending a lot of time reading its documentation. I wish I found it easier to use. :-(

lathiat2• 3 years ago

I also love gron, if nothing else to find the paths I need to use with jq later.

But ChatGPT has genuinely solved my suffering writing jq, it does a pretty good job. It even almost replaces gron, if you feed it an exmaple json and ask for jq, it gives you something. It usually needs a little adjusting but it gets me 90% of the way there and saves me a bit of time.

I rarely use it for much else but its a jq winner :)

akoboldfrying2• 3 years ago

Looks handy, but I'd rather go the other way and extend grep (and diff, etc.) to also work on things that aren't restricted to be lines in a text file. The number of times I've needed to go through contortions for things that should be easy solved problems (e.g., grepping for patterns of line pairs, grepping for records that use a different delimiter when the record data itself could contain linefeed, etc.)...

cryptonector22• 3 years ago

You can trivially approximate gron with jq, and you there are jq implementations of gron too, like https://gist.github.com/mkfmnn/7b637aa0ec9b0422b3b75ec181a13...

E.g.,

  # gron
  gron "https://api.github.com/repos/tomnomnom/gron/commits?per_page=1" |
  fgrep commit.author
  json[0].commit.author = {};
  json[0].commit.author.date = "2016-07-02T10:51:21Z";
  json[0].commit.author.email = "mail@tomnomnom.com";
  json[0].commit.author.name = "Tom Hudson";

  # jq
  curl -L "https://api.github.com/repos/tomnomnom/gron/commits?per_page=1" |
  jq -r 'paths(scalars) as $p | getpath($p) | "\($p|join(".")|select(contains("commit.author"))) = \(.)"'
  0.commit.author.name = Tom Hudson
  0.commit.author.email = mail@tomnomnom.com
  0.commit.author.date = 2022-04-13T14:23:37Z

  # jq with grep outside jq
  curl -L "https://api.github.com/repos/tomnomnom/gron/commits?per_page=1" |
  jq -r 'paths(scalars) as $p | getpath($p) | "\($p|join(".")) = \(.)"' |
  fgrep commit.author
  0.commit.author.name = Tom Hudson
  0.commit.author.email = mail@tomnomnom.com
  0.commit.author.date = 2022-04-13T14:23:37Z

With just a bit more work you can get it to output valid gron, and even to parse valid gron.

AndyKluger• 3 years ago

A couple more alternatives:

https://github.com/kellyjonbrazil/jello

https://github.com/wwkimball/yamlpath

commotionfever• 3 years ago

on a similar note I wrote little tool to convert from many formats to many formats

the biggest usecase for me is taking some csv, toml, xml, whatever and converting that to json so I can pipe to jq

https://github.com/sentriz/rsl

cryptonector22• 3 years ago

Nice!!

wraptile• 3 years ago

Another great alternative is JSONPath[1] which unfortunately not as widely supported and known despite being brilliant!

It's inspired by XPath so it's very familiar instead of a complete new DSL. The killer feature imo is the recursive key lookup so you can write `people..address` and it'll find all "address" keys that descend from "people" anywhere in the JSON. It's by far my favorite parsing language for JSON and I wrote an introduction blog on how to use it in JSON dataset parsing [2] :)

1 - https://github.com/JSONPath-Plus/JSONPath

2 - https://scrapfly.io/blog/parse-json-jsonpath-python/

cryptonector22• 3 years ago

FYI jq also has recursive traversal.

gobdovan• 3 years ago

Thought this is an array programming language considering the name's made out of J and Q. Nice util

fiddlerwoaroof• 3 years ago

The way it works is sort of reminiscent of array programming.

jfoutz• 3 years ago

jq is great. I don't know how many times I've had to explain to engineers the invalid numeric literal error means your json is bad. no really, don't trust me, copy it into the ide. it's not jq. your message is malformed.

Strangely, I also have ECMA-404 and RFC8259 open in other tabs. mostly annoyance with the occasional flashes of anger over number formats and duplicate keys.

d_t_w2• 3 years ago

This is great, JQ is brilliant.

I love JQ so much we implemented a subset of JQ in Clojure so that our users could use it to munge/filter data in our product (JVM and browser based Kafka tooling). One of the most fun coding pieces I've done, though I am a bit odd and I love writing grammars (big shoutout to Instaparse![1]).

I learned through my implementation that JQ is a LISP-2[2] which surprised me as it didn't feel obvious from the grammar.

[1] https://github.com/Engelberg/instaparse

[2] https://github.com/jqlang/jq/wiki/jq-Language-Description#:~....

chatmasta2• 3 years ago

I can't stand jq. I realize this is an unpopular opinion, and our codebase at work has plenty of jq in the bash scripts, some of it even code that I wrote. I begrudgingly use it when it's the best option for me. But something about it rubs me the wrong way - I think it's the unintuitive query syntax and the need to search for every minute step of what I'm trying to do, and the frequency with which that leads to cryptic answers that I can only decipher if I am some sort of jq expert. But I have this instinctive reaction to all DSL languages that embed themselves into strings, like htmx and tailwind (both embedded in attribute string values). I realize some people like it, and it's a well-made piece of software, and I will even admit that sometimes there is no better choice. But I guess I just hate that it's necessary? I guess I could also admit it's the least-bad option, in the sense that it's a vast improvement over various sed/awk/cut monstrosities when it comes to parsing JSON in bash. Certainly once you find the right incantation, it's perfect - it transforms some raw stdin into parsed JSON that you can manipulate into exactly what you need. But for me, it ranks right next to regex in terms of "things I (don't) want to see in my code." I hate that the jq command is always some indecipherable string in the middle of the script. The only real alternative I've ever used is piping to a Python program that I define inline in a heredoc, but that ends up being at least as nasty as the JQ script.

cryptonector22• 3 years ago

Python is so much harder to process JSON data in than jq, that that is how I got into working with and on jq almost a decade ago.

papichulo2023• 3 years ago

Community refuses to admit that powershell is much better alternative to bash/python combo and here we are stuck in this mess.CI/CD scripts spaguetti is usually the most unstable piece of code in a company.

bfung• 3 years ago

Agree - by the time I need more than grep and reach for json parsing, it’s already complicated enough for a Python script. stdin pipped to json.loads ain’t that bad.

Def. seen jq thrown into sed/awk scripts where a readable programming language was the right move. People spend hrs finding the right syntax to these things ~ not always well spent.

nikau2• 3 years ago

Use gron for adhoc, then jq to script once you know what you need to do.

matthewtse2• 3 years ago

I love and hate it.

For simple things like navigating down one key, or one array entry, I know by heart, and it's incredibly useful. But anything more complicated, and I'm too lazy to lookup the documentation.

jq will fall into the bucket along with sed/awk of "tools I once wished to become an expert on, but will never do so because ChatGPT came along".

Would also put regex into that bucket, but they're so ubiquitous that I've already learned regexes. I wonder if the new wave of coders learning coding via ChatGPT will think of regexes the same way I think of sed/awk.

hn927268194• 3 years ago

I hate that the jq command is always some indecipherable string in the middle of the script

It might be worthwhile to just learn how jq works. At the end of the day, you need to learn some language to parse json. I hate DSLs too, but I cannot think of anything as useful and concise as jq.

but that ends up being at least as nasty as the JQ script

That's exaxtly why jq is so nice. Nice alternatives just don't exist

alexvitkov• 3 years ago

The fact that you have to learn a new language to parse JSON is frankly insulting. If you've gotten to the point you're parsing JSON with a shell script, you should've switched to a real language a week ago.

Some people are weird and awe at the ellegance of piping 8 obscure commands, but if I'm given this shit and have to keep it working, I'm rewriting it on the spot.

irjustin• 3 years ago

But for me, it ranks right next to regex in terms of "things I (don't) want to see in my code."

Yeah if you don't like jq you likely won't like Regex, xpath, etc. Any syntax that is incredibly terse and complex.

Like Regex though, jq is too powerful to ignore and many times the best tool to use.

dygd2• 3 years ago

Also not a big fan, but I hate it for a very specific reason:

  echo 12345678901234567890|jq
  12345678901234567000

viraptor2• 3 years ago

I've got similar feelings about it and recently I started experimenting with writing scripts in Nushell rather than bash + jq. I get the json object as a proper type in the script, get reasonable operations available on it and don't have to think of weird escaping for either the contents or the jq script. It cuts down the size my scripts by about a half and I'm very happy with the results.

larodi• 3 years ago

In my opinion (potentially nor popular) JQ has this appeal to nerds the same way that stuff like Perl does. I say this as someone who did Perl for 20years but now prefers python or JS…

For many people regexes are as bad as the jq queries… and vice versa. I would not recommend to write python script instead of regexp, but indeed it may work the same for small data and be more readable.

I love régex and been mastering it since 1999. So much that in 2013 I used it in production to parse binary protocol with dynamic sized fields. I believe the project is still talking 10k plus devices. Google must’ve just released protocol buffers… I would love to finally see regexes which can work over custom flow of objects and also on trees.

I also loved XPath which is very powerful and very comprehensible, then there is CSS1/2/3 which are again for queries to structures tree like data.

The prospect of now learning jq does not appeal me that much even though I appreciate its ingenuity. I may recommend it to dev/ops colleagues now and then, but for me this syntax is a lot of additional cognitive pressure which does not necessarily pay up. Of course if there is large amount of JSON data - it is the Swiss knife.

But nowadays I’ll likely use some LLm to generate the jq query for me. Also would joke with my bash-diehard colleagues who would love one more DSL…

mrzool• 3 years ago

I understand where you're coming from and often feel the same, but I'm also afraid that this is a clear case of inherent complexity: querying JSON is just a complex problem and requires a complex query language, regardless of how well a piece of software implementing it is designed. The same is valid for regexes of course.

hnlmorg2• 3 years ago

`jq` solves the problem of JSON in legacy shells. But I think the real problem is that the world is stuck using Bash rather than a more modern shell that can parse JSON (as well as other data structures) as natively as raw byte streams.

1vuio0pswjnm7• 3 years ago

"I can't stand jq."

jq is way too much for what I need. I hacked together a filter in C to reformat JSON and I like it better than every JSON library/utility I have tried. For simple reformatting, jq is slow and brittle by comparison. Also, I can extract JSON from web pages and other mixed input. All the JSON utilities I have tried expect perfectly-formed JSON and nothing else.

spion2• 3 years ago

The main problem is treating one-thing and many-things the same way. Its not a great PL design choice (and its why we can't have slurp as a filter). If streams (not arrays) were also first-class, we would easily have `smap`, `sselect` etc and the code would look like a functional programming language where | is the pipeline operator.

Otherwise, its fine if you try to keep the thought "everything is a 'filter' or a composition of filters, and a 'filter' is a function that either maps, flatMaps or filters things" in your mind at all times

kybernetikos• 3 years ago

Maybe you'd like the nushell approach more, have the structured data understanding and extraction tools built into the shell language.

adolph2• 3 years ago

it's the unintuitive query syntax and the need to search for every minute step

I love jq as a power tool and have the same challenges. I think the best path would have been for JavaScript to adopt something akin to JsonPath, although I more often reach to jq out of familiarity than use it in kubectl.

https://kubernetes.io/docs/reference/kubectl/jsonpath/

cpuguy83• 3 years ago

"Indecipherable string" to me means you likely don't understand the language or how it works. The language itself works very well for what it needs to do.

It does not work the same way as something like parsing an object and manipulating it in python. It is a query language. You are building up a result not manipulating objects.

Definitely unintuitive if you are coming from a programming language. Once learned it makes a lot more sense and is even preferable depending on your needs.

yoz2• 3 years ago

The responses to this comment seem to miss a vital point that the comment is making: languages executed within a different primary language are usually opaque to the tools in use. Those tools are usually aimed purely at the primary language, not any secondary languages used within it. Tools for the secondary language are now much harder to use because they (usually) have to be invoked and used via the primary language.

If I’m working on a Python script which has some jq embedded in it, then these problems probably exist:

- My editor will only syntax colour the Python, and treat jq code as a uniform string with no structure

- My linter will only consider Python problems, not jq problems

- My compiler, which is able to show parsing errors at compile time rather than runtime, will not give me any parsing errors for jq until execution hits it (yes, Python has a compilation step)

- jq error messages that show a line number will give me a relative line number for the jq code, rather than the real line number for where that code lives in the Python file

- My debugger will only let me pause and inspect Python, and treat the jq execution as a black box of I/O

I’m discussing this as a jq problem, but this happens far more commonly with SQL inside any host language. No wonder ORMs are so popular: their value isn’t just about hiding/abstracting SQL, it’s about wrangling SQL as a secondary language inside a different primary one.

- Microsoft’s LINQ for C#

- Webdev-focused IDEs which aim to correctly handle HTML and Javascript inside server-side languages (e.g. PHP)

- what else?

michaelcampbell• 3 years ago

`jq` and `GNU Parallel` share a world in my brain where I know they're wonderful tools, but I spend more time grokking the syntax of each one as rarely as I need either, than just writing a bash/sed/awk/perl, ruby, or python script to do what I need.

dekelpilli• 3 years ago

Just adding that a JQ implementation already exists on the JVM[1], although it's not 100% complete.

[1] https://github.com/eiiches/jackson-jq

mnutt• 3 years ago

jq is great for letting users munge their data; we do something similar letting users provision an incoming webhook endpoint, send us arbitrary json data, and set up mappings to do useful things with their data, along with regression tests, monitoring, etc. jq makes the majority of cases straightforward (basically json dot notation) and the long tail possible.

mingodad• 3 years ago

I also just add a JQ parser/grammar to the online LALR(1)/FLEX grammar editor/tester at https://mingodad.github.io/parsertl-playground/playground/ select "Jq parser (partially working)" from examples then click "Parse" to see a parser tree of the source in "Input source".

Any feedback is welcome !

cryptonector22• 3 years ago

Hey, that's an awesome playground! Thank you for this!

yoz2• 3 years ago

Thanks to this, I found your fantastic list of interesting and powerful parser projects: https://mingodad.github.io/parsertl-playground/

Thank you so much for compiling it!

A related question for you and anyone else into this kind of tooling: if you had to automate some structural edits across a codebase that contains a wide range of popular languages (say: C++, C#, Java, Ruby, Python), and you had to do it with a single tool, which tool would you use?

Dionakra• 3 years ago

Is your product available yet? I would be glad to try it out!

d_t_w2• 3 years ago

Yes, sure is! https://kpow.io/

You can get started with the Community Edition for free, it includes our JQ implementation (we call it kJQ - https://docs.kpow.io/features/data-inspect/kjq-filters)

Just shout if you need any help.

tambourine_man• 3 years ago

If you’re like me and use jq infrequently enough that you have to consult the documentation every time, try gron. It’s greppable json

https://github.com/tomnomnom/gron

Galanwe• 3 years ago

Damn thats incredibly well thought!

You can even `gron | grep | sed | gron -u`.

Awesome tool thanks for sharing.

zimpenfish3• 3 years ago

I do this for archiving my YouTube watchlist into git - all the video/thumbnail/subtitle URLs are dynamic with expiry which makes them pointless to keep around. A quick `gron | sed | gron -u` replaces them all with "URL" and my diffs are much smaller and happier now.

xiphias2• 3 years ago

If you like gron, try fastgron :)

It supports everything gron supports, but 50x faster.

preommr• 3 years ago

Wow, that's a pretty cool - so simple but so useful.

I'll also share that I've been using `curl cheat.sh/jq` (cheat.sh in general is a great resource) for years.

Although now I'd probably use something like chatgpt.

jkrubin• 3 years ago

Chad is super helpful with jq!

dewey• 3 years ago

I was always struggling with the jq syntax...but I'm consistently impressed by ChatGPT coming up with the right commands from an example JSON.

paradox460• 3 years ago

jq and miller[1] are essential parts of my toolbelt, right up there with awk and vim.

[1] https://github.com/johnkerl/miller

porridgeraisin• 3 years ago

Thanks for pointing me to miller. Too good.

pyrolistical• 3 years ago

Love jq. Solved many problems

But at one point I started write long jq modules and while it was pretty straightforward, there are less people familiar with jq.

So I declared jq bankruptcy and rewrote it as a nodejs script. The rest of the team was relived

cryptonector22• 3 years ago

I started write long jq modules

If you're writing long jq modules, you probably do want a different (faster, better) language.

nikolay2• 3 years ago

In addition to my previous comment about jq-like tools, I want to share a couple other interesting tools, which I use alongside jq are jo [0] and jc [1].

[0] https://github.com/jpmens/jo

[1] https://github.com/kellyjonbrazil/jc

goku123• 3 years ago

And jless [1] and gron [2].

This is the first I'm hearing of gron, but adding here for completeness sake. Meanwhile, JSON seems to be becoming a standard for CLI tools. Ideal scenario would be if every CLI tool has a --json flag or something similar, so that jc is not needed anymore.

[1] https://jless.io/

[2] https://github.com/tomnomnom/gron

dang• 3 years ago

Recent and related:

First release of jq in 5 years - https://news.ycombinator.com/item?id=36951830 - Aug 2023 (27 comments)

wanderingmind• 3 years ago

A slight tangential question. I use ripgrep-all along with fzf to interactively search through files from cli. Is it possible to integrate jq (or some equivalent) into this search ecosystem to search through json files?

lordgrenville• 3 years ago

Yes https://gist.github.com/reegnz/b9e40993d410b75c2d866441add2c...

black_puppydog• 3 years ago

Adds new builtin pick(stream) to emit a projection of the input object or array.
jq -n '{"a": 1, "b": {"c": 2, "d": 3}, "e": 4} | pick(.a, .b.c, .x)'

This is a godsend! Thanks to the contributors! <3

nivekastoreth• 3 years ago

This is a fantastic new feature. I would also love a version of ‘pick’ that works on streaming data, since that doesn’t seem to be possible currently without reassembling the stream first

nnutter• 3 years ago

I just installed from Git the other day to use this feature. It’s very useful!

wwader6• 3 years ago

If you don't need to digg deep it's also possible to do:

  $ jq -n '{"a": 1, "b": {"c": 2, "d": 3}, "e": 4} | {a, e}'  
  {
    "a": 1,
    "e": 4
  }

arwineap• 3 years ago

The next killer feature for postman alternatives is integrated jq support for your json output.

Postman handles this especially poorly...

cryptonector22• 3 years ago

What is postman?

Omatic810• 3 years ago

It's an API tool: https://www.postman.com/

ParetoOptimal• 3 years ago

https://github.com/pashky/restclient.el#usage

kenrose• 3 years ago

It's great to see jq get some releases again.

It's a big part of our check evaluation infra at OpsLevel.

lwhsiao• 3 years ago

I'll give a plug for jaq [0], a clone focused on correctness, speed, and simplicity. It only implements a subset of jq, but I've been enjoying it so far.

[0] https://github.com/01mf02/jaq

faizshah• 3 years ago

I wish there was a faster version of jq, I was only able to get a few mb/s throughput out of jq vs few hundred mb/s throughput out of ripgrep.

I often use ripgrep to setup quick bash pipelines for rapid data analysis, would love to be able to use jq for that purpose. These days I am setting up scripts with simdjson but the cost of writing a script vs quickly setting up jq or ripgrep in a bash pipeline are orders of magnitude different.

cryptonector22• 3 years ago

Did you try gojq or jaq?

thiht• 3 years ago

These are great release notes. Looks like the project is in good hands

ithkuil4• 3 years ago

One of the reasons I like/tolerate jq is that it's stable, i.e scripts written for it a few years ago still work the same today.

I have some code around for yq instead that keep breaking because yq keeps improving in non backward compatible ways (I didn't investigate how often yq introduced backwards incompatible changes, but the issue affected me several times in unrelated places, CI scripts or whatnot, that by their nature end up running with different versions of base tooling and update them at various pace)

I was always thus grateful to the great wisdom of the jq maintainers for their understanding of the importance of backwards compatibility.

I hope this announcement doesn't mean that this stability was just an accidental side product of stagnation and that once stagnation is "fixed" it will be done at the expense of stability.

El_RIDO• 3 years ago

Reg. yq/jq incompatibility - unsurprisingly there are many projects that provide yq functionality as wrappers around jq. To list but a few:

- https://kislyuk.github.io/yq/

- https://github.com/kyle-long/yq#yq

- https://github.com/up1/yq-1#yq-command-line-yamlxml-processo...

- https://github.com/onixspot/yq-2#yq-command-line-yamlxmltoml...

- https://github.com/simonrupf/convert2json#convert2json

ainar-g3• 3 years ago

What are you using yq for? Unless you use YAML-only features (e.g. integers, arrays, and objects as object keys), it seems like it would be easier to just pipe-convert your YAML to JSON and process it with jq.

dividedbyzero3• 3 years ago

Not them but I use it in a few places to make automated edits to yaml files.

sumek83• 3 years ago

"Improving in non backwards compatible way" sounds like "deteriorate"

dividedbyzero3• 3 years ago

That's a very negative view. If Excel were to fix their 1900-is-a-leap-year bug, I'd call that a clear improvement, even though it would break some spreadsheets that work around the bug. Seen through that lens every major version of almost all programming languages would be a deterioration.

ithkuil4• 3 years ago

yes I tried to not be too aggressive towards yq's authors who surely don't deserve bad words, but at the same time I wanted to express how painful is even the smallest backwards incompatible change to a tool that may end up being used in many tiny dark corners of your automation that everybody forgets to maintain.

wwader6• 3 years ago

Hey! as one of the maintainers i can safely say that we take backwards compatibility very seriously

ithkuil4• 3 years ago

awesome! Any plans for jq to get native support for YAML?

gray_-_wolf• 3 years ago

I wonder how often someone will develop a script on Archlinux and later be surprised that it will not work in our Debian CI. One nice property about jq was that 1.6 was everywhere, remains to be seen how annoying this will be. Probably not that much.

Is there a way to get jq version inside the script?

donatj• 3 years ago

jq is a tool that is so powerful and useful it really should be made part of the POSIX standard. It should come with everything preinstalled. It’s an amazing tool like that of an earlier more refined age.

kouru225• 3 years ago

Reading “JQ” in this context makes me realize how terminally online I’ve been all these years

ta8903• 3 years ago

More programmers need to be aware of the jq.

anonymoushn• 3 years ago

A new jq release is exciting because it means that jq can finally stop mangling numbers that it does not do any math on.

Dowwie• 3 years ago

'jq' is a lot easier to use with ChatGPT assistance. Simply provide sample JSON and tell the LLM how you want to use JQ to parse and transform.

VoodooJuJu• 3 years ago

Works well enough, but it really tripped me up at first with the syntax. Had to constantly look things up while using it.

  jq -r \
  '.apps.http.servers.srv1.routes[0]
  | .match[0].header.[env.AUTH_USER_HEADER][0] = "$username"'

That's an error because you can't select an env var key '[env.AUTH_USER_HEADER]' in the middle of a chain like that, only immediately following a pipe:

  | .match[0].header | .[env.AUTH_USER_HEADER][0] = "$username"'

But then I need to preserve the parent object after the assignment and that pipe throws it out. Thankfully, parentheses fix that:

  | (.match[0].header | .[env.AUTH_USER_HEADER][0]) = "$username"'

After working out the gotchas, it's quite powerful, like regex, but a little clunky, though not nearly as much as regex.

rewmie• 3 years ago

Almost feels like I'm learning regex again.

But isn't that what jq is and always has been? I mean,what led you to believe that an entirely unrelated pattern matching language would work?

cryptonector22• 3 years ago

jq -r \ '.apps.http.servers.srv1.routes[0] | .match[0].header.[env.AUTH_USER_HEADER][0] = "$username"'
That's an error because you can't select an env var key '[env.AUTH_USER_HEADER]' in the middle of a chain like that, only immediately following a pipe:
| .match[0].header | .[env.AUTH_USER_HEADER][0] = "$username"'

You can use `env.AUTH_USER_HEADER` as a key the way you wanted. The issue is that you had to write `... | .match[0].header[env.AUTH_USER_HEADER] ...` -- no `.` between "header" and the index operator!

This complaint is a fairly frequent one, so in fact we did "fix" this in 1.7! You can now write `.a.[0]` and it works.

crest• 3 years ago

The addition of raw 0-byte separated output will make reliable scripting (e.g. piping into xargs -0) a lot easier.

ccakes• 3 years ago

There seems to be a few comments from people that dislike jq. If you're that way inclined, gron[0] might be more your thing.

[0] https://github.com/tomnomnom/gron

goku123• 3 years ago

Seeing this news today, I decided to give jq another try and ended up discovering jq-mode [1] for emacs. It doesn't just support jq filter file editing, it supports jq in org-mode and something else called 'jq-interactively'. This interactive mode allows you to apply jq interactively on a JSON or YAML (with yq) buffer. The buffer contents become the filtered value when you finish editing the jq filter with a return. This is especially impressive to see in yaml files.

[1] https://github.com/ljos/jq-mode

G3rn0ti• 3 years ago

Didn’t know this. Thanks for the tip!

Personally, when I test REST APIs, I use „restclient.el“ all the time which also comes with a great JQ integration („jq-set-var“ for example for deriving request variables from responses). For traversing larger responses I use „counsel-jq“ in a customized JSON mode: https://github.com/200ok-ch/counsel-jq

But I’ll give the major mode a try, too.

niam• 3 years ago

Is this a proper summary of the use cases over Deno/Python scripts?

* Run directly from the command line

* Small interpreter ~1MB

* Compact language (for better or worse)

* Stable

cryptonector22• 3 years ago

This is a common refrain. So much so that when we changed the release artifact names in 1.7 we broke scripts and docker recipes that would download jq from the github releases 'latest' URIs! (This is fixed now.)

qalmakka• 3 years ago

jq is amazing, and I'd like it even more if it supported CBOR too. The forks are just too finicky and are quickly abandoned, and the format is basically binary JSON.

cryptonector22• 3 years ago

I'm working on binary support, then we can have CBOR, yeah.

pantsforbirds• 3 years ago

ChatGPT has massively improved my ability to work with jq. I use it just infrequently enough that I constantly have to read the docs for anything non trivial, but ChatGPT lets me pump out scripts quickly. It's been super nice for my work flow.

cjvirtucio• 3 years ago

my favorite tool; makes writing a bash fun for me.

pygar• 3 years ago

It would be nice to have this in something like coreutils so that I can be reasonably sure it's included in most systems.

reegnz• 3 years ago

Let me drop my jq-zsh-plugin here: https://github.com/reegnz/jq-zsh-plugin

It's a line-editor, and allows me to query json documents ultra-fast. It optimizes for a really fast feedback.

jq 1.7

Related Stories