TC39: Add Object.groupBy and Map.groupBy

https://github.com/tc39/ecma262/pull/3176

by moritzwarhier • 3 years ago

80 61 3 years ago

This seems like a nice addition. Is it soemthing that a lot of tools roll their own, or is something that can’t be done in user space? I’m relatively ignorant of the limits of js

tibbar3• 3 years ago

It's fairly easy to roll your own. An example implementation of groupBy in lodash, for example: https://github.com/lodash/lodash/blob/4.17.15/lodash.js#L939....

More broadly, this is an example of a utility function for organizing data in a data structure, which JavaScript and all other major programming languages are well-equipped to implement in arbitrary ways, so this is just a convenience function. It's not like a hook into underlying environment APIs that can't be independently shimmed.

jauntywundrkind• 3 years ago

Trying to read lodash can (to put it mildly) be difficult. Lots of code reuse.

This is probably missing checking for a bunch of corner cases, but this is the general idea:

  // quick & dirty Object.groupBy
  function groupBy(iterable, callbackFn) {
    const out = {}
    for(const [key, value] in Object.entries(iterable)) {
      const name = callbackFn(value, key)
      (out[name] || (out[name] = [])).push(value)
    }
    return out
  }

I literally wrote something very like this yesterday (after checking & finding it only got added to Node 21, and deciding not to pull in a dependency).

Edit: had keyBy impl in here, fixed! Thanks commenters!

ricardobeat• 3 years ago

It's very funny to point to lodash as an example of simplicity. Here is what happens when you start unraveling the code you linked to:

https://gist.github.com/ricardobeat/040af80971273f5abd71c2bf...

I got tired, but wouldn't be surprised if the whole thing goes beyond 1000 lines. It's basically a black hole. Better hope you really, really trust their test suite, it's impossible to debug and there must only be a handful of people familiar with the whole codebase.

erulabs2• 3 years ago

It can be trivially rolled by hand, but it’s also a pretty common function in other languages. I’d guess most node programmers reach for lodash for this.

paulddraper• 3 years ago

a pretty common function in other languages

Except oddities like C, C++, Perl, PHP, and Go :)

paavohtl• 3 years ago

It can absolutely be written in JavaScript (it is a Turing complete language after all), and it is a common utility function found in most general utility libraries like Lodash and Ramda.

moritzwarhier5• 3 years ago

It is indeed something that you often roll your own, write an utility function for, or include libraries like lodash for.

JS standard library is notoriously deficient.

This is a good addition in my view too, especially including Map.

When dealing with DB/API results, one can of course always say that such grouping in the client is an anti-pattern.

But in reality it can be reasonable and useful.

If you write one-off algorithms in JS to transform small amounts of data on the client, you probably habe done this.

And even for data-heavy applications, it could prove to be a performance benefit for functions that use Map instances during computation.

Though that's just wishful thinking until the runtime developers choose to optimize these functions, if possible.

pmontra• 3 years ago

When dealing with DB/API results, one can of course always say that such grouping in the client is an anti-pattern.

It depends on how many data you're working on. This groupBy is useful for filters in data tables.

If you have to download a zillion of records and filter them, client side filtering is a bad idea. If you have a few thousands of them, it could possibly be the faster solution overall.

thenbe• 3 years ago

You can roll your own or use a utility library. A simple zero-dependency library would be something like just-* [1]. Although I now prefer remeda [2] as it seems to have the best typescript support, especially the strict variants such as `groupBy.strict`.

[1] https://github.com/angus-c/just#just-group-by

[2] https://remedajs.com/docs#groupBy

jackconsidine• 3 years ago

Anyone know if `keyBy` will also be supported? I suppose it'd be pretty trivial to take a `groupBy` and make it `keyBy`, but then again it's pretty trivial to implement `groupBy` from scratch.

I no longer install lodash that often anymore

derefr• 3 years ago

I'm a big fan of the design of Elixir's Enum.group_by, which has one-parameter and two-parameter forms. The one-parameter form is like any other language's groupBy — but the two-parameter form takes two mapping closures; passes each element into both of them; and uses the output of the first closure as the grouping key, and the output of the second closure as the value to be registered under that grouping key.

This flexible primitive enables you to do basically any grouping transform you want (incl. the Lodash-style keyBy) in a single short line of code.

It's a lot like a sortBy operation, in that to emulate it, you would have to do a map to extract the key, producing pairs; sort (or in this case group) the pairs; and then deep-transform the pairs inside the data structure by unwrapping the keys off them. In other words, it's something that's a bit too high-friction to reach for if the language doesn't just give it to you (you'd probably do what you're planning to do some other way); but if the language does give it to you, you'll use it quite often.

edflsafoiewq2• 3 years ago

For reference, keyBy returns an object with the same keys as groupBy, but the value of a key is the last element to produce that key, instead of an array of all of the elements that did.

bakkoting7• 3 years ago

No current plans for `keyBy`, and I don't know that it's really that well-motivated. (I am on TC39.)

moritzwarhier5• 3 years ago

You can do that by nesting

  Array.prototype.map

in the parameter for

  Object.fromEntries

in an easy way (probably not as optimized as a built-in, but that might be irrelevant for most cases, since its just a duplicated iteration, not quadratic)

koito172• 3 years ago

Nice, the JS standard library is gradually getting functions I take for granted in ClojureScript. :)

I noticed this part in the proposal

  groupBy calls callbackfn once for each element in items, in ascending order, and constructs a new Object of arrays. Each value returned by callbackfn is coerced to a property key

Does JS allow arbitrary objects as keys? I am asking because `group-by` in ClojureScript is quite flexible. e.g. you can find anagrams like so

  (def words ["meat" "mat" "team" "mate" "eat" "tea"])
  
  (group-by set words) ; set is a function that creates sets from collections
  
  ;; => {#{\a \e \m \t} ["meat" "team" "mate"]
  ;;     #{\a \m \t}    ["mat"]
  ;;     #{\a \e \t}    ["eat" "tea"]}

I am wondering how one could translate this using Object.groupBy as specified in the proposal.

tibbar3• 3 years ago

JS does indeed allow mapping arbitrary objects to keys. However, it is the memory address of the object that is hashed, not the semantic value. So, for example, if you do:

    objects = {}
    items1 = ["a", "b"]
    items2 = ["a", "b"]
    objects[items1] = 1
    objects[items2] = 2

then objects[items1] will return 1, and objects[items2] will return 2, but objects[items1] !== objects[items2].

EDIT: Sorry, this only works for dictionaries that are Maps, not Objects! See the responses. My fuzzing about in the console led me astray; you should start with `objects = new Map()` instead of `objects = {}`.

thegeomaster• 3 years ago

This is not true. I think you're confusing JS objects with Maps. This will just coalesce the key to string and overwrite one element of the object with the other.

With a map it works like you described:

    objects = new Map()
    items1 = ["a", "b"]
    items2 = ["a", "b"]
    objects.set(items1, 1)
    objects.set(items2, 2)

mediumdeviation• 3 years ago

Actually that only works with Map. For plain objects the key is always the stringified cast of the key.

    > o = { [{}]: 1 }
    { '[object Object]': 1 }
    > k = { toString() { return 'a' } }
    { toString: [Function: toString] }
    > o = { [k]: 1 }
    { a: 1 }

koito172• 3 years ago

Ah, I didn't know Map in JavaScript allowed arbitrary keys whereas Object always serializes to strings. I guess that is the reason for having a Map.groupBy in the proposal.

Thanks for taking the time to explain, everybody

moritzwarhier5• 3 years ago

JS allows arbitrary values (including objects/references) as keys in Maps, but not in objects, there it is cast to string by default (e.g. "object [Object]", "null" or "undefined", this is not intended usage of course).

The symbol primitive is also allowed as an object key, using square bracket access.

And arrays are "exotic objects" that have some special behaviors around their keys (auto-updating length property)

ForkMeOnTinder• 3 years ago

Does JS allow arbitrary objects as keys?

Object doesn't, but Map does. It's generally a good idea to use Map anyway for dynamic keys.

freedomben• 3 years ago

You ClojureScript people seem to be stalking me :-D

I've wanted to learn Clojure for years but haven't found the right reason, until discovering Logseq. Such a cool language!

Waterluvian• 3 years ago

When it comes to helper functions on existing types, I feel like you really can’t have too many… within reason.

If we added like four new kinds of object/map, that complicates things… but just formalizing more common access and iteration patterns for existing structures seems low risk.

What I can’t wait for are all the set operations that would make ‘Set’ truly powerful.

mbStavola• 3 years ago

This is one of those functions I end up implementing in pretty much every single non-trivial JS project I work on. Glad to see some movement here.

pqdbr• 3 years ago

On a side note, I really wish I could learn to tell exactly if a new feature like this is already supported in a project I'm working.

babel.config.js is just a mistery to me. @babel/present-env, targets: { node: 'current' }, forceAllTransforms, useBuiltIns, 'corejs', modules, @babel/plugin-proposal-object-rest-spread.

I mean, I generally dump it into Chat GPT and ask for an explainer, but... how can I know for sure I can use `array.reverse()` and it will be correctly handled in older browsers?

And how does the babel version in yarn.lock relate to all this?

What about the .browserslistrc which contains 'defaults' ?

My god.

n2d4• 3 years ago

https://compat-table.github.io/compat-table/es2016plus/

Enable the "show obsolete platforms" checkmark to see the data for older versions of Babel/core-js. But probably you should just upgrade that instead.

Vt71fcAqt72• 3 years ago

I hope they stick to these useful helper functions instead of adding more complexity like Type Annotations that provide no type soundness or runtime checks of any kind[0]

[0] https://github.com/tc39/proposal-type-annotations

wffurr• 3 years ago

That’s an interesting take when the stated purpose of that proposal is “to enable developers to run programs written in TypeScript, Flow, and other static typing supersets of JavaScript without any need for transpilation”.

That seems like a fine goal. Allow runtimes to execute JS with type annotations as-is.

Vt71fcAqt72• 3 years ago

The issue for me is four-fold:

1. Adds complexity to the language

2. Adds complexity to engines

3. Adds complexity to developers, especially new developers ("wait is it typed or not")

4. Most importantly, all but guaranties we will never have true types in JavaScript for things that could benefit from it like node or electron where instant compile time isn't necessary.

All this for a feature only helping some developers some of the time so they can run code that will be stripped out in production to reduce size anyway on their local browser a bit more easily.

aragonite3• 3 years ago

What's the thinking behind attaching Object.groupBy to the Object constructor? With the other Object.X() functions I can easily see why it makes sense they exist on the Object constructor, because they generally follow the pattern of taking an object as argument, and say, freezes that object, returns that object's [[Prototype]], returns that object's keys, check if that object is sealed, etc. By contrast, Object.groupBy is a utility function that takes an iterable and a callback, and doesn't seem to have anything distinctively to do with "Object" per se.

(I guess one exception is Object.is, which takes two values as arguments. But even Object.is makes a lot of sense existing on the Object constructor because it exposes an important abstract operation (SameValue). Object.groupBy is only a utility function)

danvk• 3 years ago

What about Object.fromEntries?

See https://github.com/tc39/proposal-array-grouping for why this isn’t a method on Array.prototype.

flqn• 3 years ago

Constructs an object from kv pairs, definitely assoctiated with the "Object" prototype. Groupby is more dubious

LegionMammal978• 3 years ago

Map.groupBy() returns a Map with the keys mapped to array values. In parallel, Object.groupBy() returns an Object with the keys cast into property names. Thus, I'd say it fits the pattern of Object-related static methods, since it's using the Object mechanism as a map.

zeroimpl• 3 years ago

The part I’m confused on is the documentation saying the returned object from Object.groupBy doesn’t extend from Object.prototype. Is it still an object?

    The return value of groupBy is an object that does not inherit from %Object.prototype%.

bakkoting7• 3 years ago

We originally tried to put it on Array.prototype, under multiple different names, and that broke various pages in various ways. So we gave up and had to put it somewhere else. And it's conceptually similar to `fromEntries` - they're both ways of making an object. So the Object constructor was the obvious choice.

replygirl2• 3 years ago

it's a noble approach, but feels like it's at its limits if no one could find a reasonable name that works with Array. there's gotta be a point at which it makes sense to EOL third-party stuff like that: set a date far in the future, and any pages where document.lastModified is greater get Array.groupBy as well

no_wizard• 3 years ago

This is why we need a global Iterator / Iterable type. These are iterators at the end of the day after all. That would also signal the fact they could be used for more than one data structure (Array, Map, POJOs)

dagurp• 3 years ago

I would like to have .partition as well but I guess I can get by with groupBy.

Array.product would be sweet as well (i.e. like product in python's itertools).

mattlondon• 3 years ago

In case you need a example, there is a good one here: https://github.com/tc39/proposal-array-grouping/

javajosh• 3 years ago

I don't think they should do this especially since they don't even have an Array.prototype.peek()

gabrieledarrigo• 3 years ago

Holy moly, the static method is pretty horrible. Quite sad.

TC39: Add Object.groupBy and Map.groupBy

Related Stories