Go 1.21 may have a clear(x) builtin

https://utcc.utoronto.ca/~cks/space/blog/programming/GoFutureClearBuiltin

by aviramha • 4 years ago

104 126 4 years ago

utcc.utoronto.ca

halpmeh• 4 years ago

The issue with NaN equality is interesting, but is that really why they're adding a clear(x) builtin? What if you want to remove a single-NaN key from a map? clear(x) seems like a band-aid at best if the Go team is strictly trying to fix removing NaN keys in maps.

erk__• 4 years ago

The way Rust dels with this issue is by saying that the Key must implement both `Hash` and `Eq` where Eq is a strict equality compared to `PartialEq` which allows the equality relation to be incomplete. While integers implement both PartialEq and Eq, floating point values only implement PartialEq.

https://doc.rust-lang.org/std/collections/struct.HashMap.htm...

https://doc.rust-lang.org/std/cmp/trait.Eq.html

https://doc.rust-lang.org/std/cmp/trait.PartialEq.html

There is also a partial ordering relation `PartialOrd` (and `Ord` for complete relations) Floating point values only implement the partial relation as well because of NaN's. This means that it is a bit harder to sort them, though the floating point standard also has a seperate complete comparison relation with some extra rules which you can use instead.

https://doc.rust-lang.org/std/primitive.f64.html#method.tota...

duped2• 4 years ago

This is one of those rough edges where Rust is more annoying than helpful imo. It's also inconsistent.

I'd rather panic on NaN comparisons than complicate the fundamental comparison traits. Treat it like integer overflow or array bounds access. We don't have different traits for indexing that might fail or overflow that might happen. There are methods on the trait to handle those cases when you care about them.

But that said, this feels like an XY issue. If you're comparing hashes of floats as keys somewhere you don't want IEEE 754 defined equivalence. You almost certainly want bit equivalence, which means the keys should be type punned to u32/u64. Classic example is memoizing function calls with floats. You don't want to compare the semantic values of the arguments, but the literal bits. For floats that's not the same.

tialaramex6• 4 years ago

Rust's HashMap and similar containers have clear() and many of them have drain()

Drain is interesting, because it resolves the other problem if things are in the collection which can never be retrieved from it. Drain gives you each of the things in the collection, one at a time, to do with as you wish, the collection is now empty. So this means even if the collection has a dozen SillyNonsenses in it, all of which insist they're not the one you were looking for whenever you go looking for a SillyNonsense in the collection, you get all twelve of them out in Drain, and can examine them as you wish.

Of course if you just wanted to fish one thing out and then throw the rest away, you can do that, once you drop the result of Drain the rest of the things being drained are dropped.

martisch• 4 years ago

This for loop is less efficient than clearing a map in one operation

For maps with keys that are reflexive with == the Go compiler already optimizes the range loop to a single efficient runtime map clear call: https://go-review.googlesource.com/c/go/+/110055

tapirl• 4 years ago

Yes, for most cases, the for-loop is as efficient as the proposed "clear" function. I am not sure whether or not this is still true if there are NaN keys in the cleared map.

inglor• 4 years ago

Interestingly JavaScript handles this correctly and specified Object.is equality that is different than `===` equality around NaNs so `const m = new Map(); m.set(NaN, 3); m.get(NaN)` returns 3 and `.delete` similarly works.

ch_123• 4 years ago

I'm curious why this is not implemented as a method on the map type (and others like list) instead of being a top-level builtin. I suppose it is consistent with other collection operations such as append and len... which I guess makes me wonder why those are builtins as well.

esprehn• 4 years ago

The answer is lack of generics in all previous versions of Go. There's no way to express a generic interface to have methods for map or slice (which are not interfaces, but compiler magic instead). All the operations on builtin types must be implemented as globals handled by the compiler.

hesdeadjim• 4 years ago

Been using Go since the early days, and 'make/append' always felt clunky and out of place. Especially when you could just use literals for a map/slice...

urzaj• 4 years ago

From the go FAQ:

Why is len a function and not a method?
We debated this issue but decided implementing len and friends as functions was fine in practice and didn't complicate questions about the interface (in the Go type sense) of basic types.

I imagine similar reasoning applies to "clear" here.

https://go.dev/doc/faq#methods_on_basics

saghm3• 4 years ago

That's a pretty weird rationale IMO. "It doesn't break anything" should be the minimum requirement for a major design choice, but it certainly isn't sufficient on its own. They sort of imply that making it a method would "complicate questions about the interface of basic types", but I don't really understand what that would even mean "in the Go type sense". Are they worried that someone might make an interface with a `Len` method and therefore be able to pass slices and maps to it? That honestly seems like a feature, not a bug, and certainly not worth throwing out the nicer syntax for.

mseepgood• 4 years ago

Russ Cox explains in the issue why they don't want it to be a standard library function:

https://github.com/golang/go/issues/56351#issuecomment-12914...

"We talked about defining maps.Clear in #55002, but how? It would not be possible to write in pure Go code. Instead it would need some kind of hidden entry point into the runtime.

In the past I've referred to that kind of library change as a "back-door language change", meaning it's a language change - it adds something not possible in pure Go - but pretends not to be one. We try hard to avoid doing that."

AtNightWeCode• 4 years ago

Most obvious flaw in Go is the top-level bs. Why on earth are they adding things to that? It is off course down to other poor decisions made historically that forces this design. But anyway. They should fix the lang and scrap the top-level nonsense all together.

For a beginner in Go it makes absolutely no sense at all. Most of the builtins should be methods.

morelisp3• 4 years ago

Why would they be implemented as methods when they can be functions?

akira2501• 4 years ago

You've already got zero and negative zero as an outlying case. I'm not sure why anyone would feel comfortable using floats as keys in a map. To anyone who has done this.. why? What was the use case?

craigching• 4 years ago

Go has no built-in set data structure, you use map if you don’t want to bring in a third party. Maybe this is why?

Ferret74462• 4 years ago

A set of floats sounds like an equally questionable idea as a map of float keys, for the exact same reason.

wallstop• 4 years ago

For a game I'm making, I have a helper function that takes in a specified time (as a float) and returns a (heap based) wait instruction. The times are literals sprinkled throughout the code base of callers of this function, so I maintain a cache of time -> wait instruction to reduce allocations.

iudqnolq3• 4 years ago

Wouldn't your life be much simplier if you'd represented time as integer milliseconds? I totally get it being too late now, but I think that's an argument for making floats a little bit unpleasant to start working with.

zmj• 4 years ago

C# had the same NaN behavior until recently: https://learn.microsoft.com/en-us/dotnet/core/compatibility/...

rawling2• 4 years ago

Oof, I don't like that Matrix3x2 type. "It's a 3x2 matrix, but we're going to pretend it has a last column going 0/0/1 so we can get a determinant or multiply them together, but when we negate or add or subtract them it's still going to be 0/0/1."

_old_dude_3• 4 years ago

It's the classical representation of affine transforms (translate, rotate, scale, etc) using homogeneous coordinates.

https://en.wikipedia.org/wiki/Transformation_matrix#Affine_t...

kangalioo• 4 years ago

Would anything actually break if NaN == NaN equalled true?

moomin2• 4 years ago

Yes, in surprising places. But the most obvious thing is that comparing float values would become significantly more expensive.

derriz3• 4 years ago

Could you explain how/why?

derriz3• 4 years ago

I can't see it. I've no idea what problem was solved by the current weird IEEE behavior.

It must have been a biggy because this solution means your domain now loses the property that, in general, x == x. This property is a fundamental axiomatic feature of any notion of equality. A non-reflexive equality isn't just weird - it's daft/stupid/surprising.

morelisp3• 4 years ago

I've no idea what problem was solved by the current weird IEEE behavior.

The same problem that's solved by having a lot of proofs say "for all y != 0, ..." If NaN == NaN, then you can no longer assume that x/z == y/z implies x == y. Which is also a fundamental (though non-axiomatic) result of arithmetic!

NaN is an effort to represent partial functions in code that usually expects totality. It's not the best way but it's pretty good, and making NaN == NaN is also not how we'd do it with decades of hindsight.

josephcsible• 4 years ago

Isn't "if(x != x)" the canonical way to test whether x is NaN? That seems like quite a big thing to break.

jeroenhd3• 4 years ago

Most languages have some kind of isNaN(value) method in my experience. Even C and C++ have an isnan method (C since ±1999, but other platforms had the macro before: https://repo.or.cz/w/glibc.git/blob/HEAD:/sysdeps/ieee754/db...)

People do probably rely on NaN ≠ NaN but I think that's more of a side effect of compatibility than anything else.

sedatk• 4 years ago

Yes that would mean that sqrt(-1) equals to 0/0 for example, which might open a new portal to hell.

yawaramin2• 4 years ago

OK, now I understand why OCaml's Float.compare function compares NaNs as equal to each other: https://discuss.ocaml.org/t/assertions-involving-nan/10762/4

_old_dude_3• 4 years ago

In Java, the primitive type double uses the IEEE-754 semantics and java.lang.Double uses the bits by bits comparison semantics [1] so List<Double>.remove() works correctly.

[1] https://docs.oracle.com/en/java/javase/19/docs/api/java.base...

uluyol• 4 years ago

This won't help you in a future version of Java where you can write List<double>, right?

masklinn5• 4 years ago

No.

What will help you is that `clear()` is part of `Java.until.Collection` so just about every container has it.

_old_dude_3• 4 years ago

It depends if List<double> is an alias for List<@PleaseSpecialize @NotNull Double> or not.

derriz3• 4 years ago

Java has a specific hack for this which I discovered by accident a few years ago. Normally, given (primitive) double values d0 and d1, then boxing them does not affect equality testing - i.e. d0 == d1 if and only if Double.valueOf(d0).equals(Double.valueOf(d1)). However if d0 and d1 are both NaN, then the boxed versions ARE considered equal while d0 == d1 is false.

This inconsistency infuriated me when I discovered it but the Javadoc for Double.equals explicitly states that this anomaly is there to "allow hash tables to work properly".

bobbylarrybobby• 4 years ago

Using floats as hash keys is insane, no?

metadat4• 4 years ago

Yes, why would you ever want a map with floating point keys?

I'm struggling to think of a valid use case where there is no better alternative.

Any design utilizing this language "feature" seems masochistic and begging to get sliced by sheet metal edges.

vbezhenar• 4 years ago

No. JavaScript does that all the time.

chrisseaton8• 4 years ago

It's done all the time in JavaScript. Why not in Java?

bushbaba• 4 years ago

Not if you’re hash table is designed to accommodate key conflicts, and the float is deterministic.

zbentley• 4 years ago

Not at all.

If I get floats from $somewhere, I might want to index them into a hash data structure without creating a huge sparse array.

If you assume float numbers have "identity" such that, whatever the precision, "x == x" always holds (other than NaN), then this is a perfectly valid thing to want.

xg156• 4 years ago

Note that the contract for the equals() method requires that the method always returns true when called on itself. So the designers had to decide whether to break the contract on NaN or the contract on equals(). I could imagine that honoring equals() was the solution with the least expected breakage with downstream users.

unwind• 4 years ago

That was weird. Makes sense once you know about NaN's incomparability, but still surprising.

I had to try it in Python (3.10.4, Windows on x86), it worked fine:

    >>> import math
    >>> a={math.nan: 1}
    >>> a
    {nan: 1}
    >>> del(a[math.nan])
    >>> a
    {}

vhcr• 4 years ago

This works as expected only because id(math.nan) is always the same, if you build the nan dynamically, it will fail:

    >>> d = {float('nan'): 1}
    >>> d
    {nan: 1}
    >>> del d[float('nan')]
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    KeyError: nan

masklinn5• 4 years ago

It's because cpython shortcuts through identity: https://github.com/python/cpython/blob/main/Objects/object.c...

    /* Quick result when objects are the same.
       Guarantees that identity implies equality. */
    if (v == w) {
        if (op == Py_EQ)
            return 1;
        else if (op == Py_NE)
            return 0;
    }

`math.nan` is always itself, and thus this shortcut is taken and the key works.

chubot2• 4 years ago

Meh using floating point as keys for maps in any language is just asking for trouble -- it’s not just NaN

I don’t think there’s any real use case for it

I’d say clear() is good for clarity, and that’s it

mprime12• 4 years ago

I don’t think there’s any real use case for it

Histogram where you want to keep a count for each float value you’ve seen (maybe rounded to some precision to reduce the number of buckets)

Not disagreeing with your comment, just saying it’s not uncommon

iudqnolq3• 4 years ago

Why not just multiply each key by ten to the power of your desired precision, then convert to int?

If you're rounding anyway it's basically the same amount of work.

chubot2• 4 years ago

Hm I don't agree that example means it's "not uncommon" -- why would you want to hash the float rather than the bucket number?

Hashing the floats themselves isn't going to give you a useful histogram.

Is there a codebase that hashes float to a useful effect? If it exists, I'd be interested to see it

kazinator5• 4 years ago

This looks like catching up to GNU Awk, which allows "delete a" to clear an associative array. That's an extension over POSIX, which specifies "delete a[key]".

karmakaze• 4 years ago

Usage of NaN as hash key reminds me of the two uses of NULL in SQL. One is as unknown values which are never equal to anything even other NULL values. Another use is as a key for aggregate grouping. In that case the entry represents the aggregate for all the unknown values which aren't equal but still grouped together. Different uses have different meanings so invalid in one use does not invalidate other uses.

kazinator5• 4 years ago

This for loop is less efficient than clearing a map in one operation

That is not obvious; a compiler could have a pattern match for that exact AST pattern, and transform it to a delete operation. (Except for that pesky issue where two fail to be equivalent due to NaN keys.)

Quick and dirty, not entirely correct proof-of-concept in TXR Lisp:

  1> (macroexpand '(my-dohash (k v some.obj.hash) (print [some.obj.hash k])))
  (dohash (k v some.obj.hash
           ())
    (print [some.obj.hash
             k]))
  2> (macroexpand '(my-dohash (k v some.obj.hash) (del [some.obj.hash k])))
  (clearhash hash)

Impl:

  (defmacro my-dohash ((kvar vvar hash : result) . body)
    (if-match @(require ((del [@hash @kvar]))
                        (null result))
              body
      ^(clearhash hash)
      ^(dohash (,kvar ,vvar ,hash ,result) ,*body)))

coder543• 4 years ago

https://news.ycombinator.com/item?id=33673792

anontrot• 4 years ago

Hold on a sec, the whole clearing map scenario. Is it for reusing within your fn or is it trying to free memory? If latter, I’ve just been hoping GC does its job

yencabulator• 4 years ago

I'm more surprised they went with a new built-in instead of making the compiler recognize something like

  m = make(map[A]B, cap(m))

just like it already recognized

    for k := range m {
        delete(m, k)
    }

and many other similar idioms.

(cap because len is not quite the same -- note, cap is not currently defined on maps)

EDIT: Likely because that's an assigment on m, not a mutation of it, so it can't be done e.g. in a function that gets m as argument.

Go 1.21 may have a clear(x) builtin

Related Stories