kgm

I have used id() in production code, but the circumstances were extremely specific: It was in some monitoring-related code that was trying to dig down into the guts of the interpreter for the purposes of tracking memory usage on the level of individual pages. It was a whole thing.

Which is to say: Users of Python almost certainly don't want to call id(), unless they are mucking around with something related to the functioning of the interpreter itself. The only other circumstance in which I've used it in anything like real code was in a quick hack where what I really wanted was to put a bunch of dicts in a set, but you can't do that, so I made a set of their IDs instead (but this wasn't in what I would call "production code").

In general, most of the "wtfs" listed here are pretty banal. If you're using `is` on immutable objects, you're basically just asking for trouble. The functioning of string and integer interning and constant folding are implementation curiosities, and if you ever write code in which such differences matter, then you have made an error.

When it comes to profiling in Python, never underestimate the power of the standard library's profiler. You can supply it with a custom timing function when instantiating the Profile type [1], and as far as the module is concerned, this can be any function which returns a monotonically-increasing counter.

This means that you can turn the standard profiler into a memory profiler by providing a timing function which reports either total memory allocation or a meaningful proxy for allocation. I've had good results in the past using a timing function which returns the number of minor page faults (via resource.getrusage).

[1] https://docs.python.org/3/library/profile.html#profile.Profi...

My own experience was with the 3D-enhanced version of Mechwarrior 2: Mercenaries, specifically. It is possible that the equivalent version for Mechwarrior 2 was more stable. It's also possible my 3D drivers of the time were the cause of the issues I had, but Mercenaries was sufficiently buggy in all sorts of other ways (even the DOS version loved to crash) that I always just blamed the game.

Part of the issue there is that the 3D-acceleration-enabled versions of Mechwarrior 2 (and Mercenaries) were buggy as heck even at the time. (Mercenaries was even pretty buggy before they did the 3D card patch, and it only got moreso.) It really doesn't help this kind of games preservation that the games themselves could be kind of junk, from a technical standpoint.

There's a whole generation of games from the mid to late 90s (and perhaps into the early 00s) that's remarkably annoying to play these days because of this issue. The game I've personally tried to get running, every once in a while, is Mechwarrior 3, but it's a complete disaster. The last time I tried to run it, I actually managed to load into the first mission, but the in-game physics were remarkably broken, with the amusing result where the first enemies you face in the game (a couple of little tanks) drove up towards me, hit a little bump, and then immediately rocketed into the sky.

At least part of the issue is that the game uses multiple threads, but was designed for systems with a single CPU, with a clock speed that's considerably slower than is present in modern systems. Something about this difference in timing breaks the whole thing in ways which are diverse and inexplicable.

Now, this comment thread contains plenty of possible solutions I could attempt, but if it's really a matter of the game relying on something like the CPU speeds of contemporary hardware (not to mention contemporary graphics hardware) then I start to think that I'd need to track down some kind of Pentium 3-era gaming PC to really make it work.

I am taken a little aback by the name. There is, of course, already the D programming language, which has even had a major version number of 2 for quite some time.

It also brings to mind the JavaScript library d3, which, while not strictly for making diagrams, can easily lend itself to the purpose.

Calling this thing "D2" seems potentially confusing.

All right, behold my idiotic story about the time I got a job for one of the major tech companies.

It's the late 00's. I am in my early 20s. I have an associate's degree from the local community college, and I have been messing around with programming since I was 8 years old. I am unemployed, having not long before been laid off from a minimum-wage job at a failing retail establishment. I am living at my parents' house, spending most of my time messing with code and hanging out on IRC.

I am contacted on IRC, one day, by a recruiter for one of the major tech companies. He tells me that I sound like I know what I'm talking about, and would I like to interview for a job?

Well, yes, of course I do.

This major tech company has an office in a city a few hours' drive from where I live. I go there for the interview process, which is somewhat grueling.

I don't get the job. I had a brain fart and whiffed some basic question about computational complexity; I'd forgotten what O(n log n) means. So it goes.

Perhaps a week later, the recruiter calls me back, saying that, out of the blue, a different subsidiary of the major tech company would like to interview me. This subsidiary is an acquired startup, a major website in its own right, and would I be interested in interviewing there?

Well, yes, of course I am.

This subsidiary's office is located in the Bay Area, which is a somewhat greater journey, but I fly there and do the interview process again.

I got the job, that time. I aced the interview, in fact. Pro tip: It is a good sign if you manage to blow through all of an interviewer's prepared questions, and force him to resort to asking riddles, and then manage to answer the riddle correctly, too.

It was only later, after I started to work there, that I learned the full story. The website's ops team needed to hire someone. Although this subsidiary was owned by the major tech company, it was still in many ways a separate company, not yet assimilated into the greater corporate entity. Thus, they basically just walked over to HR, and asked to poke around in their resume database.

They did a simple keyword search, and my resume popped up. It was a total coincidence. One of the keywords they used was the programming language they used, which I had also happened to use in some open-source stuff. Another keyword was related to the subject of the subsidiary's website, which, entirely unrelatedly, also happened to be a word used in the name of the retail job I had recently been laid off from.

My resume was very nearly a blank sheet of paper, aside from these things. I was told that the site's lead architect, on seeing it, reacted along the lines of, "Oh, we have got to interview this guy."

So they did, and I spent the next several years working there.

This is an incredibly stupid story. There is absolutely no part of this process which I would point to as advice for other people. To the extent that this has led to my success, it was blind, stinking luck, and I doubt it would ever happen again.

I'm not sure what moral you could take from this. "Know your shit" seems patronizing. I would like to think that possessing technical know-how with no relevant degree, certifications, or experience is still a state which can lead to success, but I suspect that such extraordinary luck is still a necessary component as well.

For another breakdown of the details of floating-point precision, I recommend qntm's "0.1 + 0.2 returns 0.30000000000000004":

https://qntm.org/notpointthree

"Mars was empty before we came."

Red Mars, Kim Stanley Robinson

I would also recommend https://www.joelonsoftware.com/2003/10/08/the-absolute-minim...

I once wrote a popular browser-based tool for a popular game. Other tools existed in the space at the time I started the project, but I felt the need to go at the problem my own way, and I managed to add some mathematical sophistication that the other extant tools lacked. I did all of this purely for my own benefit: I was playing this game, and I wanted this tool to exist to aid in doing so.

I threw it online for other people to use, and it quickly gained traction. I'd estimate that it has had over ten thousand users, possibly multiple tens. Some of these users expressed, unprompted, an interest in giving me money for providing this tool. I simply set up a page on a payments website that caters to creators, and linked it from the tool. I provide no benefits for contributions; it is purely to provide an avenue for those that wish to do so.

If you total up the amount of work I've put into the project, and the total amount of money I've received, I would estimate I've earned somewhere in the vicinity of $1/hour for my work.

I didn't do the project for the money. I did it because I wanted the thing I made. It is an entirely selfish project. That others find it useful is gratifying, but the motive wasn't profit, and to try to pivot to a profit motive would be nonsense to me. (Not to mention, I don't want to insult the developers of the game for which the project is an aid, by charging money off the back of their work.)

Some things just aren't businesses, you know?

There is no sense in which English pronunciation can be described as "consistent." As supporting evidence, may I introduce the poem "The Chaos":

https://ncf.idallen.com/english.html

I just finished a re-read of the "Commonweal" series by Graydon Saunders, which begins with The March North:

https://play.google.com/store/books/details?id=MoIOAwAAQBAJ

I've never been sure how to sum up these books. They're thoroughly strange fantasy books, oddly written and frequently chewy reads. They have an interest in politics and metaphysics; but they also have magical railguns, the sorcerous application of FOOF [1], and a five-ton battle-sheep named Eustace.

The first book is fairly straightforward: A quiet backwater of the Commonweal is under threat, and its only defense is an understrength territorial battalion, a handful of experimental artillery pieces, and three of the mightiest sorcerers of the age.

Then the second book (A Succession of Bad Days) isn't at all about the military, and is more like the weirdest going-to-sorcery-school book I've ever read. It also has an extended, detailed section on using sorcery in canal construction. If you ever wanted a book about the best ways to use magic in the service of civil engineering, this series is your jam.

These books really aren't for everyone, but I kind of love them, and they aren't widely known, so I'll always take an opportunity to shill them.

[1] https://www.science.org/content/blog-post/things-i-won-t-wor...

One thing to note, in the gritty details of Python's actual implementation of the list type, is that there's a little more to say about how list resizing works than is immediately obvious. The relevant source is here: https://github.com/python/cpython/blob/main/Objects/listobje...

As the article points out, the size progression, as detailed in the comment in that source, is relatively modest. Your "classic" dynamic array will simply double the size as needed. This progression does less than that: 16, 24, 32, 40, and so on. But: The underlying implementation is using the standard C library function realloc(). This will perform some amount of in-place resizing on its own, and will lend a lot to the performance of a Python list, albeit in a way that might be difficult to nail down, since it depends on the overall usage of the C library's heap.

The standard array module was more relevant in the distant past, before version 2.6 added the bytearray type. Otherwise, numpy arrays are considerably more useful in virtually every circumstance.

Yeah, but we're not talking about pure mathematics. We're talking about floats, and I find that it's very important to be clear about the limitations. It's easy to get some nasty bugs if you start assuming that you can cram just any int into a float.

And I have no objections to the article's description of the bool type.

This is a neat article, but it does have some errors.

One subtle point that the post gets wrong:

So where does that come from? The answer is that Python stores everything inside dictionaries associated with each local scope. Which means that every piece of code has its own defined “local scope” which is accessed using locals() inside that code, that contains the values corresponding to each variable name.

The dictionary returned by `locals()` is not literally a function's local namespace, it's a copy of that namespace. The actual local namespace is an array that is part of the frame object; in this way, references to local variables may happen much more quickly than would be the case if it had to look each variable up in a dictionary every time.

One consequence of this is that you can't mutate the dict returned by `locals()` in order to change the value of a function-local variable.

Another, less-subtle error in the post is this:

int is another widely-used, fundamental primitive data type. It’s also the lowest common denominator of 2 other data types: , float and complex. complex is a supertype of float, which, in turn, is a supertype of int.

What this means is that all ints are valid as a float as well as a complex, but not the other way around. Similarly, all floats are also valid as a complex.

Oh, no no no. Python integers are arbitrary-precision integers. Floats are IEEE 754 double-precision binary floating-point values, and as such only support full integer precision up to 2^53. The int type can represent values beyond that range which the float type cannot.

And while it is true that the complex type is just two floats stuck together, I would very much not call it a supertype. It performs distinct operations.

Accessing an attribute with obj.x calls the __getattr__ method underneath. Similarly setting a new attribute and deleting an attribute calls __setattr__ and __detattr__ respectively.

Attribute lookup in Python is way more complex than this. It's an enormous tar pit, too much so to detail in this comment, but __getattr__ is most often not involved, and the `object` type doesn't even have a __getattr__ method.

The podcast Behind the Bastards has a two-part episode on the history of the US border patrol titled "The U.S. Border Patrol Is A Nightmare That Never Ends," which I listened to just recently. It is entirely horrifying, and this report is entirely consistent with the sort of behavior I'd expect from ICE. It has always been thus, for as long as the border patrol has existed (which has not been forever; this did start somewhere).

I recommend giving it a listen if you want more context on the history and current state of how the US polices its borders (especially its southern border).

Part 1: https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkcy5tZWdhcGh...

Part 2: https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkcy5tZWdhcGh...

You joke, but Factorio is a sufficiently "crunchy" logistics game that it has prompted me to study a number of serious topics, from learning linear programming to determine optimal production ratios, to dipping my toe into learning about the broader subject of systems dynamics, with stocks, flows, feedbacks, and so forth. I maintain that it's an excellent educational tool for these sorts of subjects.

I taught myself linear programming because of Factorio. It is easily the single game in which I have spent the most time, probably close to 4,000 hours. It is delightful to see that the team is continuing to improve it, even after the 1.0 release.

I am seeing other analyses that put the size of the explosion at about one tenth of this size.

The analysis in this Twitter thread correlates the observed degree of damage with the distance from the explosion. It comes up with an estimated 240 tons TNT-equivalent, though naturally this comes with a considerable margin of error.

https://twitter.com/GeorgeWHerbert/status/129071971954515968...

This Twitter thread is making the point that the explosiveness of ammonium nitrate can vary significantly depending on how it is stored. 2750 tons of AN at 20% efficiency gives 550 tons TNT.

https://twitter.com/ArmsControlWonk/status/12907955327014256...

This range is also apparently consistent with the explosion registering as a 3.5 seismic magnitude.

https://twitter.com/ArmsControlWonk/status/12907348507695267...

There are still a great many unknowns, and none of this should be mistaken for the real analysis which will come later, but these disparate sources of information all seem to agree with each other, to a first approximation.

This is clearly about Python, so I will note that this is not actually the case: The generator expression introduces a new scope, and so it does not reassign the value of x. Python does not require the "global" and "nonlocal" keywords in order to access variables, but to reassign names brought in from an outer scope.

Python 2 did have the issue where list comprehensions (though still not generator expressions) didn't introduce a new scope, and so you could indeed get issues quite similar to your example above. This was corrected in Python 3.

All that said, I will agree that Python's scoping rules are pants-on-head crazy. In very nearly every other programming language under the sun, you declare variables at the point where they exist. In Python, you declare a variable when it exists somewhere else (and you then want to reassign it). This does mean that you don't need to put a "local" or "var" or what-have-you in front of each new variable you declare (which is basically the reason Python did it this way), but the semantics of the thing can take some getting used to.

Though "when" the object is created isn't always so straightforward:

  >>> x = 257
  >>> y = 257
  >>> x is y
  False
  >>> def f():
  ...     x = 257
  ...     y = 257
  ...     return x is y
  ...
  >>> f()
  True

A few years ago, a Dr. R. Andrew Hicks of Drexel University designed a non-reversing mirror that is a single sheet of glass. The reflection is slightly distorted, however.

News coverage: http://techfragments.com/inventions-non-reverse-mirror/ https://www.newscientist.com/gallery/dn16585-amazing-mirrors

Photo from his website: http://www.math.drexel.edu/~ahicks/images/opticks.jpg

To be pedantic (which I think is warranted here), range does not return a generator, it returns a sequence called a range object. This object can be indexed, sliced, and (relevant to this discussion) supports the 'in' operator.

"x in range(10)" will operate in constant time and memory in Python 3. Whether it is actually more efficient than "0 <= x < 10" is another matter, of course. I would expect the call to range() to dominate here, and indeed, it is much slower in this microbenchmark:

  $ python3 -m timeit -s 'x = 8' 'x in range(10)'
  1000000 loops, best of 3: 0.351 usec per loop
  $ python3 -m timeit -s 'x = 8' '0 <= x < 10'
  10000000 loops, best of 3: 0.0732 usec per loop

The intention is to default-initialize values to things that are invalid. There's no obvious "invalid" value for a plain integer, so they get zero. But char gets 0xFF, because a char is defined to hold UTF-8 code units, and 0xFF is not a valid UTF-8 code unit. Similar logic extends to the larger character types, and to using NaN in a float.

The idea is that you should be initializing values explicitly before using them, and default-initializing to values which will rapidly fail makes it easier to debug use-before-initialization bugs than (for example) it is in C, where values are not initialized at all.

If you explicitly want to disable default initialization, you can use void as an initializer, as in:

int x = void;

A while back I hacked up a script for doing digraphs in tmux. It's not precisely equivalent to screen's digraph support, but it does the job and (given that it's just a Python script) can be easily extended.

http://paste.pound-python.org/show/6aCbTG3lKgCWdrRygkBG/

The built-in profiler can be told to dump the results of the profile to a "pstats" file. A simple way to do this is to run your code under the module itself, as in:

  $ python -mcProfile -o foo.pstats foo.py

Once you have a pstats file, you can do some interesting things with it. You can load it into an interactive pstats session with this:

  $ python -mpstats foo.pstats

You can also use the third-party gprof2dot script[1] to turn the pstats file into a GraphViz dot file, and thence into a visualization of your code's performance. For example:

  $ gprof2dot.py -f pstats foo.pstats | dot -Tpdf -o foo.pdf

[1] https://code.google.com/p/jrfonseca/wiki/Gprof2Dot

Some of these quirks are explained incorrectly. I find the real explanations interesting, so please allow me to provide them.

999+1 is not 1000

The examples given in this section do not actually show what the poster thinks! Python's interning of small integers is not relevant. It only applies to integers in the range -5 through 256.

The real explanation for why "1000 is 1000" evaluates to True has to do with the Python compiler. By evaluating this expression as a single statement in the interactive interpreter, the compiler notices that the constant value 1000 is repeated more than once. Therefore, it is able to re-use the same object.

But if you provide the value in more than one statement, the compiler is unable to do this:

  >>> a = 256
  >>> b = 256
  >>> a is b
  True
  >>> a = 257
  >>> b = 257
  >>> a is b
  False

  >>> def f():
  ...     a = 257
  ...     b = 257
  ...     return a is b
  ... 
  >>> f()
  True

Apparently Ellipsis is always “bigger” than anything, as opposite to None, which is always “smaller” than anything.

Not so. Ellipsis follows Python 2's default rules for the comparison of unrelated types.

  >>> Ellipsis < ()
  True

1) None is the smallest object. 2) Followed by numbers. 3) Followed by all other objects. Objects of distinct types are compared by the lexical ordering of their type names.

This can easily lead to senseless orderings, when two types define an ordered relationship between themselves, but another type happens to have a name that is lexically between them, as with str, tuple, and unicode:

  >>> "def" < (1,)
  True
  >>> (1,) < u"abc"
  True
  >>> u"abc" < "def"
  True

The actual use of Ellipsis has nothing to do with recursive containers printing an ellipsis in their repr. It's part of a wacky special syntax that exists for NumPy's benefit:

  >>> d = {}
  >>> d[...] = None
  >>> d
  {Ellipsis: None}