HN user

tfm

287 karma
Posts4
Comments85
View on HN
Fast Python loops 9 years ago

The idea is to reduce the amount of redundant copying of characters: you end up doing a few more concatenations in the outer loop, but the concatenations in the inner loop are of short strings.

Importantly, if you remove the restriction of the input list being "exactly 256 items", then the method is still quadratic.

A linear-time algorithm for this would copy each input character exactly once, which is effectively what the method based on array.tostring() does.

The chunk size of 16 is not as significant as the technique of constructing+concatenating chunks, although it is optimal for input length 256. In general I think you'd want a chunk size about the square-root of the expected input length, to minimise the number of copied characters.

EDIT: maths

Concatenating strings of length M and N is linear in O(M+N), because that's how many characters you're copying.

Number of characters copied if you construct a string of length N by concatenating one character at a time

  = (0+1) + (1+1) + (2+1) + (3+1) + ... + ((N-1)+1) + (N+1)
  = (N +1)*N / 2
Number of characters copied if you construct a string of length N by concatenating a chunk of length 16 each time
  = (0+16) + (16+16) + (32+16) + ... + ((N-16)+16)
  = 16*(0+1) + 16*(1+1) + 16*(2+1) + ...
  = 16 * (N/16 +1)*(N/16) / 2
         ^^^^^^^^^^^^^^^
This is where the "technique" comes in: although the algorithm is still quadratic you're effectively moving a constant factor out the front.

Note that you also have the cost of constructing the chunks each time, which becomes the dominant cost if you have too many chunks.

In general, if you have a length kM string which you construct from k chunks of length M, the number of characters copied

  = M * (k+1)*k / 2  +  k * (M+1)*M / 2
... which (rounding and integer constraints aside) is minimised for M = k, i.e. when the chunk size is the square root of the input length. Hence, for input length 256 we take chunk size 16.
Python for Android 9 years ago

Yes, but with the caveat that you'll be running interpreted .py using a cross-compiled Python executable, rather than Python compiled to Android bytecode. Kivy itself (the project that spawned p4a) is a full cross-platform library for UI development; for any heavy computational lifting you'd want to farm it out to a C API library and import it the usual way, modulo some p4a recipe incantations.

The new £1 Coin 9 years ago

Fortunately the domain name is rather less ambiguous and primed me for the weak pun. I'm ... glad(?) that the wordplay survived the marketeering sessions and civil service vetting.

BTW: dodecahedron is the solid object, dodecagon is the 12-sided polygon. D&D style dice as currency would be pretty awesome but it might make it difficult to resolve dilemmas using pocket change.

The bit that really stood out for me: """ He was never once bored. He wasn’t sure, he said, that he even understood the concept of boredom. """

Almost seems (too? as a consequence?) that he was lacking a type of curiosity, whether as an existing character trait, or something that developed from having to periodically use all his wiles to obtain the means of survival when surveilling the cabins.

This is why I think the ability to ask good questions is a better indication of understanding and intelligence than the ability to generate answers.

Certainly! Synthesis rather than reformatting (or, more commonly, regurgitation). Analysis and abduction are more than just "put it in your own words". More useful too.

There is something of a rush on at the moment to generate chat-bots to replace FAQs. Every Slack/Fleep/Blern/Crank channel appears to have five or six memoisation bots. Seems to be largely a solved problem!

When we can start having bots that can be sensibly interrogated for a summary (or even a "hey, you've been away for several hours: here's the key points"), we can finally abandon the chatrooms and let the generative bots flood them with abductive content, and the precis bots can then ping you every couple of weeks when something important comes up.

If we're limiting ourselves to deductive reasoning, then yes – the facts as stated do not give enough information to deduce that Greg must be white.

If instead we use abductive inference, we might seek the simplest and most likely explanation given our universe of observations. Sherlock Holmes was a big fan of abduction!

Much of real-world reasoning is abductive to a greater or lesser extent. There is a well-known joke about some motley band of engineers, logicians, mathematicians, statisticians, etc etc catching a train through the Highlands. They see a black sheep, the engineer says "look, all sheep in Scotland are black!", the statistician says "no, you can't say that – just that MOST sheep in Scotland are black", another says "no, we can only say that at least ONE sheep is black", another says "no, it's only black on at least one side", then the one you're stuck next to at the party says "you're all wrong, we can only say that at least one sheep in Scotland is black on at least one side at least some of the time". The last statement is fully deductive; the rest of them are abductive, and more-or-less useful.

In this particular case it could be argued that Google's unofficial "don't be evil" motto explicitly framed this as a moral issue. Tongue-in-cheek, to be sure, but certainly an appeal to happy squishy emotions.

That said, RMS's stuff is showing up in a few posts today, and framing as a moral issue is certainly a large part of his standard toolkit. Considering any situation from an ethical viewpoint tends to make my eyes roll and skin crawl (ooh, the moral dimensions of public hygiene? yes please!), but appealing to hearts and minds is pretty basic old-school rhetorical technique.

If nothing else, coming from a technical background, it's occasionally useful to come at the problems from a different, squishier, direction.

Lena handled it very gracefully. She gives a very quick synopsis of what her part of the talk would have been around the 41 minute mark.

The whole presentation was plagued by gremlins; the substance begins with Sönke Iversen's part of the talk around 13:45.

Inasmuch as there is blame to be apportioned in this case, it's due to JavaScript / ECMAScript having broad definitions of acceptable variable names, and (arguably[0]) the fact that browser JS implementations will generally accept arbitrary 8-bit data within multiline comments, rather than the strict Unicode code units specified by ECMAScript.

JPEG comments exist for the same reason that EXIF tags exist – it's handy to store metadata alongside the image data, it gets copied around when the file gets copied, the tags can be transferred if the image gets re-encoded. There are enough error recovery mechanisms built into browsers that one could likely make a polyglot by just abusing the data segment, maybe even while crafting a legitimate standards-compliant JPEG.

Ultimately, bytes are bytes! Interpreting them with a variety of content types can give a variety of results, so keep it in mind.

[0] Resynchronisation / recovery from bit errors is one of the explicit motivations behind the design of Unicode encodings, so the browsers get a pass from me on this one. It's almost certainly possible to craft a suitable JPEG using legitimate code points anyway.

Alas, there's no guarantee that the order of key/value pairs within an object will be preserved (not even within JS anymore!). But that's okay, we can always reformat the ordered data as an array or write a custom parser ... oh.

I don't see any reason to include parsing directives in JSON either, but it's a wild world out there and people do all sorts of strange things. Seems that a few of those folks were working at Yahoo and made the mistake of letting Doug see their code when JSON was in prototype phase, so no JSON comments for anyone. Whoops!

Perhaps it was determined by trial and error! Or maybe the earlier oven iterations happened to get lucky. We don't know that embedding a flyscreen on the door of the food chamber was necessarily the look that people would have gone for!

Now of course there are statutory "radiation leakage" limits in most markets. One might easily imagine that engineers would take a few goes at implementing the Ezy-Look-Into oven sketched by the folks over at industrial design, measure the emissions levels with thinner wire screens, and shrug it back with several binders full of readings. When experiment and theory are in disagreement, the product manager is unlikely to fund too much research into picking apart Maxwell's Other Equations. Given the known configurations for "good enough" microwave shielding, presumably the design team gets to sign off on a suitable colour instead of insisting on mechanically-etched glass impregnated with nanowires.

Official local datum is GDA94, which is usually within about a metre of WGS84. /me pats pocket protector

I think the idea here though is that the underlying terrain has scooted around a bit in the last twenty years, so a new datum would save on the need to print new maps and update the databases containing trig points and such. New coordinates can be labelled as explicitly aligned to the new datum, get some versioning happening.

Unfortuately it seems impossible to confirm this, because the Geoscience Australia website does not appear to make any reference to this purported new effort that's been making the rounds of late. New website layout, maybe their CMS is broken.

Pokemon Go API 10 years ago

I'd certainly agree that someone who went to the effort of setting up a system for spoofing (even if it was just downloading an extra app) is, in some sense, more motivated than a very casual player.

I don't think though that Niantic have much of a moral hazard to consider here. Looking at what's purchasable in the Pokémon store, there's nothing that would be attractive to anyone who was able to virtually wander the world at all hours from the comfort of their couch, especially since anything that can be bought with cash money could be obtained using coins earned in-game. If a player's motivation in spoofing was to "catch 'em all" by whatever means necessary, it seems unlikely that they'd draw the line at restocking from Pokéstops along the way.

Comparing with the dark side of Ingress, there is a ludicrously well-organised black market economy offering purchases for every in-game commodity – all, of course, completely against the T&Cs, all completely abhorred by legitimate players, but all offered with consummate professionalism (think of the slick ransomware scammers offering a support number). Niantic don't see any of that cash. It is likely to have had a major impact on their design decisions for the PoGo store, and the game in general. If, for instance, there is no way to trade items between players, then it severely limits the options for a parallel economy.

Pokemon Go API 10 years ago

Probably good to regard these first few weeks (months?) of Pokémadness as an "open beta" period, before the security measures get turned on. We can look at Niantic's previous project, Ingress, for a roadmap.

The two major categories of cheatifying in Ingress are falsifying one's location and multi-accounting. There's precious little that can be done about the latter, so Niantic focus on banning players that appear to be "spoofing" their location.

Given the wealth of different devices and playing scenarios, immediate detection of GPS spoofing is infeasible. Things like WiFi router locationing idiocy (or even just dodgy GPS antennae) play havoc with the utopian dream of perfect positioning every time. If a player performs actions seconds apart that are separated by thousands of miles then the game temporarily ignores them, but after some time in the naughty corner they can resume play.

Hardy spoofing detection instead depends on longer-term profiling. Ingress has a similar API to Pokémon Go – JSON chunks (rather than protobuf) over HTTPS, most fields out in the open – but each request from the app includes a monolithic "clientBlob" containing device characterisation. The format of this has been (presumably) reverse-engineered by a few hardy souls but it is certainly closely-protected Niantic knowledge. We could safely assume that it's a proprietary blend of signal strengths, gyroscope readings, touch events and timings, secret herbs and spices etc.

The clientBlobs lend themselves to offline processing. There are conceivably servers continuously trawling through a backlog looking for tell-tale patterns of bad behaviour, but it also provides an audit trail if a particular player is suspected of spoofing. Occasionally Niantic indulges in mass purges, which presumably follow from a new cheat detection heuristic being run on all the collected data for some period. These "ban waves" have a reputation for penalising unusual device configurations (the most recent major wave appeared to target, amongst other things, players with modified Android variants that might mask GPS falsifying code, including cheaper Chinese knock-offs, and Jolla phones running Sailboat).

Occasionally during major Ingress gaming events – so called "XM anomalies" – there is some level of human supervision to quickly identify and remedy clearly-fraudulent player behaviour, but for day-to-day operations it seems that account termination, so-called "hard bans" and shorter-lived "soft bans" are entirely automated, and based on offline player data analysis.

Getting back to the New Cruelty: the clientBlob was not part of Ingress's initial implementation; for a while after it was introduced was ignored, and then it became mandatory. A similar opaque chunk of data is included in the Pokémon Go requests, so we should look forward to its imminent deployment when Niantic scrape together enough Pokécoins to buy a few new servers for batch processing. At that time these convenient APIs won't have long to live.

It takes real skill to walk in a straight line, even with a compass. The best navigator I know taught me the trick of lining up good landmarks and concentrating on those, head straight for them, then choose another, but by crikey it takes effort to do it even halfway decently.

This would almost certainly have been a case of answering the call of nature, going a very little way off-track to preserve decency, getting ever so slightly turned around to lose the trail, then an escalating series of "corrections" as it becomes clear that it's all going wrong. It's a very sombering and saddening thought.

Well, you'll need water, which generally finds itself at lower ground ....

Are folks not taught stuff about lighting smokey fires to attract attention, or would there have been local factors that prevented it in this case?

At CDG airport a few weeks ago we encountered a straightforward process I'd not seen before – items going through the screening are associated with your boarding pass. It had no significant effect on the speed of the procedure, and it provides accountability for objects. I don't fully understand why it's not standard practice.

Fairly warned be thee, this is a Jonathan Franzen essay, so it's not everyone's cup of tea, but if you like lengthy light prose peppered with his usual shout-outs to degenerative diseases you're in luck.

Franzen went on a luxury cruise around the Falklands, South Georgia Island and the pointy bits of the Antarctic Peninsula. He is a keen bird-watcher. He has eyes of glass and the heart of a tape recorder. He found Antarctica beautiful and finds the notion of global warming slightly upsetting, but life goes on. The cruise was a bit over three weeks long, but he has known his family for much longer than that so they feature proportionally.

There are no photographs of Antarctica because he resolved ahead of time not to take any photographs. He did not make it to the end of the world, nor does he substantially discuss the end of said end.

RIAA may not be involved in this particular tilt.

Having radios everywhere is obviously always in the radio industry's best interests. Back in 2010 the music industry got involved because they were trying to wring extra performance rights fees out of radio, which the radio folks clearly were not interested in, but the proposed compromise was that music industry lobbyists would throw their weight behind the "radios everywhere" thing, and radio would presumably benefit enough from that to cover the new hefty performance fees.

The Performance Rights Act didn't go anywhere, looks like music industry tried again last year with the "Fair Play, Fair Pay" bill.

It was not so very long ago that the National Association of Broadcasters and the RIAA were lobbying Congress for the inclusion of said FM chips [0] (or [1] if you want the tldr version).

NAB is involved [2] in this latest heroic campaign for consumer choice too, of course, although you wouldn't know it to look at the website [3].

[0] https://arstechnica.com/tech-policy/2010/08/radio-riaa-manda...

[1] https://www.penny-arcade.com/comic/2010/08/18

[2] http://www.prnewswire.com/news-releases/industry-wide-consum...

[3] http://freeradioonmyphone.org/about/

It's short for "pork barrelling", which refers to public money that is allocated by government to projects in specific areas to sweeten the local voters. Typical examples might be the announcement of new roads or sporting facility, or the awarding of a large government contract to a major local employer, especially just after an election has been declared.

The term itself was not originally pejorative (a barrel full of pork can feed a lot of mouths!) but it is generally used negatively by the press or cynically by certain government sectors.

From a logical standpoint, standardising language certainly works as an idea, but humans are not so logical! The history of language is that of languages merging and diverging. It kinda depends a lot on who you want to talk to at any given time.

It's not clear that language is inseparable from culture; there are arguments to be made that language shapes thought (see "1984" for an extreme thought experiment!), and culture might well be shaped by the network effects of such frameworks.

As for cultivating a world wide culture, perhaps we can already see it happening! And perhaps global language would follow. A few years back there were great claims made that in a century we'd all be speaking so-called "Panglish", the sort of global pidgin that would emerge from mashing lots of languages into English. Maybe ... maybe not. If there were a "standard language", chances are it would be at least partially based on English, and supplement local languages. But it depends entirely on whether everyone really cares about understanding each other, to the extent of learning a common tongue.

Short answer would be availability of tagged data, in turn dependent on historical research, which is a laggy history of who's been funding the research.

This is changing, thankfully; check out e.g. the European Language Resources Association's catalogues, http://catalog.elra.info/ or any of the very incomplete list on wikipedia: https://en.wikipedia.org/wiki/List_of_text_corpora or in fact the various international Wikipediae themselves!

As for this particular project being English only ... itch to scratch probably ;-)

The software worked remarkably well in this instance because the system was well constrained, and the students weren't on the lookout for it. If they do start quizzing their new unseen TA (Dr Colin V Olutionalnet) on random trivia it could presumably deflect the query or have a few canards pre-programmed.