kurumo

Imagine I want to send a letter to someone, but I am worried that it may be intercepted (by the government, for the sake of argument). So instead of sending it directly to my recepient, I make arrangements with some reliable friends so that if they receive a letter from one of us, they take it out of the envelope and put it in different one, addressed to another friend. To make sure the letter doesn't travel forever we add dots at the end; once there are more than three (four, five...), we send the letter to the original intended recepient. Nobody knows for sure who is the person who sent the letter originally (as I can put different number of dots at the end of the letter). We can also make it so that none of the intermediate parties know what the message says (possibly even who is the final recipient?) by encoding the message at every step and using extra envelopes. At the outset I write my letter, add some dots at the end and send it to one to my friends, picked randomly, together with an extra envelope addressed to the intended recepient.

That should do as far as Tor goes, but the more general problem of explaining why is such a thing needed by your mom is much harder. (Unless she lives in North Korea or some such place).

While not free, 'Machine Learning: A Probabilistic Perspective' (http://www.amazon.co.uk/gp/aw/d/0262018020) is the best book I have found so far. I also second the recommendations for Tibshirani's and MacKay's books; the former for mathematical foundations, the latter for the intuition.

There is an IM in Boston who routinely gives people, mostly without regard for their rating, 30 seconds to 5 minutes odds with no increment. I have seen him win these controls against other masters. So comparatively it is not too surprising that Carlsen would be willing to accept these odds against an unrated amateur. Chess with this kind of time controls is basically a different game, which requires emphasis on different skills than do standard time controls.

That is simply incorrect. A GM can perhaps give a strong amateur (2000+) 1 to 5 time odds, maybe 1 to 7, in under a minute time controls. Larger odds would be arrogant, foolish or even suicidal, depending on the specific GM (rating, opening repertoire, etc.)

Fair use or not, if you could do it I would buy it :) Fine, forget conference papers. If you can demonstrate fast NER in multiple languages, across domains, with competitive precision/recall metrics, I will buy it. The rest of it is not particularly interesting to me because it's frankly not that hard.

Thanks, that's somewhat helpful. I am not particularly interested in the summarizer plugin itself (mostly because we have one, built in house), but I would love to talk about the underlying pipeline. If you have e.g. a named entity recognition library that performs as well as you say in Romance languages on standard data sets, you have material for at least one conference paper, and furthermore a product much more valuable than the summarizer itself.

My question about speed referred to syntactic parsing specifically. I am sure you can do entropy scoring faster than 200ms per sentence, but unless you have access to parses you are unlikely to be able to do more than purely extractive summarization. That's what Summly does, and every other summarizer on the planet as well. (Except perhaps Columbia's Newsblaster, but that's a bit of a different story).

What precisely makes this particular engine 'the most powerful in the world'? Does it do domain independent named entity recognition with an F score better than 0.8? For what classes of entities? Is it at least adaptable without oodles of training data? Does it do syntactic parsing? With F scores of 0.9 or better? Faster than 200ms per sentence? Across domains? Does it do anything at all in languages other than English? If there is a page on that site where it answers these types of questions I couldn't find it..

It's actually 1/6 * Sum_n (5/6)^2n = 1/6 * 1 / (1 - 25/36) = 6/11. The first move provides a small advantage, as intuition dictates.

It tells me if they can at least minimally apply their knowledge and reason about a problem. It's a filter question. If a person with a college degree in math cannot solve this, what's the likelihood they will be able to solve an actual real problem I need them to work on?

I have been using this when interviewing people with math degrees:

Two players play a game with a single six-sided die. The player that starts can only win by rolling a 1. If he or she doesn't win, the other player gets to roll; he or she can only win by rolling a 6. The game continues until one player wins. What's the probability the first player wins (eventually)?

We do (Bloomberg), but on equities, not commodities. We are working on it though :)

Now, how hard would it be to design a sort of minimal virtual machine that can run this in parallel with e.g. a Windows host OS? Distribute it via some existing delivery vector, et voila...

New York, NY; Skillman, NJ; London, UK; other locations.

Bloomberg is hiring, Intern, H1B candidates are welcome.

http://careers.bloomberg.com/hire/internsearch.html

http://careers.bloomberg.com/hire/experiencesearch.html

C++, Java, .NET developer positions are available, as well as network engineers, UI designers, sysadmins, etc. Lots of real time and low latency infrastructure work, generally financial domain.

That is not entirely correct. If you have already acquired credibility (somehow, doesn't matter how for the purposes of this exercise), your estimate or advice can (and frequently does) become a self-fulfilling prophecy. Knowing that a significant number of investors will follow your advice gives you information which can be used in the market. It then stands to reason that the most profitable thing you could do would be to give this advice to the maximum possible number of people.

29,500, non-native English speaker (but studied in the US). Retook the test and omitted all the words I could not define with total confidence on the spot; the original score was 31,400. The test is peculiar in that the distribution appears to be uneven. Subjectively there is a sharp break between words that one would know from Shakespeare, Tolkien and Dunsany, and words no one would ever know unless they studied the OED. For statistical significance they would need more words.

Cold War 2.0 China edition.

I think this is the original article: http://www.nejm.org/doi/full/10.1056/NEJMoa0802905

Not bad at all as far as interface goes, but their timer is buggy. In 5 0 it ate 15 seconds of my time, apparently due to lag. A thought I had for a while: do analysis on games as they occur and try to estimate opponents' strength, as a way to detect cheating of the type where one of the players mimics a computer. Computationally expensive, but would be fun to try.

http://senseis.xmp.net/ is not bad

Bloomberg, NY, London & elsewhere.

News R&D is looking for people to work on a real time news search engine. Great place to work on interesting problems.

http://careers.bloomberg.com/hire/jobs/job27500.html http://careers.bloomberg.com/hire/jobs/job27499.html

Also,

http://careers.bloomberg.com/hire/experiencesearch.html

Is nobody bothered by the prices of the actual books?

Robert Anton Wilson

Schrödinger's Cat trilogy especially

The full list: http://careers.bloomberg.com/hire/experiencesearch.html Filter to R&D. We are mainly hiring in NY, NJ and London. It's a great place to work. Regardless of the problem domain you prefer, there is something interesting to do.

I work in this field. That is not the first paper written about this system; the previous was "Sentiment Analysis of Financial News Articles", Schumaker et. al. This is not the only academic project describing something like this, see papers by Mittermayer and Lavrenko for other examples. Short summary of the results is basically this: you would have to be insane to trade purely on output of a system like this. There are multiple issues with this, the primary being that most of these systems do not and cannot distinguish documents leading the trend from those lagging the trend. Furthermore, to actually be able to trade on the indicators derived from news you need to encode the market expectation into your model, which is a separate (and difficult) problem. The margins these systems produce in simulated trading are small, and for some reason nobody takes transaction costs into account. This is not to say that systems like this do not exist in the real world - they certainly do, but the people that build and/or use them generally will not discuss the details, for obvious reasons.

Generally speaking, CS programs out there seem to lack both depth and breadth, be out of touch/time with "the desert of the real", and suffer from what Dijkstra called "the infantilization of curriculum". Too often do I see people who are proficient (for some value of "proficiency") in one (1) language, usually the wrong one. I believe that a CS program ought to introduce you to several languages, to a level where you can read and write programs of small to medium size (~1 kloc). At a minimum, in this day and age I would expect C, your least favorite mainstream language (though I personally would avoid Java, due to combination of a lack of an explicit reference type and a misguided OO fetish), something from the Lisp family (Scheme), and Javascript. Something from the ML family would be a bonus, though I would skip type theory altogether. Having been exposed to several radically different languages helps. I would ask everyone to write an interpreter for a simple language (in a language of your choice): something with variables, conditionals and function calls. A compiler for the same would be a nice next step. As projects go, this would be sufficiently involved to address Dijkstra's complaint part of the way.

A few people said something to the effect "the real world is missing". It's true. Majority of CS programs appear to be years out of date with respect to the industry practice, whatever "the industry" happens to mean for you personally. To some extent this is inevitable, though we could be doing a lot more to address the issue - say, as someone suggested, via a separate course, one that is updated biannually. A course like this could and should include a discussion of things like source control tools, build tools, etc. Things like resource management idioms in languages of the day should be discussed explicitly: too often graduates of CS programs assume infinite resources or automagical cleanup, especially if they come from a background in languages that are garbage collected. A course in parallel programming presented via different languages and their approaches to parallelism would be fantastically useful; I would pay money to see it on OCW or similar.

Somewhat related to Dijkstra's complaint is the following concern: people are terribly bad at thinking about complexity, at all levels - from a single source file to a system composed of hundreds of separate processes. We would do well to discuss managing complexity explicitly in CS curriculum: to make people think about difficulty of maintenance, changes to a working system, rollout procedures for uptime, etc. A seminar on the subject of complexity management, from small to large scale, with examples, would be very useful.

Incidentally, this is something I have been thinking a lot about over the last two years, in part due to conducting a metric shitload of interviews for intro positions. There are things that my CS program was missing, and things that a majority of programs out there seem to be missing; understandably, what I can say mostly applies to the former.

http://www.ted.com

http://www.wikipedia.org

http://ocw.mit.edu

Thanks, I will try it. Application-level solution would still be preferable, I think, and is an interesting problem, aside from the immediate usability benefits.

I have the Pogo stylus. The problem with it, or rather with ArtStudio, Sketchpad, Brushes and all the stylus note-taking apps is the same: you cannot rest your hand on the screen. It registers touches from the hand instead of the stylus. That makes all of these apps pretty much useless as far as I am concerned: it is quite inconvenient to draw or write using these applications, even though some of them are extremely impressive otherwise. This, frankly, boggles my mind - granted the screen sensor API is closed, but is it not possible to do velocity distribution thresholding on different touch events, or maybe something more sophisticated, a Kalman filter or something, to distinguish these touches and ignore the 'stationary' ones?

I guess Chinese Newspeak doesn't have "freedom" in it. And I thought 1984 was bad :(

This is actually very interesting. It may have been led by China. Chinese market fell on Monday due to reserve requirements change (though the slow decline actually started quite a bit earlier), and it was widely reported on Monday night. Commodities followed on Chinese news and European debt on Tuesday; Europe continued falling on Wednesday on Greece and Portugal speculation, then this went around the world one more time and was reported at about 3am today (emerging markets decline), and then later in the day as decline continued. Speculation on European state debt compounded the issue (witness the Spain rumours). Then at 13:10pm El Erian was reported to have said that there is a possibility that European banks will stop lending, at which point the US markets hiccuped and went into accelerating decline (look at Russel, SPX and Dow from noon onward).

I will try to add links when I get home later. Clearly it was much more complicated than this, but this pattern of falling dominoes going around the world is at least suggestive.

Update:

China reserve requirements:

http://www.businessweek.com/news/2010-05-04/china-s-stocks-d...

Commodities, etc.:

http://www.google.com/hostednews/ap/article/ALeqM5jpti0ArQEl...

http://www.theaustralian.com.au/business/markets/global-mark...

http://www.businessweek.com/news/2010-05-06/corn-soybeans-wh...

Emerging markets:

http://www.livemint.com/2010/05/04214121/The-Chinese-contagi...

http://online.wsj.com/article/BT-CO-20100505-713490.html?mod...

El Erian - the best I can find is this. The lending comment was attributed to him when I saw it, but here it is a "trader speaking on condition of anonimity".

http://www.cnbc.com/id/36992469