Derander

Python is the language of choice for many ML teams — it’s flexible, expressive, and has the right ergonomics for real-time workflows. But it’s also slow.

To make Python faster, I built a Symbolic Python Interpreter to make Chalk’s “Python resolvers” into optimized Velox-native expressions — enabling high-performance execution without sacrificing developer experience.

This post explains the problem I encountered, why it’s so challenging, and how we solved for it.

That's odd. I work in a 100% typescript shop and about half the devs use intellij and webstorm full time without much trouble.

Jump to definition/auto import/implement interface/etc all work.

Arbitrary example: If you're using SHA256 you're emitting 256bit hashes. Hopefully the distribution of SHA256 outputs is indistinguishable from random (or close enough) or else we've got big problems in other domains.

If you're drawing at random from a pool of size k you need approximately sqrt(k) draws until you reach a ~50% chance of a collision[0].

With 256 bits, there are 2^256 possibilities, so following the rule-of-thumb you'd need 2^128 draws until you had a 50% chance of a collision.

2^128 > # of atoms in the universe.

If you adjust your risk tolerance you'll have different numbers come out, but the chance of a collision in any realistic scenario is negligible.

[0] https://en.wikipedia.org/wiki/Birthday_problem

If you're using typescript and emacs, https://github.com/ananthakumaran/tide is a very good integration with the built-in language server for semantic auto-completion and "jump-to" support. I use it every day on large projects.

I think that vscode uses the typescript language server behind-the-scenes for plain javascript analysis.

I attempted to use clang-format on a mixed typescript/tsx codebase and found that it mangled jsx expressions. This wasn't surprising but I suspect that typescript users probably overlap heavily with react users so it's not a great solution as is.

It's not hard to automate with raw gulp or webpack, but it does add a significant amount of time to compilation. It roughly doubled our incremental compile times. My team decided it wasn't worth it, but it's possible that a cleverer person could decrease the overhead.

TypeScript has dealt with this in the past by adapting to the introduction of `<Foo>` from jsx, which clashed with typescript's casting operator. If this ends up being a problem I'm sure that the TypeScript team will just pick a different syntax.

In my experience, the overhead caused by TypeScript is far outweighed by benefits in tooling and confidence that my code works (mostly) after refactors.

Lodash has many features that underscore does not have. Also, I am told that Lodash has been implemented in such a way that functions are largely independent of each other so that you can build a custom lodash package that only includes the subset that you use.

I TA for this class. The class is optional for non-engineering majors but has enrollment >= 700 people per quarter. So many people at Stanford take the class that it's a bit of a cultural phenomenon.

There are three "intro" programming classes at Stanford. CS106A is "intro to java", CS106B is "intro to C++", CS106X is CS106A + CS106B + a bit more in one quarter. People who already know how to program are encouraged to take one of the latter two, although there is always a contingent of people with /some/ background in 106A.

I'm not the OP but:

The analytical gradient is actually a general tool used in multivariable calculus. It's vector-valued, in that if you take the gradient of a function of three variables: f(x, y, z), then you'll get a 3-vector back. Vectors are defined by two characteristics: a direction and a magnitude. The gradient's direction is the direction of greatest increase and the magnitude of the gradient is the instantaneous rate of change.

The gradient is being put to work here in order to optimize a function using a process called gradient ascent. Intuitively it makes sense that in order to optimize your function you'd want to "move" in the direction of greatest increase at every step. However, the size of the step that you take is tricky. As you point out, in this case, we can increase the objective function value more if we double the step size. However, you're not actually doubling /the gradient/.

If you look at the expression that you wrote you should see:

    x + 2 * dx * step, y * 2 * dy * step.

In this particular instance, the surface that you're trying to optimize is pretty simple so basically any step value will do the trick. It may take a different number of steps in order to compute the global maximum, but most reasonable step sizes will get there.

[1] http://en.wikipedia.org/wiki/Rosenbrock_function

I've heard nothing but good reviews of the online version of the class. I took the class at Stanford (and actually worked on the system mentioned in the article) and I found its content to be useful. I believe that the online version contains less theory but this is not necessarily a bad thing if all you want is an introduction.

I've also found reading papers to be illuminating: often the first article about a given classifier is fairly well written and accessible if you have a strong background in math.

This is also a useful thing to keep in mind: http://scikit-learn.org/stable/tutorial/machine_learning_map...

I'd be surprised if the average high school algebra course covered group theory :-).

The standard argument is that people shift jobs so much because it's the most reliable way to get a large pay raise. If companies stopped making it necessary to hop jobs in order to make closer to what you're worth, then it might help retention.

One of the interesting things about classic ciphers is that (as far as I know) they're obviously insecure by modern standards.

Most of the time what people want out of modern crypto is something called "semantic security" -- in other words, an attacker should not be able to learn even a single bit of the plaintext for any message, even if they are allowed to see the encryption of polynomially many chosen plaintexts (ignoring length, of course.)

What this means is that deterministic encryption schemes are all broken. Consider a game:

The attacker asks to see the encryption of messages a and b, receiving a' and b' as ciphertexts of a and b.

Now, the attacker is "challenged" with a new ciphertext of a or b, c. If the attacker can (with reasonable probability) distinguish which plaintext c is the encryption of, then the encryption scheme is not "semantically secure."

Notably, since our encryption scheme is deterministic, c = a' or b'. Therefore, with probability = 1.0 the attacker can distinguish which plaintext encrypted to c.

So, by modern security standards all deterministic (and therefore all classic, I think) ciphers are broken.

This is the sort of thing which is covered in the coursera class. Modern crypto is (usually provably) substantially more secure than what used to be used, and the kinds of techniques used to do modern crypto don't have much overlap with things like Caesar ciphers.

I'm a current Stanford junior. In my experience most people feel stressed if they haven't committed to an offer by early February. The major career fairs for internships take place mid-January, with a more full-time oriented fair occurring in October.

The majority of CS majors I know were locked in before mid-February.

There is probably useful analysis that they can perform in aggregate across all of their users. Perhaps people who read email are more likely to do <x> at a certain time and that signal is only noticeable with more data than you as an individual generate.

For those sorts of things I think it's completely reasonable to drop into emacs, vim or sublime and then just copy paste back into your primary IDE.

When it's a text editing operation that you're trying to do (as opposed to a defined transformation on valid source) then it makes sense to use a text editor.

I keep emacs and intellij open usually.

I care about cell phone performance because I'm excited for the day when I can realistically start booting a general purpose OS on my phone and call it good enough w/ a monitor and external keyboard.

To a certain extent that's what the JVM is -- you just have to target its bytecode :-).

One thing to note about this specific problem: this is an example of the traveling salesman problem with a metric. This makes efficient (good) approximation algorithms a possibility.

Many NP-hard approximation algorithms classes teach a 1.5 approx known as the Christofides algorithm. This algorithm is guaranteed to provide an approximate solution that is no worse than 1.5 times the optimal total distance, and often much better.

NaCl is actually designed to interoperate with JS in a fairly rich way and this interoperability is designed to be provided entirely by the browser. The idea is that you pick the runtime that you want -- probably the one provided by your browser -- and it pokes (hopefully secured) holes in the safe box that native client guarantees. This includes a sort of networking API that JS can call in to.

http://www.cs.utexas.edu/users/EWD/

Feel free to read one of the thirteen hundred things he published. Most relevant to you (probably) is that he championed the use of "structured programming" -- aka, if, for, while, and other narrowly focused constructs -- as opposed to GOTO: he's the author of "Go To Statement Considered Harmful ".

Also, remember also that many algorithms seem trivial once you've learned them.

For those who are unaware, Lessig is a professor of law and ethics at Harvard, and has been a professor of law at Stanford and the University of Chicago in the past.

First, in order to get a clearance you have to be sponsored. Companies have a limited number of sponsorship slots allocated to them by the government.

Then, the process behind getting cleared is fairly arduous. You may have a clean record but you have to prove it exhaustively even for the lowest level of clearance -- I believe the application form is on the order of 40 pages. You have to list all of your friends and sexual relations from the past 10 years, for instance, so that they can go an interview every single one of them.

Essentially, being born and raised in america and having a clean record while being sane is just a baseline requirement.

I'd be surprised if it were worth it. Even with fairly efficient GPU mining rigs it's getting close.

The need for that form of reasoning seems to me to be a symptom of the way that we go about writing code. We don't need to worry about if line 10 executes before line 11 within a given scope -- we just know this. If we come up with a similarly simple way of writing asynchronous operations that have dependencies then we can read it just as fluently.

In my experience, promises do fill this role if they're used in a suitable scenario.

>> but reading files in parallel would presumably be slower since you'd be jumping around on disk

There is a lot of engineering that goes into making parallel reads go fast. Some combination of the file system and disk controller will probably be smart enough to recognize the opportunity for sequential reads and execute them as such if possible.

This is not always true, and it does not undermine the rest of what you have written. I just think it's interesting to keep in mind that operating systems implement a lot of helpful machinery that user-level programmers forgot about.

The actual quote reads: "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil."

O(n) vs O(n^2) is not a small efficiency for non-trivial data sets. This is the point that was being made. Many people forget that the quote hinges on the word "small" and then use that as an excuse to disengage their brain when it comes to basic things.

If we're being really picky it is possible to deterministically find the median of n elements in \Theta(n) time so we're able to deterministically select as our pivot the median element.

This gives deterministic \Theta(n log n).

As mentioned elsewhere this algorithm has a fairly large constant factor and is not used in practice.

It seems significant that this is exactly one week later.