pbh

You're probably in a much better position to evaluate than I am! My guess is that more Octave-based classes would translate into more users and more code written for Octave down the line, but I'm not sure how to encourage more use of Octave in the classroom in the first place.

I've always been a bit sad about it, but everyone involved is probably a rational actor.

Computer science professors probably view a couple hundred dollars per MATLAB network license as a tiny expense on a $1m+ grant (whereas statistics grants are apparently often smaller), and they may be charged for it in departmental overhead anyway (removing the incentive to cut costs).

The type of people who could contribute either core code or toolbox type code to Octave often have an extremely rare quantitative skill set that is worth hundreds of dollars an hour, so there is a huge incentive to get paid to do similar work instead. There probably isn't much community recognition (to balance things out) for implementing a library in Octave. (Though, in the R world there are certain recognizable superstars like Hadley Wickham.)

Graduate students (who might work for cheap on these problems) are probably more focused on publications and networking.

As long as all of this is the case, Octave will always kind of just be a worse MATLAB that happens to be open source, so a new user choosing between them will probably just choose MATLAB by default.

This is a great description of R usage (import, clean, fit models), but I think a slightly erroneous explanation of R history.

John Chambers created "S" at Bell Labs. S was a programming language designed for interactive statistical analysis. Much like gcc and icc are implementations of C compilers, R and S-PLUS are implementations of S. S-PLUS was/is the primary proprietary implementation of the S language, whereas R is the primary free one (also, sometimes called GNU S). (SAS and SPSS are completely different languages/systems as far as I know.) I think that statisticians at some point made a conscious effort to publish their work in R, rather than S-PLUS (or any other statistical system like SAS) because it was more widely available. That in turn led R to be a viable competitor to S-PLUS (and other systems) because it had vast amounts of recent statistical libraries, often implemented by the people who developed the techniques. That said, SAS and SPSS seem to pretty much still have social science students locked up --- the market for R is probably statisticians who are also excellent functional programmers.

This history is in really marked contrast to MATLAB and its corresponding free version Octave, where computer scientists pretty much refuse to use Octave, despite MATLAB's massive price tag to pretty much everyone involved (even with 90% discounts).

(That said, if anyone lived through the change over from S-PLUS to R, I'd love to hear if this history is wrong!)

ezl does not seem to have gone all the way down the rabbit hole with the oDesk tracking link. (I work at oDesk, but not on this, so I was really curious.) After about 3 HTTP redirects, the www.dpbolvw.net link seems to turn into some sort of affiliate link. So my guess is that a (possibly errant, but presumably very data-driven) affiliate was the one sending him e-mail about oDesk, rather than oDesk itself.

That seems to raise a broader question, which is: to what extent should companies be blamed (and thus try to control) for the actions of the people in their affiliate programs? Would you be unhappy about a blog purporting to be by a cute ice cream eating girl filled with recommended books with Amazon affiliate links? What are the standards in this area, exactly? (On reload, see dabent's comment below as well.)

Actually, I thought his comment was quite delicate.

I used to think that DB comparisons were bad because of vested interests, like Oracle refusing to allow anyone to benchmark them. But I now think that Antirez's view is much closer to the truth: poorly informed people can't set up benchmarks correctly because they don't understand the underlying DB model and possible optimizations, and well informed people can't set up benchmarks because they can't think of a fair benchmark that doesn't explicitly handicap one model or set of optimizations versus the other.

(Of course, in this case, the comparison is pretty bad. Other commenters have noted that an inverted index is more sensible here and that Postgres' text search support should have been used. But the big issue to me is that the graphs are incredibly misleading, with totally different units on the y axis, so that a 0.004 second win for Mongo looks like as big a win as a 3 second win for PG.)

I think the most succinct response to this question I've found comes from the Starcraft II casting archon, "Tastosis" during one of those filler periods where nothing is happening in the game:

Tasteless: I'm a philosophy major, man.

Artosis: Hmmmm? That's good.

Tasteless: You major in that, people are like: "What are you going to do with a philosophy major?" ::outraged mumble:: I'm like: "I dunno, properly navigate the world with my mind? Geez!"

Is there a way to fix the recursion depth problem with this solution?

To me, the memoized version is more clear and should be just as efficient as the iterative version. However, this version of fib_mem dies before fib_mem(1000)!

Apparently, you can increase the recursion depth limit using the sys module. Is that the right thing to do? And, if you do it, are there guidelines for doing so without causing memory problems?

This tutorial was a few months ago, though it was quite good. The slides are here if anyone is curious:

http://www.slideshare.net/ipeirotis/managing-crowdsourced-hu...

I know a lot of Daphne's former and current students and she seems to be a great advisor who genuinely cares about producing both excellent research and top quality research talent.

Further, in the department, I think she is one of the people who cares most about teaching. She runs the undergraduate summer research program. She re-does her class on PGMs substantially almost every time she teaches it to try to make it better. (Though such a high rate of change may or may not be a good idea.) Daphne is almost certainly one of the key people behind the *-class effort at Stanford CS.

For those with a negative impression of Daphne, my guess is just that they are misinterpreting her directness. If she thinks you're wrong, or you're doing the wrong research, you'll know about it.

(Also, Ullman is one of the nicest people in the department in person, which is crazy given that he wrote the standard texts in compilers, databases, and arguably automata. He's emeritus these days though.)

Is this a CVPR reference, or just an amazingly accurate comment?

Yes, I think computer vision is a common term for the area (at least, among academics and researchers), and CV is a common abbreviation.

Examples include Intel's OpenCV and the CVPR conference.

(I actually thought that it was weird that they called themselves "machine vision" rather than computer vision in the title.)

I agree with your point, but with an important caveat.

You are right that application developers should not be ignorant of databases, and DBAs should not be ignorant of what application developers are doing. All too often, this is the case.

However, the point when people came to think of application and database development as two separate things was the origin of databases themselves, and arguably one of the most important historical points in computer science itself. The notion that data has a life outside of any given application, and that it deserves itself to be managed, was and is incredibly powerful.

Unknown.

However, the three classes are intended to match up with Database Systems: The Complete Book (which, in the interests of full disclosure, is written partly by my former advisor). If you want to learn the material from the second class, I would just read the book. I've actually read the book cover to cover, and I think it's one of the best textbooks for that purpose. There are tons of examples everywhere, it's pretty talkative, and there is very little pretension (insert Russell/Norvig complaint here). On the other hand, some portion of the Internet seems to take the opposite view (e.g., some people just view the talkativeness as repetition).

To answer a few of the database questions that keep popping up in this thread:

1. Is it worth understanding relational algebra? Yes. Definitely. It's not that hard, there's more or less a one-to-one mapping of relational algebra to SQL. Relational algebra is both a useful mathematical tool and indispensable when trying to understand query optimization. Relational algebra is also necessary to be able to read pretty much any of the database literature, if that's one of your goals. (Just be thankful you don't have to learn Relational Calculus or Datalog.)

2. Is it worth learning XML features like XML DTDs, XPath, XML Schema, and XQuery? Yes, definitely. I don't like XML, but there are tons of places where you can use XPath. (The most interesting thing about XQuery is probably how similar it is to SQL, though.)

3. Is it worth learning about SQL triggers? Probably. Ultimately, the web development world is split between people for whom separation of concerns means doing more in the database, and those for whom separation of concerns means doing less in the database. If you think doing validation and transactions in the database is sensible, triggers are also a sensible thing to learn about. However, people like DHH disagree and want validation in your app server with a dumber backend.

4. Is this vocational training? Well, this is the first in a three class sequence at Stanford. This class is mostly about schema and query languages, so in some sense, it's the most useful to practitioners (or at least, non-DBA practitioners). The second class is about systems and implementation (on-disk layout, indexing structures, query optimization). In the third class, students either learn about distributed databases or actually build their own from scratch. However, schema and query languages have enough theory behind them, and enough generality, that I wouldn't consider it any more vocational training than a class on compilers.

Sure, of course, premature optimization is the root of all evil and all that. I presume you're reacting to a bad experience in the past you had with some young developer not understanding database indexes or memoizing everything unnecessarily or something. I certainly don't want to create another one of those experiences!

That said, he asked about how to get web application speed. One major answer (as discussed elsewhere in this thread) tends to be "add a cache somewhere." Of course, you have to architect things right so the cache works (possibly with a layer of indirection like app servers, or figuring out where a cache would be helpful, etc.) and the code that the cache is obviating needs to be slow in the first place.

That said, the general rule (add a cache for speed and scalability) has been the case from Slashdot to MovableType/Wordpress to Facebook to App Engine.

I'm not an expert, but this is what I've cobbled together as a fellow Ruby/Rails startup person.

Fault Tolerance: Just use Heroku. We've seen maybe an hour of downtime in a few months?

Speed (and fault tolerance): Cache everything you can, assets on S3/CF.

Testing: Rails Test Prescriptions by Noel Rappin [http://pragprog.com/book/nrtest/rails-test-prescriptions]. Then you can choose what you like, but I like Test::Unit, Mocha, FactoryGirl.

Code Abstraction: Rails is already pretty sensibly organized, and if MVC + tests + static assets is not a good fit for your webapp, you really should not be using it in the first place. One minor point: Noel and others will tell you to use skinny controllers.

Curious what other people consider best practice for Ruby/Rails startups.

Nobody seems to have picked up on the "external" spam filter part of the response.

My guess is that Y! uses some somewhat dodgy external providers of spam/content filtering. Historically, external providers that block inappropriate sites have considered such lists to be their proprietary information, but when those lists have been revealed, they tend to have a pretty heavy political agenda.

Peacefire has some reports that detail this:

http://www.peacefire.org/

(So, I think the reason for the Y! block was likely an unaccountable third-party, but I think the Twitter block is pretty odd.)

This is really helpful. I will definitely check it out.

Also, I'm really amused that apparently the creator of DataMapper is (a) selling me on using Sequel and (b) the testimonial on their front page. (Not that there's anything wrong with that.)

SQLAlchemy (and its excellent documentation) is the reason I switched to using an ORM.

(Now that I am in Ruby land though, I am a little sad that arel/ActiveRecord/DataMapper do not seem as on their game.)

What's most interesting to me about this article is that the management of GAE seems to actually be getting worse over time.

GAE has always had two main disadvantages. First, there is vendor lock-in because you code specifically to the data store, worker API, and so on (though arguably there are alternative platforms that implement the GAE API). Second, you cannot run custom code (custom C in some virtual machine) or have a custom architecture (if, say, Redis might be useful to have around). These disadvantages probably aren't changing and are probably necessary for auto-scaling, security of Google's infrastructure, and so on.

However, there are lots of little things that GAE has been getting wrong for a while that are totally unnecessary. Lack of hosted SQL support. Lack of SSL for custom domains. Just little things that are probably annoying to implement and boring, but totally necessary for real websites or websites just gaining traction. (I know these are in varying stages of early support at the moment.)

But now, the GAE team almost seems to want to actively disappoint users. With hosted SQL being a request for years, Guido appears to have spent a bunch of time re-architecting the API for the datastore instead. With this pricing increase, they're pushing the many developers who came to their platform based on price (due to the very interesting scaling properties of the Google front-end) off the platform.

Overall, I'm very confused.

I went through this thought process. I really liked Python and auto-scaling, so GAE seemed really appealing. But the combination of lack of SQL and inflexibility of architecture led me to want to create the sort of combination architectures you describe with EC2 instances or whatever.

Ultimately, after a lengthy debate with a friend, I concluded that Heroku was a sufficiently better option that it was worth learning Ruby (which I've been writing more or less as an uglier Python anyway).

Somewhat related, I found Arrington's talk at Startup School '08 to be pretty good for getting an idea of how he views the startup tech press.

http://www.youtube.com/watch?v=HbUnatPfSgg

Sorry for the delay! We actually got a bunch of orders, so we're in the process of running brainstorms for all of them.

Since the blog post said that most of the iterations were a few days, we originally opted not to have a follow-up email, but I think that was probably a mistake. I'll modify our systems right away to do that.

This is definitely a direction we're considering going in, but we're not quite sure exactly what people want.

Are there particular features we should provide, constraints on domains, or other things that we could brainstorm that you have in mind?

(You can also e-mail me with your thoughts, my e-mail is now in my profile.)

I really like the idea of combining AdWords with domain search, though I'm not quite sure what the right implementation is.

For example, suppose that there are two domains, "[uniquequery].com" and "[commonquery].com". Do you think people would prefer the "[uniquequery].com" because their name will show up at the top when they tell people about it (and because its more unique)? Or do you think people would prefer "[commonquery].com" because it will be easier to rank for the competitive query and there might be more organic traffic?

We had a similar question, trying to decide between "brainstormdomains.com" (which seems like a much more common query, though I haven't checked) and "nolongernameless.com"/"domainrefinery.com".

Will fix that!

At the moment, there's only one option so we'll probably just switch over to having a button, but I haven't had a chance to convert it yet. (PayPal has a habit of blocking your forms if they detect any mismatches between your encrypted and form data, so I want to be careful about it.)

Should be fixed now on reload. Thanks!

The brainstorming and initial reaction points are two points I actually forgot to make in the blog post. Thanks!

Regarding brainstorming, I've found that I tend to get locked into one line of thinking when brainstorming domains, and the workers don't do that (since they're all different people). As a result, even if the specific proposal by a worker isn't great, they're often on to an interesting line of domains that I can continue myself.

Regarding initial reaction, I found that the process of honing a description to get a good name was actually really similar to honing a pitch describing a company. When the workers didn't get the concept, they could not produce good names. Further, the best descriptions start to sound like what-problem-does-this-solve customer-based pitches for websites.

1) There seem to be two things that impact the number of rounds necessary, your description and how many names are taken in that area. With the same description, usually one or two rounds is sufficient to see how well you're going to do (sometimes you'll get a bad batch of workers, but one or two rounds is usually enough to smooth the variance). However, if you're changing little things it might go longer. For example, if I was to do more rounds, I'd probably add something to the DebtEye description to ask for more "safe sounding" words, and less sort of sleazy words (e.g., "ez" or "instant").

2) $5 is below cost, our twin goals for the moment are to gather more data about what sorts of names we can do well on and to get our name out there. We'll probably do a survey or something later of real buyers to see what the actual price should be. There's also some raw-ness to the results right now (e.g., you'll get 2 or 3 good results with 47 bad results) so we're keeping the price relatively low until we have better automated filtering.

3) Unfortunately, when we were just starting out we didn't have Domain Refinery (and a naive implementation doesn't work terribly well for reasons I won't go into). As a result, we did the days of brainstorming followed by purchasing an overpriced name route. However, we've since used Domain Refinery for our naming, and in fact, Domain Refinery is named using itself. The top three names for what is now the Domain Refinery were domainrefinery.com, nolongernameless.com, and brainstormdomains.com. Ultimately, we liked domainrefinery.com best because it signaled the refinement aspect that we wanted to contrast against competitors.

(I'm one of the co-founders.)

Any chance of PL/python or similar?

I guess I'm probably one of like three users of PL/python, especially since it's untrusted. Worth a check, though!