jfischer

Hi, this is a side project started about a year ago that we’re finally launching. We combine automatically generated statistics with manual curation on the most popular data science and data engineering projects. We are independent consultants and the idea came from helping clients choose the best open source to build their applications. Rather than a lot of manual digging, you can find the relevant projects in one place, get a quick comparison, and have a starting point to probe deeper. We hope you find it useful. Your feedback is much appreciated!

Anything that I can do to help move this forward as a Sunnyvale resident and voter? Thanks!

Ha! Charlie left the software industry and moved to bigger and better things (http://www.antipope.org/charlie/blog-static/fiction/faq.html). He does have some good stories about his days at SCO on his website (e.g. http://www.antipope.org/charlie/blog-static/2009/06/how_i_go...).

I think comments are really a bad idea. If you want comments, either make them explicit properties in your data structures or use something else like YAML or XML. JSON is the most practical language out there for inter-machine and inter-program data representation.

JSON has two unique properties among data representation languages that make it useful for certain types of tasks: 1) it is very simple to write conformant parsers and serializers, and 2) the parsed representation contains all the original semantic information in the original document (excluding maybe whitespace). If you have a separate syntax for comments, then parsers must either drop the comments (making transformation tools less powerful) or parser must provide a more complex representation (e.g. like the DOM) and drop the simple map/list/scalar data model.

Please don't make JSON the next XML! Source: I spent about 5 years suffering with XML in the Enterprise Application Integration world.

In this particular situation, there was no else. I probably added an else clause with an assert return_value == False, as the function call was a virtual dispatch that could have many implementations. Of course, I wouldn't do that for every if/then/else statement in my code. In general, I'd prefer a stricter language that only permitted a boolean as a condition in the IF statement, avoiding this problem altogether.

If you are using truthy/falsey values, I think it can be a code smell that you are not doing enough to catch invalid values up front or should normalize the values closer to their creation point.

Alex Martelli gave a nice talk called "Permission or Forgiveness" about the exception handling style recommended by OP: http://pyvideo.org/video/1338/permission-or-forgiveness-0. He has some nice insights into this issue.

That being said, I think there are some situations where you want to check for problems up front (possibly in addition to exception handling). In particular, if you are parsing some data from outside the program, you may want to provide some context about what was wrong. KeyError is not very helpful to your users.

I've personally run into problems when I don't do exact comparisons with True and False. For example, I've forgotten to return a value in one path of a function/method, and then tried to use the result in an if statement in the style recommended by the OP (e.g. if fcall(): do something). After being bitten several times by this, I always do explicit comparisons.

Another approach that can work in big organizations: make sure that your boss knows that, if they ignore your advice, the responsibility for the decision is theirs, not yours. In a C.Y.A. organization this is often enough. Be sure to have written proof -- when it hits the fan, people tend to have selective memory.

This leaves a bad taste in my mouth. I've seen quite a few people use this approach in an attempt to get their boss's job, irrespective of the boss's performance in their position. In general, I think that fighting your boss only leads to a draw, almost never to a win.

It is interesting to see this from a (former) CEO's perspective. I've seen many big company executives who get stuck in stage 3. I knew that my last big company was destined for failure when the CEO bought everyone a management book about dealing with change. He seemed to be saying, "its not me, its you". If he had a bit more humility, he could have made the changes to get the company moving again. Of course, it can be hard to accept responsibility when egos are big and failures are politicized.

In a startup, you have this process not just for the company itself, but for each setback (e.g. in fundraising, customer acquisition, customer issues, etc.). Establishing a culture of accepting responsibility and learning from failures is absolutely critical. I think the biggest thing I've learned about my time in the startup space is how to deal effectively (or at least better) with failure.

I welcome the maker movement and I'm glad that what I did as a kid is now cool.

I got my start in engineering by reading the manuals of my father's HeathKit projects and by building a simple computer using a Z80 and a few other chips that I got through mail order and Radio Shack. Back then, having a personal computer was much less common and I could imagine building something roughly comparable to what was on the market (Altair, Apple II, C64,...). A generation before me tinkered with Ham Radio.

Today, HeathKit is out of business and Radio Shack sells overpriced consumer goods. Our electronics is now much more sophisticated and there exists a much wider gap between what one can do from on their own vs. commercial products. I think there was a dry period after the rise of PCs and before the maker movement, where there wasn't much opportunity for a hardware hobbyist. Limor and others like her are coming up with really interesting projects that are still relevant in today's world. I appreciate their re-imagination and invigoration of this hobby and hope to get my son interested when he's a bit older.

You might also be interested in following the progress of "Snap" [1], which is an extended version of Scratch implemented in JavaScript for web browsers. This from the group at UC Berkeley that did "BYOB" (Bring Your Own Blocks) [2], an extended version of Scratch.

I'm planning to use BYOB/Snap with my 6-year old. I think the custom blocks in BYOB will be useful for me to create relevant primitives so the programming will be less tedious for my son. Later, I can teach him about defining functions, recursion, etc. I'm also wondering if the JavaScript rewrite will allow him to hook into server-side APIs. That has some interesting possibilities...

[1]http://wiki.scratch.mit.edu/wiki/Build_Your_Own_Blocks_(Scra... [2] http://byob.berkeley.edu/

At my school (UCLA), there was not a formal requirement for publications or even an informal requirements. It looked like some professors emphasized publications more than others. I think, even if you aren't going into academia, they make a good unit of work and source of external feedback/validation.

My school did replace their Qualifying Examination with a research paper requirement (http://www.cs.ucla.edu/academics/graduate-program/graduate-s...), but it does not have to be published.

Unless things have changed recently, you certainly need more than 3 good publications to get a tenure-track academic position in CS. That means you have to either be a publishing machine while you are a student or do a post-doc.

This is just the tip of the iceberg - for some time now, many international flights inbound to the U.S. have had a secondary screening of your carry-on bags at the gate. They won't let you take any liquids into the gate (beyond the 3 oz bottles) and go through your bags. They had this last month when I traveled from HKG to SFO. Other international flights not going to the U.S. did not have this screening. I think I remember an at-gate screening at Frankfurt as well, but my memory is fuzzy.

At least there were no X-rays or pat-downs...

In this context, "In general" negates "I don't cheat...". Of course you may just mean that "nobody's perfect". However, there are people who habitually use lying as a tool to gain an advantage or place the blame on someone else. Without the opportunity to know someone better, I'd see little lies as a sign they are this type of person.

Nice article. I think it highlights the challenge of trying to balance multiple sources of priority, all of which are important to you. Of course, we can say that the family is the most important (which it should be), but, if you always prioritize family events over work, you might not have much of a career. Beyond maintaining an awareness of your internal decision process, I think it is important to communicate to everyone your commitments and stick to them. In the long run, being reliable is more important than being being a hero in a specific situation. In the Gandhi example, he did not prioritize his grandson to the exclusion of everything else, but he did make a commitment and followed through consistently.

Of course, this type of prioritization issue also applies to work within a startup. There's always more to do than you can possibly finish, and you need to pick between multiple urgent tasks...

This was an excellent video for those interested in programming models for distributed systems. Hewitt explains the key features of the Actor Model and makes the case that it more closely matches the physics of distributed systems than other models(e.g. Petry Nets/CCS/Pi Calculus). The Q&A with whiteboard format was very nice -- I got a lot of what I would get by reading an introductory paper on the topic, but in a more entertaining fashion.

Thanks for the encouragement! I think the discussion about values and work is very important and I appreciate that you brought it up on your blog. Hopefully, a bit got through on HN before it degenerated...

Well, to call anyone a socialist outside the domain of political science is being a bit facetious. However, a loose analogy could be made. Socialism is about the government owning the means of production. The analogy would be "the base of existing software" = "means of production" and "Free Software Foundation" = "government". Stallman appears to want the copyright for all software to be held by the FSF and licensed as either GPL or AGPL. The analogy breaks down in the sense that, unlike a government, the FSF has no coercive power (beyond the value of their code base). Given the FSF's (relative) lack of coercive power, I don't think anyone should take seriously people/organizations that raise alarmist concerns about it.

Thank you. I think the coffee analogy is a good one and I believe that it is important that we, as technologists, express our personal values through our work. Also, I don't get this "open source is socialism thing". Although some (e.g. Stallman) may wish to see a socialist coders' paradise, 1) he's certainly entitled to that objective, and 2) the open source movement is much larger than that. These days, picking a license is often just part of selecting a business model. Companies life or die based on the ecosystem they create around their software.

I think you can divide (useful) experience into two components: 1) a person's maturity as a programmer / team member, and 2) their level of practical knowledge with a given technology stack or your problem domain. I think that maturity is undervalued in this industry and is a must-have for senior developers. For practical knowledge, it can be extremely helpful to have someone who knows where the bodies are buried in the tech stack you are using. However, if you already have the knowledge on your team, then getting someone who can pick things up quickly might be a better focus.

Yes, Cringely can really be out in left field sometimes. A good example was his recent theory that Tim Cook is only a temporary CEO and that Steve Jobs has a secret hand-picked successor that he will name in the future [1]. I read Cringely partly for the entertainment value and partly because he really does think out of the box. When the strange happens, at least we're not surprised.

[1] http://www.cringely.com/2011/08/cupertino-two-step/

But their new strategy was described as a focus on enterprise software. Why invest further in a consumer device OS? Do they want to focus more on enterprise adoption? Maybe fill the space that Blackberry used to own? Or did everyone leave and they need to backfill to keep up appearances until they sell off the unit?

Palm has created many new job postings on monster.com since HP announced they where getting out of the tablet and phone business. Product managers, product engineers, software engineers, etc. Why?

I think you have a good point here. The list has too many items framed as "write an application..." The building blocks section should include some harder problems like optimization, planning, or code analysis.

Back in the day, I learned a lot about basic electronics and computer design from their old "Engineer's Notebooks" (e.g. http://www.radioshack.com/product/index.jsp?productId=108524...) and some books they sold about TI's line of microprocessors. If they wrote new books for Arduino, that would be a great start. I might even play around with hardware again (haven't touched it since I got my EE degree).

Hmmm. If we look at some past examples of cases in the consumer internet space where a new entrant overtook an entrenched player (e.g. Google vs. other search engines and Facebook vs. Myspace), I think you need to have something new and compelling about your application to drive adoption. I don't know of any cases where just cloning an existing app was successful for a startup. Has movement to a new player every been driven by openness concerns?

What can you do that is different from Twitter and cannot easily be copied by them?

It sounds like the leadership at your startup needs to make the hard decision to fire the co-founder outright or to reducing him to a very limited consulting role (no coding, just answering questions about his code).

I think that you would have a great story to tell to future interviewers if you can get the bad code under control and launch the product (even if the startup fails). I would have more confidence in someone who overcame adversity than someone who never faced any problems.

The "+2.5-3 sigma technical and +3-3.5 sigma big picture" stuff would be a big turn-off to me. There's a lot of really smart and talented people in this industry.

They say "Texas nonprofit corporation". Does that imply 501c3? They certainly could do a better job at organizing the information on their site.

I find this interesting not so much because of the budgeting features but because you don't have to give your personal info to the charities. I don't like giving to a charity and then finding myself on their mailing lists for the next 5 years. Even worse is when they sell your info on their mailing list -- it isn't always easy to find out how to get off that stuff. I've been thinking for a while that someone should build an app to mediate between charities and donors for this reason.

An added benefit is that they can give you a single statement for your taxes. I assume that, from the IRS's perspective, you are just giving to givv.org, a non-profit.