Bradford Cross: Big Data Is Less About Size, And More About Freedom

http://techcrunch.com/2010/03/16/big-data-freedom/

by jaf12duke • 16 years ago

43 12 16 years ago

fnid23• 16 years ago

I have used FlightCaster many times and everytime my flight has been delayed, FlightCaster has shown it as on time. I'm wondering if and when it actually works.

Has anyone had a successful delay status notification from FlightCaster?

bradfordcross2• 16 years ago

What's up fnid2. We've definitely had some known issues over the past several months at different levels in our system. We've worked out a lot of problems and are on top of others as we speak.

That said, Flightcaster works now for a lot of cases, but it is inherently probabilistic, so we do mispredict as well.

Keep using it, report issues to us when we are wrong, and we'll keep working on the algorithm and making our data better.

fnid23• 16 years ago

Hey Bradford, thanks for responding. Always good to get someone involved with the actual system to talk to!

As an individual, I'm not concerned about the probability that my flight is delayed -- I want to know if it is delayed for sure. If I go to flightcaster and it says it isn't delayed, I'm annoyed when I get to the airport and have to wait 2 more hours. If it says it is delayed and I wait, then I could miss my flight and I will be not only annoyed, but very angry -- probably at myself for trusting you.

Of course I could verify the flight status at the airline's website if you say it's a high probability, but in that case, why use FlightCaster at all? I should always just go to the more likely correct source first.

Probabilities might work for airlines or systems that are large enough to be affected by an aggregate flight status or for planning large complicated trips in advance, but if it fails to identify my flights so often, I will simply disregard it as a useful source of information -- as I suspect many individuals will.

So, what is your plan? I've read that you may license the technology to bigger firms or something like that, which sounds doable, but in my personal experience, it simply isn't a tool for individual travelers or I don't "get it." I also fly internationally, so all those flights aren't available.

What is a use case where this would be useful for an individual? Am I simply not the right user for FlightCaster? Who is your target market? From the marketing I read about it, it seems targeted to individuals -- that's usually what tc writes about anyway, so it's a bit confusing for me. I feel like it could be so useful for me, but I keep getting not so great results. Even last week I flew from canada through dallas. I couldn't get the first leg, but then the second leg was 2 hours late and on time according to fc.

What if you could add something where if I check my flight and I know the status for sure, I can "correct" the status at FlightCaster! Crowd sourced flight status! :)

mark_l_watson• 16 years ago

Great article - this is why I read HN.

I think that Bradford Cross is right on: as more data becomes available, the challenge is not building 'cookie cutter' applications, but rather using some combination of ML, text mining, statistics, etc. to create new ideas and systems.

bravura• 16 years ago

I am interested to read more about how people put together the pieces, such as hadoop, hive, pig, and rabbitmq.

nessence• 16 years ago

Great article. There's much more to be said about "Freedom". So many corporations have access to massive amounts of data and it will soon be easier to regulate access; whether by control of it's owner, or not.

tigger2010• 16 years ago

May be stating the obvious here, but why is a probabilistic prediction model primarily aimed at consumers? Wouldn't it make more sense to apply to users for whom probability is useful e.g., truck fleets to predict delays? oil fields to show likely drill spots? etc.

Bradford Cross: Big Data Is Less About Size, And More About Freedom

Related Stories