Where are the “big problem” jobs?

http://lemire.me/blog/archives/2013/03/04/where-are-the-big-problem-jobs/

by joeyespo • 13 years ago

48 55 13 years ago

soup10• 13 years ago

There's not that many because most people don't care about doing big things. Most of the wealthy are more concerned with protecting their wealth, status and success than risking looking like a fool trying to change the world.

obviouslygreen2• 13 years ago

I agree with the wealthy point; it even applies to the not-so-wealthy who are just trying to make their way and support their family and/or employees.

However, I do think a lot of people do care. It just doesn't end up mattering, because the people who have the means have been through so much red tape and related crap -- and usually have so many shareholders to report to -- that it's no longer a matter of their own priorities, it's a matter of perpetuating a business on which even more people depend.

This isn't meant as an excuse, but a reason. We're not working on "big things" because as a species we have totally failed to prioritize our own global well-being and that of our world.

cpeterso3• 13 years ago

But the super rich can't take their wealth with them, but if they invested in solving big problems, their name might be remembered in the history books.

ambiate• 13 years ago

'Research is not profitable. Sure, we could put 250MM a year into big data and helping people, but our revenue from small data keeps the business solvent. We grow, we scale, we profit. Problems are to be solved by entrepreneurs with few resources and great intentions. There are no good intentions in business, only profit.' -- Ex-Boss.

(The site was down for me, but I assume this is the direction of the article, and this quote comes to mind).

vellum• 13 years ago

Cached copy: http://webcache.googleusercontent.com/search?q=cache:http://...

epenn• 13 years ago

There are corporate exceptions. For example, Google seems to be working on a few big problems, such as the self-driving car and Google Glass.

I'm not entirely sure I would classify Google Glass as a "big problem." Self-driving cars can address driver safety, fuel efficiency and route efficiency, among other things and therefore certainly rank up there. Google Glass, although I like the concept and very well might even purchase a unit when they become widely available, seems to address the same areas already covered by smart phones, just with a new interface. I think there are huge strides that can be made in the area of pervasive computing, but I'm not sure Google Glass is what will propel that.

(I would love to be proved wrong though.)

hcarvalhoalves2• 13 years ago

> I'm not entirely sure I would classify Google Glass as a "big problem."

It's novel, it opens up new applications not possible with smartphones and it can get widespread adoption. It might not be hard because most technology necessary is already here, but I think you can say it's big by the amount of disruption it can introduce.

obviouslygreen2• 13 years ago

I don't think the potential effect of a venture can really be said to reflect the "problem" it addresses. The world is not clamoring for a way to view data without a monitor or search for the best deal on Frosted Flakes. Unless I'm missing the sense of "big problem" here --and I can't tell, since the site is down -- this seems like mistaking "difficult technical problems" for "large-scale social problems."

namank• 13 years ago

Well.. depending on how pervasive it gets, I would put Google Glasses on the same scale as the iPhone when the consideration is the bigness of "big problem".

So, how big was the iPhone?

zanny4• 13 years ago

I think the cell phone was bigger. The iphone was just an evolution of the concept. Processors became powerful enough and efficient enough to run a full OS in a phone, Apple jumped on it, the rest is history.

epoxyhockey2• 13 years ago

I think SpaceX and Tesla Motors (Elon Musk) are better big-problem solving examples than Google Glass.

bostonpete• 13 years ago

I get why you'd say that about SpaceX, but why is Tesla in that category?

dj-wonk3• 13 years ago

Well, it doesn't matter all that much that each of us agrees that building Technology X will help with Big Problem B. Why?

1. It is hard to predict how innovation unfolds, builds on itself, and cross-pollinates. For example, who predicted that 140-character-or-less social media messaging would benefit democratic uprisings in Egypt?

2. We aren't going to agree on the big problems.

My criteria is simple: somewhere/somehow in the economic system and public spheres, we need diverse investment in technological advancement across many time scales.

burntsushi• 13 years ago

There are two kinds of solutions. Solutions that solve known problems, and solutions that solve unforeseen problems.

Plenty of technological innovation has come from the latter. Google Glass is precisely the kind of thing that has the potential to solve unforeseen problems.

ChuckMcM• 13 years ago

Perhaps there are still too many "small problem" jobs that are profitable to solve so the capital is not out looking for the 'big' ones.

Of course the imprecision of 'big' and 'small' in defining problems is also an issue. You can have 'big' problems which exist not because of the science, but because of the structure. Today we were reminded that its a 'big' problem to build a house in South Florida that won't be eaten by a sink hole. Earlier we were reminded about building on eroding beaches. But those are actually "easy" in the sense that you could push piles into bedrock, 75' below the house. But we don't.

Bill Gates has made it a priority to solve some 'big' problems, sanitation, malaria, to name a couple. Other foundations have similar goals. So they are out there but as the other has seen working on them when they aren't self funded is hard to achieve.

patmcguire• 13 years ago

Every time someone complains about tech businesses not taking on the big problems, I'm reminded of the early days of computer graphics when there were all sorts of complicated algorithms to figure out which parts of a complex arrangement of polygons would be visible on the screen. http://en.wikipedia.org/wiki/Visibility_problem It turned out that the best way is generally the comparatively crude z-buffer. (I'm not super educated on this, this was just a story my prof told me).

The point is that the big technical problems that need the most research muscle aren't necessarily that important, and you can't really know what the future needs ahead of time.

nnq2• 13 years ago

I wonder if the "basic income guarantee" (http://en.wikipedia.org/wiki/Basic_income_guarantee) applied at something like country-level wouldn't dramatically increase the number of people working on "big problems" and of entrepreneurs trying "big problem businesses" - with no personal risk of failure, people would actually start chasing the "big problems" simply because they are FUN, and with no adrenaline from our little economic rat-race, this could actually become the only source of "fun" (besides hard-drugs for now and maybe some virtual-reality based entertainment) for creative and problem solving types. I don't advocate this - it would probably fail miserably and be "globally unfair" if applied at the level of a wealthy nation, but I always ponder on the consequences of such a thing. We're so "knife to neck" in this race for profit that only very few lucky and smart people have the opportunity to ponder the big problems without any other worries - maybe quantity (as in number of brains involved and pumping up ideas good and bad) as opposed to quality (very few very smart and well funded "geniuses") would yield better results on these so called "big" problems (I think of things like "general AI" or "cure for cancer" kind of problems). And the funny thing is that such an "arrangement" has never been tried in the history of mankind, so it might as well "work" (in the sense of "accelerate" the solving if "big problems").

...and traditional market forces have never been known to catalyze work on "big problems". The only good catalyst I can think of is war (think the nuclear weapons problem or the race to the moon fueled by the cold-war).

cpeterso3• 13 years ago

Access to socialized healthcare is a similar factor. If a potential entrepreneur in the US relies on health insurance from their current employer, they may be reluctant to start their own business without a "social safety net".

nnq2• 13 years ago

...yeah, it's almost easy to forget that there still are wealthy developed countries without socialized healthcare :)

colmvp• 13 years ago

"Access to socialized healthcare is a similar factor."

Having lived in both Canada and the U.S. for many years, I'm curious why I've met so many more entrepreneurial and succcessful people in the big cities in America versus my extensive time in Canada. I figured having basic healthcare for all would make it significantly easier to jump ship from a job and start ones own business.

JanezStupar• 13 years ago

Few people care about health care until they really need it. Thus I would argue that it really is not such a significant factor.

Other things are much more important. In a sense all Americans are businessmen. There are few countries that value business, entrepreneurship, business people and property as highly as Americans.

zanny4• 13 years ago

I love the idea of a BIG, because we are post-industrial and have a surplus of almost everything we need in the first world, but just don't distribute it due to the dramatic income divide.

It would also let me, say, try writing a new development IDE, or a new programming language, or a new OS kernel, or most other low-level software products you can't "sell" to people, full time. Right now you really just can't commit to those kinds of projects because they don't feed you.

The only problem is it can't happen in our current culture. At all. At least in the US, the entire political system, tax code, etc are all so horribly broken, abused, exploited, and convoluted no change can happen at such a fundamental level.

I just wish there was a new frontier to try these ideas out on. I wonder how badly humanity as a species suffer from the inability to pick up, go to unknown lands, and try something new. You just can't try new social or political philosophy anywhere anymore.

rdl• 13 years ago

I'd personally support BIG/negative income tax if it were universal and thus eliminated the administrative overhead and political power of other special interests created by special programs for the poor. Even as a libertarian who is generally against government intervention (mainly because it distorts the economy, but also because it's inefficient), this seems like the best realistic tradeoff we could make.

It's unclear to me if it should be a cash payment or basic life support (barracks-style housing, food) available for the asking; advantages and disadvantages to each. There are probably people who are incapable of managing their own affairs (the mentally ill, orphans, aged), but for other people, cash is more appropriate -- letting an entrepreneur decide that buying a new laptop is worth eating sub-standard food for a few months.

kyllo• 13 years ago

Because the biggest problems are the most expensive problems, and the most expensive problems require the deepest pockets, which belong to the government.

fennecfoxen2• 13 years ago

And when the government opens up its deep pockets, even on reasonable problems (like "low-pollution energy") you get results like Solyndra... or worse, because it's more about chasing government money than coming up with any results.

artmageddon2• 13 years ago

You could also get results like Germany who has been producing record amounts of solar energy thanks to their push in that direction. There are obstacles we'd have to overcome here to make that a reality here in the US, but it's not impossible.

bearmf2• 13 years ago

But you also get Sputnik, Gagarin, Apollo, Voyager. You get microprocessors and Internet. You get nuclear energy (though you will be right to mention industrial fusion reactors which are always 20 years or so away :))

elteto• 13 years ago

You are missing the forest for the trees. The entire technology stack behind you posting something here in HN started at some point as a government-sponsored project.

mturmon• 13 years ago

You are reciting talking points. How about GPS, weather prediction, the highway system?

Schwolop• 13 years ago

Well I for one would love to work on a big problem. The reason I'm not is because the smaller problems pay, and the bigger ones (generally) don't. I tried running my own business aimed at a big problem (admittedly not very well or for very long) and constantly hit upon the difficulties of getting anyone to pay for it.

nbroyal• 13 years ago

The openings also aren't very visible (or I'm not looking in the right places, which is entirely possible). I'm surveying the job market after shutting down my own business, and I've seen very few openings from companies that are working on what I would call "big problems." It's a bit disappointing.

dj-wonk3• 13 years ago

What one considers a "big problem" depends on your values (i.e. where you want society to go and how you want it to get there). From my perspective, many of the "big" problems are just that because they are the result of complex dynamics and structural challenges that cannot easily be solved by one actor (organization / business / government).

Many of these problems are so-called collective action problems. If you are able to get enough funding and a strong team, that may not be enough: you need more than technical chops to move the system enough to make a difference!

P.S. I'm glad this topic is getting some air-time!

dj-wonk3• 13 years ago

Debating which particular technologies are "big" and "small" is rather silly IMHO. No offense to armchair commenters on Hacker News, but this is not a new topic where we are breaking fresh ground. For example, there are numerous articles on Innovation Policy, such as http://www.itif.org/publications/global-innovation-policy-in...

cpeterso3• 13 years ago

In Robert Heinlein's novel Time for the Stars, a non-profit organization called the Long Range Foundation ("LRF") funds expensive, long-term projects for the benefit of mankind that won't show results for decades or centuries. Some of these projects turn out to be hugely profitable, but because profit isn’t in the organization’s charter, all this money has to be funneled into other longer-term and even more outrageous projects.

betterunix• 13 years ago

Solving big problems means taking big risks. Most businesses would rather do less risky things that bring in the revenue sooner than risk spending large sums of money on projects that could only bring in money years down the road. Exceptions exist (Microsoft Research comes to mind) but for the most part, businesses focus on their short-term profits, and leave research to universities and governments.

This is only a problem if the government is failing to fund research adequately. The point of government grants is to pay for riskier research. There are limits to what governments will fund, and unfortunately there are "research topics du jour," but I do not really expect companies to go wild with shareholder money. The free market is not going to cure cancer, and that is fine because the free market is not the only way to pay for research.

carterschonwald2• 13 years ago

Those big problem attacking businesses are out there, they're just often too busy to write blog posts about it! :)

That said, I'm one of those grad school dropouts who's stumbled into the whole trying to bootstrap a hard problems only business.

Its a lot of work! I'm lucky that my family is based in NYC so that I can reduce my living costs plus do hard problems only consulting part time while spending the rest of my time working on building my tech products @Wellposed.

There are uncountably many high value problems you can try to build an interesting business around. Just look at any Specialized / niche high value industry and ask "can we build better tools and how valuable would those be?". You should choose a domain where you'd enjoy talking with/getting to know potential customers personally. if there are other folks attacking the same question domain+combo, ideally they should be smart folks you respect who are solving the problem with a different combination of tradeoffs so that the products are complementary rather than substitutes. Oh, and you should be OK with building a satisfactory product taking at least a year of hard work, constant learning , and feeling stupid because you keep on seeing better ways to do every thing.

Or at least that's sort of the meta pattern behind how I'm operating at Wellposed. Its hard work, I get to feel stupid in the always learning more sort of way, and its the most fun I've ever had.

I'll find out towards the middle / end of this year if my ideas and process bear out on my first hard problem domain. I'm terrified and excited. Either I'll quickly be trying to figure out a sane hiring process, or I'll be job hunting with an interesting engineering and consulting portfolio accumulated over the preceding year, and I don't know which outcome is more likely!

That said, entrepreneurship is hard work and gets in the way of spending focused time on engineering. Good thing I'm a tad extroverted.

zanny4• 13 years ago

> Good thing I'm a tad extroverted.

Lucky bastard. I'm always jealous, being a deep introvert, of how others can engage with others so openly.

carterschonwald2• 13 years ago

Its really hard though! The thing I've got to remind myself that when theres a negative backlash, then I've just passively filtered people.

It takes work, I have my insanely self conscious days/months/years. Baby steps!

rhizome2• 13 years ago

Aren't they at the companies who ask Big-O and LSP questions in interviews?

graycat• 13 years ago

This article is not nearly the first time I thought about such things. His reason for getting a technical Ph.D. was essentially the same as mine. And I concluded the same as he did, that to use such education in business, basically have to start a business and own it.

He omitted my basic rule: For getting venture funding for such a company, the rule is that some successful research promising as the core 'secret sauce' of a new business together with a dime won't cover a 10 cent cup of coffee, that some solid research for a great technological advantage with some traction is significantly less good than just the traction.

For more, such discussions miss some big points likely especially relevant now.

First, let's consider the 'big problem': Since we've already decided that we have to be in business, the 'big' we might as well take as promising to make a lot of money, e.g., build a company worth $100 million times some factors of 10.

Second, for building this company, we come up with two lists: The list in column A is for technology we have or might invent. Column B is for problems people have and are or at least will be quite willing to pay to get solutions (if there is a big question about 'product/market fit', then leave the problem on the cutting room floor -- we want no questions about eager customers). Yes, it can be that we deliver the solution for free on a Web site and then get paid by advertisers eager for the eyeballs of the users -- that's a small detail we all understand. We want no technology sold before its time, no newly invented technology looking for suitable real problems, no products without eager customers.

Then we pick a 'crucial core' technology from column A and a corresponding problem we will solve from column B.

Third, we already know how we will deliver the solution -- over the Internet -- and we already know how the solution will be generated for the users/customers -- via software on computers. Why do we know these things? Because of the astounding, historic improvements in price/performance of bandwidth and computing, and platform software, the fact that these improvements have not yet been fully exploited, and the fact that in the current economy information generated from software can be among the most valuable, and most profitable, results to bring to the market. Biotech? Have to wrestle with the FDA, at $100 million or so a round. Energy? It's an old field, has been slow to exploit nuclear energy which was and remains one of the greatest opportunities in the ascent of man, and, thus, really is stuck in the mud. Materials science? A sheet of carbon atoms is not yet an actual product that can be sold to solve a real problem. Flying cars? I won't comment!

Fourth, here's a secret (make a billion): When we look at column A of the technology, note carefully the phrase "or might invent". No joke, guys. It's too easy and common to laugh at original research. Most of the people laughing have done no original research. And it's too easy to notice that the guys in universities that do original research usually get maybe papers, professional reputation, and tenure but no business revenue. Why? Because heavily that research is funded for US national security or US health, and in either case the problem sponsors in the funding agencies don't want just 'research' to make good products to solve real problems but another E = mc^2 or actual good progress on a cure for a major disease, especially cancer. Considering that they are spending my tax money, I'm essentially pleased. NSF/NIH money is too short now to fund the best, good, far out, fundamental research; there's no hope for it funding good original research to make money. But such research can be done. Trust me!

Then with the research, old or original, from column A and problem from column B where the research provides a good solution to the problem, convert the research to the corresponding software, load it onto a server, put a Web site or an app in front of the server software, please users/customers, and make money.

In this, research just such as I described is crucial for much of the progress people are not seeing but have been expecting.

Silicon Valley VCs, listen up: You guys will have to start taking research, original and/or old, seriously or risk funding just more social, mobile, locational, sharing apps or trying to buy tickets on airplanes that have already left the ground and are not taking on any more passengers.

Entrepreneurs, listen up: Pick a pair from columns A and B, write the software, get the server up, and go live. The software? Mostly the time is just 'overhead' that you have already covered as part of your day job. The server might cost you less than $1000. The bandwidth might cost you less than $100 a month. Or just put your software in the cloud (at maybe some risk of your crucial, core research).

That's the opportunity.

ihnorton• 13 years ago

tl;dr: Every time this comes up I wish the writer would start with a personal "taxonomy of big problems" to better frame the discussion.

At very least, I wish these writers would preface with a few examples of what kind of "big problem" they think corporate R&D should be working on, but isn't. OP's example falls flat to me:

> Private businesses seem remarkably uninterested in tackling serious problems such as energy despite soaring prices and evident problems.

They may not be interested in solving energy problems in the way OP wants them to, but there is certainly massive industry R&D in the energy space (Siemens and Schlumberger come to mind, many others). Just look at the ever-increasing yields from fracking and (ever deeper) deep-sea drilling. That requires huge R&D investments (down-bore sonar and NMR, anyone?)

If you consider the sorts of problems that are presently somewhat tractable modulo engineering {to pick a few: energy, cancer, food, genetics} I would argue that there is probably a lot more industry R&D work than the author realizes. It may not happen in the software industry, but realistically, for many "big problems", software is auxiliary at-best.

If you are considering the sorts of problems that are presently only tractable modulo magic {fusion, life extension, interstellar travel} then of course corporate R&D is not working on those.

calinet6• 13 years ago

A simple answer: too much risk!

There is a threshold to capitalism. The problem is that no one has yet figured out how to hedge the bet, attempt to solve extremely risky borderline impossible problems, and guarantee a profit. It's much much safer to go after an obvious, well-described, minor problem that over 50% of the population experiences and would pay a minor sum to improve. It's a safe bet. And there are a lot of those!

The other problem is surely capital and education. They've diverged and are being pulled in opposite directions. Higher education saddles you with debt and makes you extremely risk averse. You can't ever get access to capital because you're always playing it safe. So the system has been inadvertently designed to lock out these "big problem" jobs, because, again, they are too risky for people to undertake.

And then the rich are also risk-averse these days. The focus is on ROI and increasing wealth—look at Warren Buffet, one of the world's richest men, entirely rewarded for being risk-averse and investing in solid, tried-and-true companies. And you're surprised that the big problems aren't being solved? People like Buffet aren't investing in them, at least not at a large scale. And they're rewarded for that behavior. The system is set up to reward this sort of behavior. The idea is that once you get rich, you invest "prudently" and grow your wealth. You don't bet it all on the big problem that's probably impossible.

Except sometimes you do. Some people are solving the big problems. Elon Musk is up there doing it (in space and on the ground). Bill Gates and Buffet too are putting up a lot to fight seemingly impossible big health problems. This is all excellent, and not to be ignored.

I think the incentives are not yet aligned correctly to get the result you're looking for. What you want is to reward risk taking, and capitalism just doesn't do that in the right way. There are too many hurdles, too many punishments for failure. And the right people to solve the problems aren't the same ones who have the capital to get the solving done, for so many reasons.

There's your big problem to solve—fix that system. Align the incentives. It's the meta-problem, and if you solved that, the world would probably be a better place. (Hint: there are already some good solutions, but the America won't vote for them)

AznHisoka2• 13 years ago

Yup, don't blame the people, blame the system. That's exactly it. It's like asking why aren't all the hot girls pursuing the nice guys.

AznHisoka2• 13 years ago

Solving big problems doesn't necessarily = profit, especially in the short-term, which is what drives most companies and businesses. Unless you got billions of disposable income (ala Google), you just don't have the luxury to delve into "big problems".

We live in a modern world with many advanced technologies. But in a way, we're not unlike the hunter gatherers who had to spend their hours worrying about survival. We spend the vast majority of our 24 hours commuting, and working for The Man instead, worrying about paying our bills, loans, mortgages, putting food in the table, college tuition, etc.

Where are the “big problem” jobs?

Related Stories