Ask HN: Which industry sector should I target?

https://news.ycombinator.com/item?id=11025571

by leksak • 10 years ago

77 60 10 years ago

news.ycombinator.com

Hi, I am curious if anyone could give any suggestions as to an appropriate sector within the software development industry given my interests.

My interests within the field is not what one would call narrow. Primarily though, I have a fondness for programming language theory, compilers, parsing and writing my own DSLs so that I can make powerful compile-time assertions regarding my code as well as generate tests automatically from the DSL itself.

My StackOverflow cv has an additional set of tasks that pertain to my interests, http://stackoverflow.com/cv/filipallberg.

Regarding my primary interests, the only relevant "thing" I have come up with is web-scraping, but I find it is a dull task as it is a trivial matter to me.

Machine vision is also something that I think a lot about, however most of my ideas are consumer-facing utilities and not something to build a business around. If I get a good idea I'd be willing to go into business myself.

boothead2• 10 years ago

Whereabouts are you? If you're interested in PLT, DSLs and compilers, there are some small enclaves in finance that might interest you.

edit spelling

S4M• 10 years ago

What small enclaves in finance are you talking about? I know that banks have small scripting languages that enable semi-technical users to describe a derivative product ("if condition1 then payoff1 etc.") but those are old and I doubt they are actively maintained and that banks are hiring people for that.

boothead2• 10 years ago

I was talking about banks. I work for one where we build stuff on top of essentially this paper:

http://research.microsoft.com/en-us/um/people/simonpj/Papers...

We're even hiring: https://donsbot.wordpress.com/2016/01/18/haskell-developer-r...

mseebach3• 10 years ago

What do you want to optimise around? Stable job, 9-17, decent salary? Lost of money, caution to the wind? Be something [for one or more of several values of 'something'] to a lot of people? Fiddle with deeply intellectual CS problems, shielded from pedestrian worries about revenue and users?

I personally enjoy the knowledge of having solved a real (business) problem for real people, and being appreciated both financially and personally for this, even when the actual tech going into this isn't very complicated, and especially, doesn't tick off a lot of "hip" boxes.

I have also met good people who do not thrive in these circumstances, so it's hard to give generic advice as to what might make you happy.

throwawayyawa• 10 years ago

I also enjoy this, but not for my current employer (for several reasons I don't feel like going into).

How do you do it, as a consultant or as a 9-5 job? Any advice on getting started as someone who graduated 2014?

mseebach3• 10 years ago

I do it as an employee of a company, but it's a pretty sweet fit for consultants/freelancers (I used to be one).

I don't have any particular advice, beyond the generic stuff: You'll need a network of people who might (know someone who might) want to pay you to help them achieve business outcomes. Remember that you're selling business results, make sure every line on your invoice is part of a narrative that makes (or reduces risk of losing) money for the business.

ZeroFries• 10 years ago

Is it possible to go the "work on deeply intellectual problems while shielded from pedestrian worries about revenue and users" route without academia? I suppose just do it on the side until you invent something which is valuable to industry.

mseebach3• 10 years ago

I know of departments in some of the investment banks that are essentially CS R&D. There are some hedge funds (HFT and otherwise) that are basically playgrounds for PhD.

Of course, in both cases, your work is ultimately directed by revenue, the shield I'm talking about is more against the tight, client/user responsive iterative process of most modern software development which can make it difficult to carve out the time and space to work on something deeper.

Also, I'm not convinced academia is much of a panacea here: I hear there is a nontrivial number of years of pretty severe drudgery before you arrive in the sort of position with a lot of freedom to work on whatever you like.

danieltillett• 10 years ago

I want to suggest something that I would not normally suggest which is you should pair up with a non-technical founder. What you want to be looking for is someone with deep business experience (15+ years) in a narrow business field. Team up with them and work towards creating something amazing, but tightly focused.

pedrodelfino• 10 years ago

Have a look at the legal industry. There are quite a few legal tech startups by now. Even web-scraping applications are rare in legal. The sector is famous for being profitable (high margins), rusty and inefficient.

philodendron2• 10 years ago

Having worked at a fintech startup specializing in decision systems for regulatory compliance, I second this. There are interesting theoretical aspects stemming from the fact that domain is very tricky to model.

* There is considerable structural sharing inside finance regulations, but it is not hierarchical

* Regulations may be changed at any time, and these changes may be future-dated or back-dated

* Users insist on arbitrarily complex exceptions and overrides

* Nontechnical users (particularly legal) will insist on being able to vet and introspect

In my experience, anyone can do this badly, but to make it scalable and robust requires subtlety and a good understanding of formal logic, ontology and DSL design. It's a good fit for FP. Oh, and there's plenty of room for machine learning: Automated discovery, natural language processing, noncompliance detection, etc.

The problem of turning regulations into a expert system that is robust against future changes has not been solved, and there's demand for such systems from even the most sophisticated financial companies, even though everyone expects they'd be able to build such systems in-house.

tmaly• 10 years ago

I second that. regulatory compliance needs some better solutions

harperlee• 10 years ago

That's interesting, may I ask which startup is that? Is it still alive?

lmm5• 10 years ago

I've worked 5-6 programming jobs in 5-6 different industries. Honestly the job is pretty similar in any industry. I'd recommend working for whoever pays best and/or a team you get on with.

sklogic14• 10 years ago

YMMV. A cognitive load and inherent complexity can vary dramatically between industries. Compare web frontend development with, say, a mission critical CPU design.

sanderjd• 10 years ago

I'm always curious when I see comments like this: have you done both things professionally at scale, or are you guessing based on how things seem to you from the outside?

olalonde2• 10 years ago

You state this as if the answer was obvious... I recently switched back to front end development after months of developing a Bitcoin node from scratch as well as a custom BitTorrent server. I'd say among those three, modern front end is what I find most complex. The amount of technologies, protocols, tools and leaky abstractions to know about is just staggering and by the time you finally caught up, you are already behind.

lmm5• 10 years ago

Even then I'd say there's not a lot of difference - you naturally adjust your defect rate to what you're working on (even within a single codebase). I mean sure, when doing something like CPU design it's easy to feel like you're not using your full intellectual capacity the way you would in web frontend development, but you just need to step it up a level - the safety requirements justify putting a fair amount of time and effort into techniques like automatically generated testcases, maybe even formal methods.

One thing that can be harder to overcome is being trapped by processes that you're not allowed to change (and the kind of institutions that make CPUs are particularly prone to that kind of failure, IME). But even then, it just becomes a case of adding as much value as you can within those restrictions, and that can be fulfilling in its own way. But yeah, some organizations (in any industry) are really badly managed and if you can't live with that then your only recourse is to leave.

yitchelle• 10 years ago

I don't have any specific industry for you to get into, but I have a suggestion on how you might be able to find it. Attend as many trade shows as you can and talk to the folks to get an idea of what they are doing, although some of them may be difficult to get to due distance, cost, invites etc.

For example, Embedded World in Nürnberg, Germany provides a good cross section of industry that has embedded engineering involvement in it. There will be a similar one for telecommunications, Automotive etc.

Good luck on searching for your calling.

sklogic14• 10 years ago

I'm very much into DSLs, and I've been building and using them extensively throughout a very wide spectrum: database engines, CADs, hardware design and verification, compilers, device drivers, HPC (GPGPU).

j0rd• 10 years ago

If you like web-scraping, and you're not an idiot, you immediately have an information advantage for anything you decide to do. Do not underestimate this advantage.

I would suggest you leverage your interest in web-scraping and use it to customers in what ever niche you believe you can sell them something.

Collect your "beta" users through web-scraping, figure out a way to reach them at scale. Build/sell a product they want (you can leverage your information advantage to figure out what this is)

This is personally what I do and have created/worked on many bootstrapped companies over my career. Niche for me is fairly irrelevant, as long as I have a pool of interested customers ahead of time. I primarily use my information advantage to figure these out.

The additional skills you will need is:

* learn to think as a "user/customer"

* minor copy-writing skills (or at least understand what shitty copy is and how to improve it)

* UX

* Data mining & Analytics, Analytics, Analytics

* A/B Testing, Iterations, Incremental Improvements (see point above)

* Hypothesis Driven Development (https://www.thoughtworks.com/insights/blog/how-implement-hyp...)

My one suggestion is to avoid any projects which do not have a clear monetization strategy. If you're following my blue-print companies that don't make money from day will only incur costs as you reach your pool of customers at scale.

deskamess• 10 years ago

Can you provide an example or two of a project that involved web scraping? (without divulging client info) I can see how it could be used for price scraping in web retail, but am drawing a blank in other industries.

TheLogothete• 10 years ago

Text analytics, for one.

mtmail• 10 years ago

Address parsing from a large web corpus https://github.com/openvenues/libpostal The author uses http://commoncrawl.org/ data, 250TB/3.6b URL and a 100 machine cluster of 8-core machines.

bjwbell• 10 years ago

Learn the art of writing gpu compilers. Software defined radio is another.

If you find web-scraping dull don't pursue it. Life is short.

sharemywin3• 10 years ago

Which one are you talking about:

Damn Small Linux, a very small Linux distribution

Definitive software library

Domain-specific language, a computer language designed for a specific problem domain

shade23• 10 years ago

DSL= Domain Specific Languages

sharemywin3• 10 years ago

sorry I left the non-technical terms in there I didn't mean to piss everybody off. Thanks for answering.

jndsn402• 10 years ago

Best of luck to you, I am not in the industry and cannot recommend from that perspective.

As a consumer though, augmented reality ( eg Hololens and Magic Leap) seems like it is about to become very big, and probably utilizes machine vision.

Can you recommend some resources for learning about web scraping?

sharemywin3• 10 years ago

you should look at marketplaces. possibly even ones run by crypto currencies. There are alot of ways to configure a marketplace and it would be nice to abstract that from CRUD operations and such.

drelihan• 10 years ago

you're right, web scraping is dull ( it was a big, but important part of my job for a number of years ). The interesting, and non-trivial, part comes on how to use the data you have scraped.

cableshaft• 10 years ago

I'd just suggest looking into something up and coming that interests you, and start learning whatever you can about that.

Industries that will have specific uses for machine-vision: autonomous driving cars, drones, robotics, augmented reality, internet of things

Others: crypto-currencies (specifically digital ledgers/blockchains), virtual reality, commercial space exploration, online education, voice recognition, (more as I think of them)

Tech that's already big and probably getting bigger: streaming video, instant stock trading based on algorithms,

Certainly one of those must be interesting to you. They're all pretty interesting to me.

lafay• 10 years ago

The technology to cost-effectively store massive amounts of raw metrics / data and quickly derive insight from it is still a relatively recent development, and so far we've seen turn-key tools to leverage that in just a few areas:

1. Where developers themselves are the likely product users (think Google Analytics, New Relic, Optimizely, Appdynamics)

2. Where there is a direct and immediate connection to $$$ (finance / quant stuff)

3. Generic data science platforms (Cloudera, Hortonworks, Databricks)

I think there is still tons of opportunity for industry-specific turn-key data science tools. Especially for verticals where there is a low coincidence of skilled developers with industry domain expertise.

HFTGuru• 10 years ago

If you are interested in huge bonuses up to 400k-1M a year for being a developer, then I suggest to you high frequency trading development. The problem is you'll have to move close to an exchange, but the finance industry rewards you handsomely and takes care of it's employees.

al2o3cr• 10 years ago

IMO your question isn't really answerable: you've listed tools and techniques you enjoy but "sectors" in the industry are primarily about producing results using those. A similar realspace question would be, "I really enjoy building cantilevers and arches. What city should I look for construction work in?"

My personal recommendation would be to find people who are doing interesting things and talking about it in public - blog posts, conference talks, etc etc and reach out to them.

scardine• 10 years ago

You can do pretty well as a Google for Work reseller.

I can make 1 USD per account per month, at 10k accounts it is a pretty good source of passive income.

leksak• 10 years ago

Interesting proposition for a static income. I'll look into it

Ask HN: Which industry sector should I target?

Related Stories