Ask HN: Open problems in finance?

https://news.ycombinator.com/item?id=11047591

by edithv • 10 years ago

55 67 10 years ago

news.ycombinator.com

Hello everyone,

I've seen people working in finance or interested in finance on this board, so I thought I may ask.

According to you, what are some open problems in finance, old or new? I am especially interested in quantitative finance.

I would ultimately like to come up with a list of them, a little bit like we do in mathematics, physics and computer science.

Thanks and have a great day!

jlas• 10 years ago

Predicting the markets

frigaardj• 10 years ago

True digital cash: secure, anonymous, portable, offline etc.

codewithcheese• 10 years ago

+ universally and instantly spendable :)

IkmoIkmo2• 10 years ago

That doesn't really have all that much to do with what is considered finance. It's maybe a bit like saying an open issue in computer science is having a great keyboard.

mdevere• 10 years ago

That's not a problem, that's a product

thomasrossi• 10 years ago

Offline and secure it's the possibly impossible part, but it would be nice!!

Descarte• 10 years ago

The "breakthrough models" are not stable and fall apart as soon as the world financial landscape drastically changes. It took me 17 years to build a working model (black box) and my secret ingredient was an "organic" sub formula that allow the core structure of the model to adjust with the changing financial landscape.

ikeboy• 10 years ago

https://en.wikipedia.org/wiki/List_of_unsolved_problems_in_f...

go_prodev• 10 years ago

Not sure if this would come under your defintion of an "open problem", but in the financial risk space there are many time consuming and costly challenges implementing economic capital models.

These models are used to determine the probability of an account/customer/contagion risk group defaulting (aka PD) and the loss incurred, given a default (LGD).

The current models require a lot of historical data to train them, and need to incorporate economic cycle factors.

There are some vendors in this space, but their products are very dated, expensive and leave a lot to be desired. Also the products really provide a platform for an actuarial team to build upon, rather than having an out of the box solution (which is probably desirable in the banking space)

marincounty• 10 years ago

Good question. I would like to know what's been debunked in finance.

Personally, I cringe when I hear talking heads correlating then Fibonacci number with the stock market.

And I've come to realize, if Jim Crammer recommends a stock, don't buy it. I would like to know which professional out there has the best track record in stock speculation. It seems like something that we could accurately study, and publish the data?

I don't have the money to speculate in the stock market. It just seems like it's as bad as gambling at this point in history. It interests me though. If I had money it would go to realestate speculation. I'm so wrong on my stock picks, I've though about contrarian investing. Supposedly, there are such people?

jleahy• 10 years ago

You're not "so wrong on your stock picks", you're wrong almost exactly 50% of the time (well actually you should be wrong about 40% of the time, but those 40% of bad picks will lose you about the same amount of money as the 60% of good picks earn you).

Just like everybody else, even most professional fund managers, although they wouldn't like to admit it.

Riod• 10 years ago

Quantitative risk measures for longer term fundamental investing. Beta, cointegration, correlation, VAR all break down when there is a discrete event that completely changes the fundamental outlook.

fitzwatermellow• 10 years ago

While your interest is in fintech, do take a look at the forces shaping the broader global macro economic environment because you can literally see some of these "open questions" playing out in real-time!

List of Unsolved Problems in Econ:

https://en.wikipedia.org/wiki/List_of_unsolved_problems_in_e...

What happens when central bankers run out of ammo and realize monetary policy may not be an ideal path to fiscal stimulus? Or having the bright idea of devaluing your currency to depreciate your outstanding liabilities results in a mutually assured destruction scenario when every sovereign on earth implements the same idea simultaneously? And when "too-big-to-fail" entities amass derivative exposure that is 100X global GDP does it become time to panic yet?

But still its innovations like ZeroCoin and SmartContracts that give one hope that out of the rubble, creative solutions will arise. As long as they are grounded in economic reality and not fantasy, humanity will abide...

ThomPete• 10 years ago

Thanks for posting that list. Interesting problems.

One thing that I think is missing from that list is how to factor in technology into economic models or measurements like GDP.

Right now this dark horse which in my view have the biggest single impact on our world is mostly left out as an externality which if you think about it is not just absurd but straight out irresponsible.

Politicians listen to economists and their models and as long as it's kept out of those models they can't react politically to it. And thus technology will keep moving faster than legislation.

wj• 10 years ago

This isn't a quantitive finance problem but more of a consumer finance problem: I really think it is about time there is an API for consumers to access their data at financial institutions. I could see that opening up all kinds of innovation. (Selfishly it would allow the startup I'm working on to multiply the value we offer to users.)

Obviously a standard would be nice but in practice I'm sure each institution would come up with their own or embrace and extend the standard. e.g. FIX with trading.

ape4• 10 years ago

Ready only access would be key too.

smt88• 10 years ago

I looked into this problem extensively a few years ago. Banks are actually moving backwards. The ones that previously exposed an API are killing them off and moving to a proprietary API that talks only to their app. Every retail bank is basically creating their own version of Mint, where you add your accounts from other institutions to their app.

rajacombinator• 10 years ago

How to make money. Unsolved, or at least those who've solved aren't sharing. (100% serious answer.)

collyw• 10 years ago

Have lots of it to begin with. Pay the law makers to rig the game in your favor.

v64• 10 years ago

100% serious response: What do you mean when you say "how to make money"? One can earn wages through employment and invest that money in the stock market (specifically, something relatively safe like an index fund will give you a predictable return). Do you mean how to maximize earnings with minimum effort? How to make over $X per time period?

vinceguidry• 10 years ago

Since you insisted it was a serious answer, I'll bite. Making money relatively straightforward. Money is a type of medium of exchange, it possesses certain qualities that other media do not.

You make money first by creating the tangible / intangible artifacts of that exchange, like currency, systems of accounting, and such. You then get other people to use the system to exchange goods and services.

This typically requires a government, but other entities have made money in the past, and even today. Bitcoin satisfies all the criteria of money.

The practice of fractional-reserve banking is another example of money creation. Most banks will lend out more money than they have access to in deposits, as a relatively safe form of leverage.

This has the effect of slightly expanding how much money is available to the economy, aggregated over all the banks and loans being made by those banks, the expansion is significant.

Things like video game currencies also satisfy all the criteria of money, though the things you can buy with them are limited, sometimes you can trade them for other currencies.

So there's lots of ways to make money.

jedanbik• 10 years ago

Changing legacy business processes to align with the post-recession regulatory landscape.

jmnicolas• 10 years ago

Greed.

NPMaxwell• 10 years ago

How to calculate rate variances and volume variances. There is currently a standard used, but it is misleading.

tyingq2• 10 years ago

There's a potential unsolved problem if you think Bernie Sanders would eventually get his wish of taxing stock trades.

In fact, I suspect there's a Superpac somewhere that would fund research showing the net effect it might have. Or some organization that would want to know what that does to algorithmic trading.

I'm guessing though, the likelihood of a bill like that making it all the way is pretty low.

yummyfajitas7• 10 years ago

Canada tried such a tax, and there is already research showing the net effect it has.

http://qed.econ.queensu.ca/pub/faculty/milne/322/IIROC_FeeCh...

http://www.publications.gc.ca/collections/Collection-R/LoPBd...

The net result is that institutional investors (Goldman) won, retail investors (grandma) lost and spreads (the cost of trading) increased 9%.

I haven't been paying close attention, but I thought Bernie was contrasting himself against Hillary who is supposedly in the pockets of banks. That image doesn't really fit with such a policy.

tyingq2• 10 years ago

Here's what he has on his official site regarding it:

"Has proposed a financial transaction tax which will reduce risky and unproductive high-speed trading and other forms of Wall Street speculation; proceeds would be used to provide debt-free public college education."[1]

[1]https://berniesanders.com/issues/reforming-wall-street/

I'm trying to avoid getting into a debate about whether it's a good idea. If you think, though, it has some chance of happening...it does specifically become an "Open Problem in Quantitative Finance", which is what the OP is asking for.

abrezas2• 10 years ago

One of the problems with the Canadian tax law, was that Canada has to remain competitive to the same-language speaking country south of its borders that has a bigger and more advanced stock exchange. It is explained in the last section of your second link.

The same problems will not be necessarily manifested if the law is applied in the US, especially if Canada has similar laws. However, the points of the first link seem to hold.

ericjang3• 10 years ago

1. A question that has interested me for some time: what is the best quantitative measure of "risk"? Intuitively we understand risk to be related to the distribution of investment returns. More information means that we are more confident about our prediction, hence our risk is lower.

But pinning risk down "mathematically" is kind of tricky. There are several proxies for risk under modern portfolio theory, including variance, standard deviation, shortfall, half-normal variance. Why are there so many different measures? If analytical tractability was not a concern, what should risk models look like?

2. Multi-horizon portfolio optimization. Portfolios are typically optimized one time step (horizon) into the future. However, if one wishes to optimally allocate a portfolio with extremely long-term investment periods (think Berkshire), one needs to subdivide that period into multiple horizons and optimize over these together. This is computationally expensive - is there a way to do this efficiently?

emilioolivares2• 10 years ago

#1 I don't think there is a perfect measure. In practice portfolio managers use multiple views of risk, including multi-factor risk models, stress testing, value at risk, etc. Any tool money can buy they will use.

#2 has already been solved: One formulation is here: https://stanford.edu/~boyd/papers/pdf/dyn_port_opt.pdf There are optimizers out there already on the market with these capabilities.

ericjang3• 10 years ago

#1 - Whether or not there is a formal risk measure that is less "ad-hoc/in-practice" than the others is still up in the air, no?

dragandj• 10 years ago

The measure for risk is probability. Variance, sd, etc. are just attempts to simplify complete probability distribution.

IkmoIkmo2• 10 years ago

1: Risk, just like reward, is an umbrella term. It doesn't specify one particular thing. Take reward for work, some of it is monetary, some in terms of learning experiences, some in terms of friends from work, some reward is in working close to home or at home etc. Risk is similar. The variance you mentions measures volatility, but value-at-risk does not, it's also a risk but it measures the most you could lose at some high degree of probability. For different perspectives and measurements you have different models.

2: My biggest concern here is that the data isn't really reliable in extremely long-term investments, MPT mostly runs on historical data, if you're wanting to pick stocks to invest in 10 years from now on today's data, it's like investing in today's stocks on data from 1996-2006, which is pretty silly. Such very long-term models are just not very feasible. The weather is a great example, I think. We're pretty good at calculating patterns an hour or a day into the future, two weeks into the future is extremely hard. There's just too many small things that can explode into significant changes.

You mentioned Berkshire as an example, they're pretty much the opposite of modern portfolio theory. They look first and foremost at fundamental business analysis, i.e., is it a good business, is there a solid management etc... MPT disregards virtually all of that.

Every model has assumptions, MPT relies pretty heavily on the notion that 'all knowledge is priced in', so that analysing businesses and building portfolios like Berkshire is not going to get you anywhere. Rather, you're left with looking at historical data and building a portfolio like that, assuming that this data includes all information in the market, because it's price data and everything is priced in, so it gives the most accurate reflection of the market. Lots of counterexamples show this isn't the case, it's still a nice theory but it does better in the short (or medium) than the long term imo, and not because it's computationally expensive.

ericjang3• 10 years ago

I see - thanks for the clarification! Looks like they were only "open problems" in my mind :)

anu7df2• 10 years ago

It may be just my ignorance. But I always thought we need a better measure of the health of an economy. (All?) the proxies like GDP are either simplistic or flat out deceptive. GDP for example seems to suggest that throwing a bunch of money around without producing anything improves the health of economy. I realize this is not really a quant finance related question.

imaginenore• 10 years ago

There are many economic metrics apart from the GDP, and they are freely available. Things like national productivity, currency inflation, average amount of savings, etc.

The most telling one, for me personally, is the human condition of the bottom 20% of the residents.

anu7df2• 10 years ago

Yes. Measuring inequality or happiness etc would seem better but are they quantitative and objective? I don't know how this human condition metric is quantified. I will look it up though. Moreover this has the same problem as GDP. Supposing every one is happy and there is very little inequality in the society. But it does not tell me anything about the actual economic progress. Unless I define progress as reducing inequality, in which case this argument is moot.

fixxer• 10 years ago

Any sources of that metric that aren't plagued by political bias, be it progressives or neocons?

deoptimo4• 10 years ago

A metric that I like is mean(log(after-tax income)). It is a reasonable measure of the average "utility" that income provides to a population. If money circulates among corporations but does not change personal income, then the metric remains constant. If everyone's income increases 1%, then the metric increases 1%. If inequality increases, then the metric decreases since average utility also decreases.

yummyfajitas7• 10 years ago

If money circulates among corporations but does not change personal income, then the metric remains constant.

This is true of GDP as well, since GDP only includes final goods. I.e. if a farmer produces wheat, and the wheat is then used to produce bread, the farmer -> baker transaction is excluded from GDP. Only the baker -> consumer transaction counts towards GDP.

If you want to try and measure utility, your metric should be based on consumption - e.g. mean(log(after tax consumption)). Income isn't utility, it's only potential utility.

If we used income, your metric would unfairly penalize volatile income. I.e., a person with a stable income of $50k/year would be considered 70% better than a person who earns $100k then $0k (saving $50k in the first year and spending it in the second while having an identical lifestyle.

peteretep• 10 years ago

The most popular stack with IDE lacks most modern programming tools. I am of course talking about Excel. Where are my unit tests, version control, and so on?

A good start would be a readable and round-trippable format that Excel documents can be saved as

abrezas2• 10 years ago

A good start would be a readable and round-trippable format that Excel documents can be saved as

What happened to CSV?

Edit: Maybe you want to be able to save the equations etc?

laotzu2• 10 years ago

In my university finance classes we were forced to use calculators that were invented in 1976[1] to run calculations on formulas we had to memorize that could easily have been automated in Excel. Since then I've been prejudiced that "the invisible hand" ;) was deliberately keeping electric technology out of finance in order to maintain a legacy business model. Like that of the MPAA towards VCRs and DVDs etc.

[1]https://en.m.wikipedia.org/wiki/Texas_Instruments_Business_A...

elcapitan2• 10 years ago

No exactly an "open problem" (as in science), but an underlying problem is that there is no "google for finance". All financial data is scattered over different places, some of it not openly accessible, in crude formats, etc.

Tharkun2• 10 years ago

There are a couple of "googles for financial transactions" out there: http://www.intix.eu

emilioolivares2• 10 years ago

Agree 100%. Good data is very expensive and even then many times needs additional cleaning. Public data sources are for the most part unusable.

hullaballoo• 10 years ago

what do you mean when you say that public data sources are mostly unusable? what public data sources are there and what renders them unusable?

T-A• 10 years ago

Recently mentioned on HN: https://www.tiingo.com/welcome

And of course https://www.quandl.com/

elcapitan2• 10 years ago

Thanks, I didn't know those yet.

Outdoorsman2• 10 years ago

IMHO financial data is scattered for two primary reasons:

1) "Finance" is extremely fluid because of the speed with with transactions are conducted...(e.g., the stock market)...during the course of a day's trading market reports are given minute by minute, but the most useful "summary" of "performance" is often available only at the end of the day when the market closes...

2) Entities derive a capitalistic advantage by protecting their own data, until, and unless, they're given sufficient incentive to share it...

To me this implies that a system sufficiently complicated enough to provide "meaningful" data captures might have to be nearly as complicated as the field of finance itself...the best we seem to be able to do just now is provide snapshots--AKA, the "leading indicators", etc...

Tharkun2• 10 years ago

3) Data is protected/hoarded/scattered because for regulatory reasons. Some countries (LU and CH come to mind) don't allow "their" financial data to be stored abroad, for instance.

4) There are many (MANY!) financial standards and formats. But at the end of the day, a payment is a payment. There are only so many different kinds of financial instruments and transactions.

There's a lot of noise, but the signal is still there. And solutions exist to make sense of it all.

jharohit• 10 years ago

Remittance is still one the biggest unsolved problem according to me.

I could consider cryptocurrency as the ultimate solution but online if everywhere its being used in equal importance and liquidity. For more traditional money, banks are squeezing fees & spreads out of regular people. Companies like TransferWise are trying to resolve some of these problems but still they rely on the existing banking system to move the funds around.

So there you go - open problem!

laotzu2• 10 years ago

The measurement problem, if by quantitative you mean "relating to, measuring, or measured by the quantity of something".

There is no consistent mathematical standard of measurement by which money is created. Instead, the monetary value of some good or service used as capital for a loan is largely up to the arbitrary appraisal of the bank.

credit_guy• 10 years ago

Risk neutral model selection. In general, if your model has too few parameters, it doesn't capture some features, and if it has too many, it overfits. In statistics, the balance between fit and model parsimony is achieved by information criteria, like Akaike. There is nothing similar for risk neutral models. How do you choose between a one factor Vasicek model and a 10 factor Cheyette with stochastic-local volatility and time-dependent mean-reversions? Currently it's much closer to an art than to a science.

bbcbasic• 10 years ago

How to prevent bubbles from forming. How to provide a viable way to invest in ones retirement that provides some surety.

herbst• 10 years ago

I know not all countries have this problem. But i live in Switzerland and not a single bank offers me some kind of API. All i was asking for is a readonly interface to allow me personal tracking. (We dont do credit card payments for everyday stuff here, we use money)

Ask HN: Open problems in finance?

Related Stories