HN user

yanglet

15 karma
Posts16
Comments26
View on HN
Is FinGPT Coming? 3 years ago

Is it like a personal assistant that guides an individual through existing practical tools?

A few points to share:

1). Finance is high dynamic. BloombergGPT retrains LLM using a mixed dataset of finance and general sources is too much expensive (1.3M hours). Lightweight adaptation is highly favorable.

2). Internet-scale finance data (timely updates using an automatic data curation pipeline) is critical. BloombergGPT has privileged data access and API access. A promising alternative is "democratizing Internet-scale finance data".

3). Another key technology is "RLHF (Reinforcement learning from human feedback)", which is missing in BloombergGPT. RLHF enables learning individual preferences (risk-aversion level, investing habits, personalized robo-advisor, etc.)

1). Finance is high dynamic. BloombergGPT retrains LLM using a mixed dataset of finance and general sources is too much expensive (1.3M hours). Lightweight adaptation is highly favorable.

2). Internet-scale finance data (timely updates using an automatic data curation pipeline) is critical. BloombergGPT has privileged data access and API access. A promising alternative is "democratizing Internet-scale finance data".

3). Another key technology is "RLHF (Reinforcement learning from human feedback)", which is missing in BloombergGPT. RLHF enables learning individual preferences (risk-aversion level, investing habits, personalized robo-advisor, etc.)

NVIDIA's Isaac Gym project revealed GPU's capability of performing massively parallel simulation for gym-style environments. Detailed information can be found in the following paper:

[1] Makoviychuk, Viktor, et al. "Isaac Gym: High-Performance GPU Based Physics Simulation For Robot Learning." Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2). 2021.

At its release, people commented on Twitter that "it is the MNIST moment for reinforcement learning." And over the past year, I saw several follow-up works and tested NVIDIA's implementations.

For example, a demo by this blog

https://towardsdatascience.com/a-new-era-of-massively-parall...

The question is, does that technique help advance Reinforcement Learning, as expected?

Learn to implement deep reinforcement learning algorithms in 24 hours.

The ElegantRL library is featured with “elegant” in the following aspects:

Lightweight: core codes have less than 1,000 lines, e.g., tutorial.

Efficient: the performance is comparable with Ray RLlib.

Stable: more stable than Stable Baseline 3.

ElegantRL supports state-of-the-art DRL algorithms, including discrete and continuous ones, and provides user-friendly tutorials in Jupyter notebooks.

The ElegantRL implements DRL algorithms under the Actor-Critic framework, where an Agent (a.k.a, a DRL algorithm) consists of an Actor network and a Critic network. Due to the completeness and simplicity of code structure, users are able to easily customize their own agents.

[dead] 6 years ago

Using reinforcement learning to trade multiple stocks through Python and OpenAI Gym, Presented at ICAIF 2020.

FinRL is the open source library for practitioners. To efficiently automate trading, AI4Finance provides this educational resource and makes it easier to learn about deep reinforcement learning (DRL) in quantitative finance.

In quantitative finance, automated trading is essentially making dynamic decisions, namely to decide where to trade, at what price, and what quantity, over a highlystochastic and complex stock market. Taking many complex financialfactors into account, DRL trading agents build a multi-factor model and provide algorithmic trading strategies, which are difficult for human traders

FinRL provides a framework that supports various markets, SOTA DRL algorithms, benchmarks of many quant finance tasks, live trading, etc.