Presentation of the first complete AI investment platform. It is based on most innovative AI methods: most advanced neural networks (ResNet/DenseNet, LSTM, GAN autoencoders) and reinforcement learning for risk control and position sizing using Alpha Zero approach.

It shows how the complex AI system which covers both supervised and reinforcement learning could be successfully used to investment portfolio optimization in real time. The architecture of the platform and used algorithms will be presented together with the workflow of machine learning. Also, the real demo of the platform will be shown. The M4 conference winning forecasting method will be presented, with the application to financial time series forecasting.

- Anya is senior data scientist in seven Bullscom. She is working with the time series AI forecasting methods. Pavos Kripak is a cto in the AI investments company. We will present how to use the complex AI based forecasting methods for investments portfolio optimization.
- Time series forecasting is applicable for many, many applications. Machine learning based methods have significantly higher accuracy than statistical method. M four and M five competition show that machine learning methods are simply better than statistical one by significant margin of accuracy.
- How can we access stock price data from Python? For just learning purposes we will use Python package y finance based on Apache software license for Yahoo Finance historical data. Time series transformation might be crucial for some of the statistical models. And finally we will try create an automatic investment strategy.
- Our goal will be prediction of closed stock price. We will use daily and weekly frequency, usually stock prices. Next step will be creating a fully automatic investment strategy with reinforcement learning.

Hello, my name is Pavos Kripak and today
with Anna Warno we will present how
to use the complex AI based forecasting methods
for investments portfolio optimization. Anya is
senior data scientist in seven Bullscom.
She is working with the time series AI forecasting methods,
also other machine learning methods and I'm
a cto in the AI investments company which
aims to build the complete portfolio optimization
platform using the latest achievements in machine learning
methods. Even more, we have built the first version
of this software which is currently deployed for one
of the investment funds from the Luxembourg.
Today we want to tell you about the
time series and what are the most advanced
and available currently method. And in the second part,
Anya will show the real hands on session
how to work with the time series, how to use this
method, what are the results, and even more,
Anya will show how to use the reinforcement learning. For that
purposes, of course I want some toy example, but to
give you feeling and impression I will go through
this method listed here very briefly and in
the details. Anya will show NBIT's method
and complete this NBIT's methods to the typical statistical
method of the forecasting. That's the agenda.
Let's start why we are talking about this
topic. Recently we are observing the dynamic
development of the time series forecasting method,
especially machine learning based methods
which has significantly higher
accuracy than statistical method. Time series
forecasting is applicable for many, many applications from
forecasting sales stock, some make
some forecasting in health and also in visual recognition
and so on. But it is also applicable for
financial time series and for financial time series we
are able to achieve over 60% of
the accuracy, which is a very significant
number for the financial time series. For other time series
which are easier to forecast. Of course it's not a very impressive number,
but this one gives a significant edge
in the portfolio construction and the results of
the investments shows that. What is time
series? Time series is the ordered time list of
all values with the given attribute one or
more, and time series forecasting is the ability to
forecast the future value of that series which is
not known in the given time and machine learning
time series. AI forecasting methods are recently
developed and achieve very good results. There are
methods of the time series forecasting which are based on the
machine learning models and methods.
Till this time or two years ago,
three years ago, there was the domination of the
statistical method of the forecasting arma Rima,
exponential smoothing and other methods. But recent
evolution and recent breakthrough in
forecasting which probably the most significant point
is the M four and M five competition show that
machine learning methods are simply better
than statistical one by significant margin
of the accuracy. M competition is
probably the most prestigious and for sure scientifically
baked competition related to the time series.
AI forecasting methods organized by University of Nicosia
and Professor Spiros Makriakis. And the
fourth edition of that competition, it was
over two years ago, almost three years ago, and in
this fourth edition 1st 2nd
place were taken by machine learning based
method. One of this method will be presented very briefly today
and the M five competition has been finished.
Previous year was dominated by the machine learning method.
What is important about M four competition is that it was
on the very huge number of the time series.
There was 100,000 different time series
with different number of points and different frequency. So the
method which won was really tested on huge
data set. And this winning method is ES hybrid
methods. It was designed by smell
data scientist from Uber and there are some
very unique elements of that method.
First one is to include data preprocessing
into the back propagation. So the parameters of the
data processing which was exponential smoothing were
learned together with learning weights
of the neural networks. Second thing was very
unique approach for the assembling and
assigning models cto the given time series. And third thing
was the neural networks. In this methods
are used LSTM networks, but not typical LSTM,
but residual delighted and with attention.
So this residuality was one of the first
application of this concept. It's very heavy
applicable for the image recognition to the LSTM
and two time series forecasting and it gives
very good results, especially that suave use
different residuality rules for different
frequency of the time series. Second method
which will be described and presented by Anya
later in the session is the NBITS method. This is
the purely machine learning method comparing to the
ES hybrid which is considered hybrid
and it is constructed from
the block. So it is some kind of the stacked architecture with
also very unique construction. There are different type
of the blocks for trends, seasonality and some generic
blocks. And also there are some features related
to the explainability and transfer learning. And also
the NBITs uses advanced model ensembling.
Here we have a brief overview how the
NBITs looks. Here is also the reference
to the paper. More details are provided.
Third method, probably the easiest one to use is Facebook
profit. It is based on the general's additive models.
It is available as a library and
also provide quite good accuracy
levels. And the important thing of that method
is that we are able to calculate also the range
of the uncertainty or confidential levels.
Not only the one value and this method
is available as a library. So it
is very easy to start using this method for your
own proposals. Third method or fourth
method is the glue on TS. It is also complete framework
of the libraries also available to download and use.
It is more difficult to use than just profit,
but much more powerful. There are many methods included,
many way of the preprocessing data. It has
a support for the cloud computing and also
very strong support from the community and scientific
researchers. Also here we have the paper to the
TS. There are very interesting concept of
different methods, different network structures,
probabilistic distributions,
type denveral components, it is constantly evoluted
and new elements are added. And also it's
available, as I said, as a library so it can be download and
use. And the third method is very unique based
on that. Settling machines, also called stochastic learning
automata is very unique for the forecasting.
But stochastic learning automata itself is known for
the long time since the 60s from previous century.
Here we have one of the latest paper
related to that topic and the main
differences of settling machine comparing to the neural networks
is that it learns the probability distribution for each
parameter and each of the feature and based on that
it is able to forecast the given value.
Yeah, that's all from my side. Now Anya will tell
a little bit more, even much more than
my introductionary part, how to use
forecasting method on the example of
the NBITs and compare NBITs with the statistical
methods. And also we'll show the toy example how
to use the reinforcement learning hello.
Today I would like to show you what can be done with financial time series
data, stockpresses and Python. We'll go
through the whole process from downloading data to Python
EdA. It is explanatory data analysis,
time series data, preprocessing modeling for time series forecasting.
And finally we will try create an automatic
investment strategy. So how can we access stock
price data from Python? There exists many rich API,
but for just learning purposes we will use Python
package y finance based on Apache software
license for Yahoo Finance historical data. Thanks to
this package we can easily download daily stock prices
data with columns such as open, high, low,
close volume, etc.
We just need to pass company index. Here I have some
saved example company indices and we will run
one. And here we can see the output with our downloaded
data frame for visualization and indicators
calculation. We don't need to implement everything from scratch because
Python has many libraries which might do the work for us.
For example, I will use Quantstat library which
has around, I think 20 implemented visualizations
and functions useful for stock price analysis.
Here are some of the plots drawn with this library.
So after the general data analysis we will move cto the data preparation.
So the step before the modeling part, because the topic
of this conference is Python, not machine learning, and maybe some of
you are not familiar with data science, I will not discuss with
the details what is the purpose of some of the specific activities.
I will just say that it will help to choose the models and
hyperparameters range during the further steps.
What can be done during the time series analysis and data preprocessing?
Usually we do the stuff like seasonal decomposition
to relate the trend or season from the data. Time series
transformation might be crucial for some of the
statistical models. Outlier detection helps to
understand the data. Most of those tasks can be
done with stats models package, which provides classes and functions for
the estimation of many different statistical models.
There is belief that r is much better, at least for the classical
time series analysis, than Python. But this task model package,
at least in my opinion well, substitutes r and even
has a similar API. And here we have some example transformations
which are often applied when we are dealing with stock prices.
Some of them are extremely easy like lock, but some of them, like Kalman Filter,
are complex and harder to implement from scratch.
Fortunately in Python we have also package for that.
Maybe I will show you this Kalman filter because it's interesting,
it has very nice smoothing features.
And why do we need data transformation?
There is many reasons. Some of the models require special inputs.
Sometimes it helps also with numerical issues. Okay,
so we have very briefly went through examples of data preprocessing
which might be applied to time series data. And we can
now move to more interesting part the modeling.
So our goal will be prediction of closed stock price.
We will use daily and weekly frequency, usually stock
prices. Series with these frequencies are very,
very noisy and very hard to predict. Currently efficient
models use more data than univariate series
from single stock can provide. For example,
recently tech data like financial news or tweets are
taken also as model input for stock price prediction.
But today we just want to show very simple
examples. So the first step would be division of our
closed time series into three sets, train valve and
test. Those familiar with machine learning will know what the purpose
of this division is, but for those who are not on the train
set, we will train. Our model validation set is
for choosing the best hyperparameters,
and on the test set we can see how our model
performs on unseen data.
It's a good practice to firstly try with benchmark
models before starting to train complex state of
the art. We have three benchmarks, name prediction
where we are predicting always the last seen value,
and two classic statistical models, exponential smoothing
and Ariba. Both of them came from stat model package.
So after training the baselines, we can choose more complete
model. Our choice will be NBIT deep learning neural networks which
outperformed all models on famous, prestigious and
four time series competition where the task was prediction on 100,000
time series from different fields including finances.
There exists a few NBIS implementation, but we
will use one from Pytorch forecasting as it's very well developed
and based on very convenient pytorch lighting library.
So after the training, choosing the best hyperparameters and
evaluation of all of our models, we have time for model comparison
in the data frame here we can see calculated metrics.
The best methods are highlighted with green color first. Metrics such as
mean absolutes error or mean squared error
indicate how much values predicted by our model are close.
CTo the reals as you can see, naive method achieves surprisingly
good scores. However, from investor point of view,
its prediction is worthless. It does not give us any
information whether we can actually earn something. It's a very
common mistake. During the stock price prediction,
people look at metrics like mean absolute error,
plot presented values and everything looks fine at
the first sight. But it turns out that their
model is not better than net prediction and
that the predicted values are just shifted.
To overcome that issue, we have to calculate how accurately
model predicts prices, ups and downs. Results are
placed in the last column. Okay, so all
of these results presented here are only example numbers which
differ a lot across stocks. Sometimes they are better,
sometimes they are worse. If we decide that results are promising,
we can try CTO, trust our predictions and build an
investment strategy with larger and more accurate
predictions horizons, our investment strategy will became more and
more complete. Can we have model which will plan everything
for us? Yes, we can. So we will go to the last
step, creating a fully automatic investment strategy with
reinforcement learning. For those who are not familiar with reinforcement learning,
you probably heard about super powerful AI models which
are able to became masters of go. I think that's the most popular
example of usage of reinforcement learning. Or maybe
you heard about models which learned how to play Atari games.
Reinforcement learning is an area of machine learning concerns with
how intelligent agents also take actions in environment in
order to maximize the notion of cumulative reward. So it's like showing
model a game show that it should maximize game score
and using just on that it will learn how to win. Stock resembles
game where profit is the reward. It's a very hard game with
huge randomness factor, but still we can try use
reinforcement learning for that. So first we need to
have an environment. There is several already implemented stock
simulation environments in Python like for example Finerl.
Most of them is based on OpenAI gym library and
we can use them, but we can also create our own
environment. So I just want to emphasize that we will just
make it as simple as possible, just for fun and
in real scenarios to actually work well.
Our solution should be more complete. Take more data,
more training epochs, more agents, more complicated
policies. But just for today we'll build something extremely
simple. We'll use OpenAI gym library and
single stock data will be used. Observation space will
be just a vector with past and values. Currently owned shares
plus currently owned money plus time for end plus
predictions and predictions step ahead as predictions
are created with models from the previous steps and our
reward would be money plus owned shares multiplied
by current stock price. And you have two possible actions,
buying and selling shares in percentages. Each day
we can sell up to 100% of owned shares
and we can buy maximum free shares. Our game
will always end after 20 days. In reality it's extremely
short period for investment, but we'll leave it short
because of the training time. We'll use PPO algorithms
from stable baseline free and in stable baseline free
we can add our own Pytorch models which might be good
for further development. And for neural networks we
will use the simplest option, multilayer perceptrons.
And now I can show you the code. And the environment
is longer, so it's probably less understable.
The most important functions in environment class
are this obligatory from gym amp like
step in step function.
Each day we take action,
calculate the reward, take the next observation from
the environment, and check whether we are not done.
And here we have training and evaluation codes which
are shorter. And these codes also should
not be different if we use another environment.
We evaluate our agent on unseen data last 365
days for one year from our time series, and we are
running our model 365 times
each time with different starting day. And the biggest
question is whether our investment strategy work. Does it have
a potential? Firstly, we can check whether we are
earning anything, or maybe we are just losing money. After that
we should compare our method with other strategies. There is many methods
for backtesting investment strategy. There are even nice package
for that in Python. But for now we will just test two
very simple strategies. Buy and hold and nest strategy.
Nest strategy relies on buying if the
last n values are higher than the current
price, and sell whether the last n values
are lower than today's price, with the same limit
for buying actions as in our environment.
Okay, so here are the results from the four consecutive
runs with four different randomly selected companies.
And in real scenarios here
we have profit in dollars, but the better way is to
calculate it in percentage.
And as we can see, our reinforcement learning agent
is not the worst. Of course,
these plots here, these results cannot be treated as
a proof or anything because it's hard to think about fair
conditions for comparisons of these strategies.
But it looks like there might be potential in
this method. It seems that our model actually learned
something. Okay, and that's all for
today. Thank you for your attention.

Priority access to all content

Video hallway track

Community chat

Exclusive promotions and giveaways