In the simplest terms, algorithmic trading is using computer to make trading decisions in fully or semi automatic ways. There are several parts.
- Data Collection: collecting historical stock price, fundamental data, and any data that’s useful for prediction. Put these data in a database for backtesting, or feed straight into the prediction algorithm for real time executions. I’ll include data cleaning and some preprocessing steps in this category too.
- Feature Engineering: These are features for machine learning, or any decision making algorithms. They are calculated from the raw data.
- Model Training: Build the algorithm or machine leaning models. Fully backtested and make sure the prediction is good enough.
- Decision Making: Some algorithm, or now days machine learning models making predictions about buy / sell signals.
- Execution: Execute the trades through your broker API based on the buy / sell signals.
- Monitoring: Continually monitor the performance. Retire any algorithms and generate new ones if necessary.
- Automation: Automate everything using pipeline, cloud, etc.
The challenges with algorithm trading includes
- You can over backtest the algorithm and it will overfit the old data, and achieve poor result with any new data.
- You need a team of excellent engineers, financial analysts, economists to build a state of art system. Otherwise you have to trim down your expectations.
- Most publicly available information have been priced in the stock price. So you need more creative ways. For example, monitor parking lot occupancy using camera. Machine learning predicting truthfulness of the CEO speeches, etc. Institution mostly use FIX data, which is the most raw form of trading data. Imagine analyzing bytecode for branch prediction.
As you can see, it’s not easy to do algorithmic trading. I have already given up any ideas of predicting future price movement using historical price or volume information. But I believe there are still several promising areas for average investors like us. When I talk about us, I talk about software engineers who are interested in investing. For example you can build an automatic system to help you analyze company financial statements. Most of these statements are in XML format and freely available from Edgar Online. Build a list of promising companies, and use a separate system to alert you when their price have been dropped and P/E looks attractive. Diversify through 20-30 of these companies and you are good to go. Leave the speculation game to those algorithmic gamblers. By investing, we will achieve much better returns in the long run.