Nowadays Python is the perfect environment for developing a real-time automated trading tool. In this talk we will discuss the development of: a general-purpose multiagent-system module using Pyro and ZeroMQ; a platform, based on it, for developing automated trading strategies using Numpy, Numba, Theano, etc.; and a GUI for visualizing real-time market data using PyQtGraph and Qt.
The architecture of any system can vary and still perform the same task. A monolithic architecture is best when seeking for performance but, on the other hand, division provides more robustness in case a single module fails and allows making modifications without the need to compile (if this is the case) the whole infrastructure. Furthermore, when talking about computational intensive tasks, where the ratio between data transmission and computation time is very low, a module-based architecture barely impacts the overall performance. Also, a module-based architecture allows the creation of scallable, distributable, highly-available and parallel systems.
A multi-agent system is composed of multiple interacting agents trying to solve problems that are difficult for an individual agent. The main characteristics are:
In OpenSistemas we have developed a general-purpose multi-agent system which is written in pure Python: osBrain.
Each agent is a system process spawned using the multiprocessing module, meaning that it runs independently from the others and that it does not hit performance issues when using GIL-enabled Python interpreters.
This system process starts a Pyro server and registers itself to the name server. The Pyro server is used to serve an object: an instance of the actual agent. This implementation allows the user to access the object through a Pyro proxy and treating the agent, which could be on a remote machine, as a local object and being able to change its attributes and its behavior.
While Pyro is not the most efficient way for communication between processes, it is very convenient for deployment, allowing the creation of complex, distributed multi-agent systems in a simple way.
Agents, however, communicate with each other using ZeroMQ. ZeroMQ is more efficient and very flexible, allowing the user to define different communication patterns based on their needs. A typical agent process will be running a Pyro server in which the main thread runs a loop that simply awaits for incomming messages from the outside. This behavior can be of course modified at will but would be definitely the most common case.
Agents can use multithreading as well and are provided with an inproc socket polled by the main thread to ensure safe access to memory even on GIL-disabled Python interpreters.
In OpenSistemas we have developed a broker-independent platform for real-time automated trading: osMarkets. It can gather data from any broker and perform all the necessary computations to produce orders that will be sent back to the broker to be executed.
This platform is implemented over osBrain but having specialized agents:
In order to manage market data, NumPy ndarrays are being used. When working with real-time data, time series are always changing. In order to avoid full memory copies on each update, we have created a custom class which uses a bigger structure as a buffer. This buffer, which is an actual NumPy ndarray is filled and/or modified on update and the custom class simply updates the view of if in memory.
While Matplotlib is probably the most well-known tool for data visualization and while it is very good at displaying all kind of graphics with very good quality, it is not suitable for real-time visualizations and not very good at interaction.
PyQtGraph, on the other hand, is a great tool for real-time visualization and for interactive graphics. It is written in pure Python, so installing this package is pretty straightforward. It uses Qt and OpenGL underneath to allow fast displaying and interactions.
While it is still in its earliest stages, we are developing a tool for real-time visualization of trading strategies using PyQtGraph. This tool acts as an agent in the multi-agent system, meaning that it simply subscribes to updates on market data to router and on the outputs of selected brains.
It is able to: