A multi-agent system

The architecture of any system can vary and still perform the same task. A monolithic architecture is best when seeking for performance but, on the other hand, division provides more robustness in case a single module fails and allows making modifications without the need to compile (if this is the case) the whole infrastructure. Furthermore, when talking about computational intensive tasks, where the ratio between data transmission and computation time is very low, a module-based architecture barely impacts the overall performance. Also, a module-based architecture allows the creation of scallable, distributable, highly-available and parallel systems.

A multi-agent system is composed of multiple interacting agents trying to solve problems that are difficult for an individual agent. The main characteristics are:

• Autonomy: the agents are at least partially independent, self-aware and autonomous.
• Local views: no agent has a full global view of the system.
• Decentralization: there is no designated controlling agent.

In OpenSistemas we have developed a general-purpose multi-agent system which is written in pure Python: osBrain.

Each agent is a system process spawned using the multiprocessing module, meaning that it runs independently from the others and that it does not hit performance issues when using GIL-enabled Python interpreters.

This system process starts a Pyro server and registers itself to the name server. The Pyro server is used to serve an object: an instance of the actual agent. This implementation allows the user to access the object through a Pyro proxy and treating the agent, which could be on a remote machine, as a local object and being able to change its attributes and its behavior.

While Pyro is not the most efficient way for communication between processes, it is very convenient for deployment, allowing the creation of complex, distributed multi-agent systems in a simple way.

Agents, however, communicate with each other using ZeroMQ. ZeroMQ is more efficient and very flexible, allowing the user to define different communication patterns based on their needs. A typical agent process will be running a Pyro server in which the main thread runs a loop that simply awaits for incomming messages from the outside. This behavior can be of course modified at will but would be definitely the most common case.

Agents can use multithreading as well and are provided with an inproc socket polled by the main thread to ensure safe access to memory even on GIL-disabled Python interpreters.

A real-time automated trading platform

In OpenSistemas we have developed a broker-independent platform for real-time automated trading: osMarkets. It can gather data from any broker and perform all the necessary computations to produce orders that will be sent back to the broker to be executed.

This platform is implemented over osBrain but having specialized agents:

• Feeder is an agent which receives real-time data from the broker. It typically uses multithreading and the loopback socket in order to be able to stream tick market data while being able to send active requests to the broker (eg. request for historical data). It also converts the data so that it can be in the adequate format for the rest of the platform.
• Router is an agent which receives data from feeders. It manages the historical data and the creation of new bars using real-time tick data. Router distributes updates on the market data to all the interested agents in the network.
• Brain is the most common agent. It receives data from router or from other brains and processes them, sending the results to other brains or sending orders to be executed. This is where the automated trading strategies are to be implemented. Brains can make use of many useful packages avilable in the Python ecosystem: NumPy, SciPy, Numba, Theano... just to name a few. Brains can form a hierarchy that can be used to abstract market data.
• Trader is an agent which is designed to interact with the broker, just as the feeder, but to execute market orders (i.e.: buy/sell). Other parameters such as stop-loss or take-profit can be handled by the broker or internally by osMarkets as well.

In order to manage market data, NumPy ndarrays are being used. When working with real-time data, time series are always changing. In order to avoid full memory copies on each update, we have created a custom class which uses a bigger structure as a buffer. This buffer, which is an actual NumPy ndarray is filled and/or modified on update and the custom class simply updates the view of if in memory.

Visualizing real-time market data

While Matplotlib is probably the most well-known tool for data visualization and while it is very good at displaying all kind of graphics with very good quality, it is not suitable for real-time visualizations and not very good at interaction.

PyQtGraph, on the other hand, is a great tool for real-time visualization and for interactive graphics. It is written in pure Python, so installing this package is pretty straightforward. It uses Qt and OpenGL underneath to allow fast displaying and interactions.

While it is still in its earliest stages, we are developing a tool for real-time visualization of trading strategies using PyQtGraph. This tool acts as an agent in the multi-agent system, meaning that it simply subscribes to updates on market data to router and on the outputs of selected brains.

It is able to:

• Draw multiple candlestick charts on different tabs or windows.
• Draw and configure basic indicators.
• Zoom in and out and move the view box with the mouse.
• Allows the user to handle brains from the interface.