Global Futures Commission Merchant (FCM) and Broker Dealer (BD) Algorithm Due Diligence
A global FCM was presented with access to a highly successful algorithm by a technology provider. The provider had convinced several trading desks that this could help the FCM grow their execution desks by providing better fills than traditional accumulation strategies.
Compliance, Risk Management, and Information Technology needed proper due diligence on the algorithm and the integration with Trading Technologies. However the only testing environment available within TT was simulated markets with near real time data.
Proper due diligence for an algorithm, especially one being implemented by a global bank’s FCM, requires much more rigorous testing procedures.
Testing an Algorithm
The August 2012 catastrophic failure of Knight Capital Group’s algorithm that resulted in $440 million in losses proved that algorithm testing is essential. To avoid problems with algorithms there are a number of practices that should be followed. The Futures Industry Association lays out an approach in their Guide to the Development and Operation of Automated Trading Systems[i]. This industry recommended industry testing covers operations, integration, and testing methodology.
The FCM and Edge agreed meet the FIA guidelines and to add strong functional testing covering the following conditions:
- High volatility – Markets moving rapidly up or down
- High rate of change in prices – Markets rapidly moving slightly up or down but not moving in any one direction
- Flash Crash situations – Extreme drop in prices with corresponding liquidity changes
- Slow market conditions – Markets not moving or changing at all
- Thinly traded markets – Markets with very few contracts in the depth of market
- Volume on Bid or Offer disappearing – One side of the market not available for trading
- Loss of Market Data Feeds – A technology failure of the market data feed with and without technical notifications
- Market Open & Close conditions – Testing order types and auctions that are available during the market open and close periods
- Holiday processing (early market close)
- Halting the algorithm intraday
- Exceeding pre-trade limits within the trading system (position, fat finger, etc.)
- Administrative disabling the trader from all trading while the algo is running.
Why Edge Financial Technologies?
In order to test the algorithm the FCM needed a fully functioning TT installation but connected to simulated markets fully under the control of the testing team. Edge has simulated exchange matching engines, and market data systems that can attached to any ISV or trading system. These components allowed the testers to be able to conduct fully systematic testing.
The Algo Test Results
The test results proved that the algorithm did not meet its stated goals. It failed on many occasions to fully fill the order quantity entered by the broker.
Technology issues in the algorithm became quickly apparent during the strong functional testing. The algorithm showed a number of issues especially when dealing with adverse market conditions. These problems included:
- Failure to detect thin market conditions where the algo order quantity would overwhelm the rest of the market.
- Inability to respond to or understand market halt conditions (limit up, limit down, early close)
- Inability to respond to orders being rejected by pre-trade risk control breaches.
- Inability to respond to loss of market data.
- Inability to deal with rapid price changes like those in a flash crash.
- Failure to maintain order state information resulting in unknown status of quantity working, quantity filled, and quantity left to be submitted to the market.
These failure conditions were further hidden and complicated by technology decisions of the algorithm designer and developer. Error messages were generic, only indicating something had occurred. The developer did not want others to see or be able to understand how the algo functioned and therefore made a decision to be “very general” in the trader feedback messages. Operational support people and traders were therefore unable to determine causes of issues.
The Business Results
The FCM determined that this particular algorithm would not be suitable for their business.
As a broker dealer the FCM needed to know that orders that were entered into the market would be fully filled. This particular algo, originally developed for a buy side trader, was more interested in “best pricing” and not in filling the full order quantity. This alone was enough to cause the broker not to implement the algo.
The technical test results also showed the broker that the algo was not ready for full utilization. While the algo initially appeared sound, and had good references from buy side traders, it was determined to be dangerous for the FCM and would likely increase the number of trading errors.
The FCM was very pleased to have strong statistical and technical reasons for making their decision and were able to convince the sales desks that the algo was inappropriate for their desk.