Quantitative hedge fund manager Man AHL has released Arctic, its MongoDB-powered financial tick store, on code collaboration and management site GitHub as a freely available open source project.
By building Arctic on MongoDB, Man AHL realised a 40x cost saving when compared to the legacy time series data store. Processing performance improved by 25x. Man AHL is based in London, Hong Kong, Oxford and Pfäffikon, and has USD17.9 billion in assets under management as of September 2015.
Gary Collier (pictured), Co-CTO of Man AHL, says: “MongoDB has provided tremendous value through the improvements it delivered to our Arctic tick store. It has enabled us to be more agile and it has delivered dramatic performance processing gains.”
A tick store is a specialised type of data tool for storing high frequency financial services market data. For instance, Arctic can be used to store time series data like the fluctuating cost of stocks as well as lower frequency data. Arctic is the primary market data store for the hedge fund manager’s quantitative researchers (quants).
“Making Arctic available as an open source project is just a natural extension of our company’s culture of freedom and collaboration. My hope is that we are planting a seed that will disrupt how financial data management tools are purchased, used and shared,” says Collier. “What we’ve done with MongoDB has shown there may be a better and more powerful way to process modern financial data. Arctic’s technical performance has been truly outstanding and we’re excited to see what the community makes of our Tick Store.”
Arctic, which has been under development by Man AHL since 2012, can now scale to hundreds of millions of rows per second for each MongoDB instance. It can also comfortably ingest 800 million financial messages a day – 50 per cent more messages than there are tweets on Twitter in one day.
MongoDB was originally selected because it fit so well with the development team’s preference for the open source Python programming language. However, early tests found the non-relational database to be a significant improvement in terms of development speed and orders of magnitude more performant than the incumbent technology.
Arctic was then put into production and MongoDB replaced a number of disparate technologies in the tick store’s data stack. With MongoDB, Arctic saw radical improvements that enabled quants to fit models 25x as fast as they could previously. The new tick store also cut disk storage down to 40% of the previous solution.
MongoDB is also used in other parts of Man AHL’s business, including as a replacement database for its Low Frequency Data, where MongoDB was 100x faster in retrieving data.
“The processing speed that Man AHL has achieved with MongoDB is truly staggering. A 25x improvement on a system that wasn’t slow to begin with? That’s incredible, and it’s the sort of thing that changes how industries operate,” says Joe Morrissey, VP of EMEA, MongoDB.
“We always support customers looking to give back to the open source community. It’s fantastic to see Man AHL take a leading role in adoption of, as well as contribution to, open source in the financial services.”
Man AHL is already seeing keen interest from the community in Arctic on GitHub. The team also believe that the tick store has use cases far beyond the financial services. For example it could also be deployed for a range of write-intensive workloads such as internet of things applications and social media feeds. One company has created its own fork of the code and using the tick store for sentiment analysis.
