The rapid adoption of artificial intelligence across the data industry is creating a new problem for hedge funds: the growing quantity of information available to investors may be coming at the expense of data quality, according to a report by Business Insider.
The report cites research by data consultancy Neudata as revealing that around one-third of investors that purchase external datasets say the quality of the information they receive has deteriorated over the past two years, with some market participants attributing at least part of the decline to the growing use of AI.
Alternative data has become an important component of the investment process at many hedge funds, providing information outside conventional sources such as regulatory filings and exchange data. Managers use datasets covering areas ranging from consumer behaviour and corporate activity to web traffic and other indicators in an effort to identify signals ahead of the broader market.
The expansion of AI has lowered some of the barriers to collecting, processing and packaging such information, contributing to a proliferation of data providers. But buyers say the technology can introduce new problems when it is used to generate, classify or interpret datasets without sufficient human oversight.
Daniel Entrup, co-founder of data product business AggKnowledge, said providers were increasingly using AI either to produce the datasets they sell or as a reason to reduce or redeploy staff responsible for maintaining data quality.
One fundamental equity investor who buys external datasets told Business Insider that some newer providers appear to be relying heavily on large language models to process information, raising concerns about the reliability of the resulting data.
For hedge funds, the issue extends beyond whether an individual data point is accurate. Managers also need to understand how information has been collected and processed, particularly when using external datasets in ways that could raise regulatory or data-privacy considerations.
Daryl Smith, head of research at Neudata, said funds could encounter problems when vendors are unable to explain adequately how their datasets or reports have been produced.
There are also concerns over the quality of the AI systems used by vendors themselves. Hedge funds with significant technology budgets and internal data-science capabilities may prefer to take raw information from providers and process it through their own infrastructure rather than rely on a vendor’s proprietary AI model.
That approach can also help managers retain control over one of the more closely guarded elements of their investment process: how raw information is transformed into trading signals.
“Funds trust AI with their workflow much more than they trust it with their alpha,” Smith said.
The shift towards AI-led data processing has also coincided with an increase in more basic data-quality problems, according to Entrup, whose company works on correcting and cleaning datasets used by investors.
For hedge funds, even relatively small errors can have significant consequences when datasets are being used to construct investment signals. A flawed observation can potentially distort a model or create a misleading indication that is subsequently incorporated into a trading strategy.
“Once you have a failed signal, it’s polluted data,” one hedge fund manager said.
