How Artificial Intelligence Will Impact the Future of Investing

Artificial intelligence and machine learning are going to play an important, possibly dominant role in investment decision-making, as the combination of high speed data analysis and predictive quantification make it a competitive necessity.

But as the following articles explain, getting the algorithmic assumptions right  is proving harder than expected. JL



Nishant Kumar reports in Bloomberg, Bradley Hope and Juliet Chung report in the Wall Street Journal:

Machine learning’s prowess in finding investing opportunities beyond the reach of humans makes the technology alluring. “If computing power and data generation keep growing at the current rate, machine learning could be 99% of investment management in 25 years,”(But) the protean nature of the markets means yesterday’s relationships can vanish as investors figure them out and take advantage. This isn’t a problem faced by machine learning in other fields, such as converting human speech to text; engineers can count on speech to have the same basic characteristics.
Bloomberg
It was AI versus Warren Buffett.
The artificial intelligence was unleashed by Winton, the London hedge fund, to test an old principle of the Berkshire Hathaway Inc. chairman with a view to trading on it: that major acquisitions usually hurt the buyers’ shareholders. Researchers collected and analyzed data on almost 9,000 U.S. deals back to the 1960s.
The result? Winton says Buffett’s thesis doesn’t hold up — big acquisitions don’t inherently destroy value.

“It prevented us from trading on a false signal and potentially losing money,” said Daniel Mitchell, who runs a team of data scientists at the $30 billion hedge fund. Buffett didn’t respond to a request for comment sent to an assistant.
Bit by bit, AI is laying a claim to the future of investing after many false dawns going back decades. Giant money managers like Two Sigma and Goldman Sachs Group Inc. and smaller players like Schonfeld Strategic Advisors have adopted it as a cornerstone strategy or research tool.
From this foothold, how far will AI go?
Man Group Plc’s Luke Ellis sees a slow takeover coming. The $103.5 billion firm in London already devotes about $13 billion to several hedge funds using machine learning. In 10 years, it will play a role in everything Man does, from executing trades to helping pick securities at the firm’s discretionary unit, Ellis, the chief executive officer, said in an interview.
“If computing power and data generation keep growing at the current rate, then machine learning could be involved in 99 percent of investment management in 25 years,” Ellis said. “It will become ubiquitous in our lives. I don’t think that machine learning is the answer to everything we do. It just can make us better at a lot of things that we do.”
The human toll could be severe: 90,000 jobs in asset management, including fund managers, analysts and back-office staff, out of 300,000 worldwide will go poof by 2025 because of AI, according to estimates by consultancy Opimas from a survey of financial firms.
Quant pioneers like Man Group and Winton have a head start in their AI revamp. The obstacles are daunting for almost everyone else.

Artificial Intelligence Gets Real: A Timeline

AI’s recent feats – beating a Go champion, navigating driverless cars, making money for investors – were many decades in the making.
1950: Alan Turing developed the Turing Test for recognizing machine intelligence
1956: John McCarthy coins “artificial intelligence” at Dartmouth College conference
1957: Invention of Perceptron, an algorithm that could be trained to classify images
1964: Computers understand natural language enough to solve algebra word problems
1968: Stanley Kubrick's 2001: A Space Odyssey features intelligent computer HAL 9000
1979: The Stanford Cart, an autonomous vehicle, navigates across a room full of obstacles
1982: James Simons starts quant investment firm Renaissance Technologies
1988: David Shaw founds D.E. Shaw, an early AI adopter among hedge funds
continued below ...

There’s a paucity of scientists who can create profitable strategies. The wizardry is hard for investors to grasp, keeping some on the sidelines. And the high costs of the technology and data are a burden to firms already suffering fee pressure from the flow of assets to passive funds.
But machine learning’s prowess in finding investing opportunities beyond the reach of humans makes the technology too alluring to ignore. Firms now use AI to prep reams of messy social media and smartphone data, forecast company earnings and sales faster than analysts, decipher the sentiment of executives from documents and create entire strategies.
“Machines will be doing more of the grunt work of discovering opportunities,” said Vasant Dhar, who 20 years ago founded one of the first machine-learning hedge funds, the $350 million Adaptive Quant Trading program at SCT Capital Management. “They can generate hypotheses, test them, and then tell humans, ‘This is interesting, go dig deeper.’ As machines add more value, it changes the nature of work humans do.”
AI strategies also have to wrestle with the assault from passive investing as BlackRock Inc. and Vanguard Group gobble up assets on their way to potentially managing $20 trillion. Index and smart-beta funds threaten to arbitrage away AI’s edge in picking value or growth stocks. But machine learning is showing it can get ahead of the passive wave and exploit patterns in markets that haven’t been discovered, almost becoming a superior version of smart beta.
Investors, fed up with years of lackluster performance by discretionary firms, are buying in. Assets in quant funds, many of which use AI, have surged by 86 percent to $940 billion since 2010. Last year, when fundamental hedge funds suffered $83 billion in outflows, quants took in $13 billion, according to Hedge Fund Research. The trend continued this year through September.
For all of AI’s power with data, its limitations are just as profound. AI lacks imagination, or the human ability to anticipate events — from political to macroeconomic — if such occurrences haven’t happened in the same way many times before. While hedge fund manager John Paulson saw the subprime mortgage meltdown coming, AI would have had no clue, because it wouldn’t have had enough relevant historical data to make comparisons and form an opinion.

A Timeline, continued ...

1990s: AI advances in machine learning, case-based reasoning, data mining, virtual reality
1997: IBM computer Deep Blue beats world chess champion Garry Kasparov
1990s: Web crawlers, other AI-based information programs, become Internet mainstays
1999: Sony AIBO, a robotic pet dog, understands 100 voice commands, learns and matures
2005: Sebastian Thrun’s Stanford team wins DARPA’s 132-mile driverless car race
2011: IBM Watson, a system capable of answering questions, wins quiz show Jeopardy
2012: Google’s self-driving car gets license in Nevada
2014: Man Group starts using machine learning algorithms to manage client money
2016: Alphabet’s DeepMind AlphaGo computer program beats Go champion
2017: AlphaGo Zero learns by playing against itself, beats AlphaGo by 100 games to 0
2017: Facebook switches entirely to neural networks for 4.5 billion translations a day
2017: First AI Powered Equity ETF driven by IBM’s Watson computer starts trading
2017: Two Sigma, a hedge fund that deploys machine learning, crosses $50 billion in assets under management
2040s: AI could be involved in 99 percent of investment management, according to Man Group.
Sources: Bloomberg, Man Group, Superintelligence by Nick Bostrom, Winton

“A machine would have no basis for predicting a crisis since each one is unique,” said Dhar, who’s also a professor of data science and business at NYU. “Humans are good at reasoning about things like a crisis and can sometimes predict it, but we are often wrong. Look at the predictions about interest rates over the last few years.”
Right or wrong, fund managers and their market views will play a major role in the era of AI. Fundamental analysts face a bigger threat.
Firms are sometimes paying almost $1 million in annual compensation for experienced machine learning specialists who can exploit big data. That leaves less money for analysts who research company fundamentals. They may have to learn to code to save their jobs.
“As active managers are forced to spend more money on engineers as their revenues fall, they are going to be forced to slash spending on human equity analysts to protect margins,” said Martin Taylor, who shut down his discretionary hedge fund Nevsky Capital last year in the face of competition from quants. “It’s very depressing for humans.”
Quant firm Acadian Asset Management, where assets soared 79 percent to $93 billion in the past five years, offers a clue to how roles may change in the future.
Managers’ intuition about economic trends are the foundation of Acadian’s long-short and other strategies. Quants then deploy machine learning to refine and improve the 20 most influential factors, from cash flow to unusual events like fraud, that fuel those economic themes to make better predictions. The factors are then plugged into an automated system that takes positions on about 10,000 different stocks across several months or quarters.
Acadian managers and analysts are polymaths: They all have a sophisticated understanding of statistics, and almost everyone writes code and has market experience, said Ryan Stever, director of quantitative global macro research.
The Boston-based firm is investing in AI and big data to better forecast metrics, such as sales, that are key to a company’s performance. If Acadian could wager on sales data before it’s publicly released, the firm would gain an edge.
“You could use machine learning to get the metric earlier, faster and more accurately,” said Wes Chan, director of stock selection research. “If it works, that’s pretty significant.”
An even bigger ambition for some firms is mastery of deep learning, a smarter AI that powers Google’s search and Tesla Inc.’s self-driving cars. Deep learning machines, which loosely mimic activity in the multiple layers of neurons in our brains, require fewer instructions from humans. They make discoveries without being told what to find.
“You will see neural networks become better predictors and better tools for all kinds of trades,” said Juergen Schmidhuber, who helped lay the groundwork for modern AI systems and is a consultant to hedge funds. “Many trades will be executed by self-learning algorithms, with a few high-level guys occasionally injecting human decisions. That’s near-term future.”
Ultimately, the future of AI will depend on its ability to make money. Today’s small group of fully automated AI strategies are off to a middling start. Their performance beats the broader hedge fund industry but not the stock market. Thirteen AI funds gained an average of 10.6 percent annually in six years through 2016, and rose 8.5 percent through October, according to an Eurekahedge index.

The same is true for old-school stock pickers, who will always have a job as long as they produce healthy returns for investors.
AI may have toppled one of Buffett’s pillars. But with Berkshire returning 12.5 percent annually from 2011 through 2016, machines have yet to beat the legendary investor.

         Wall Street Journal:    
For Michael Kharitonov, building a hedge fund based on machine learning has been a rule of threes: It was three times as hard, and it took three times as long, as anticipated.
“Most of the things we’ve tried have failed,” said the co-founder of a little-known firm called Voleon Group.

Machine learning, a set of techniques that empowers computers to find patterns in data without using rules prescribed by humans, has been producing advances in a range of fields, from robotics to weather forecasting to language translation. The technique is at the heart of efforts to build self-driving cars.
Why not use it to crack financial markets? The notion has led to an arms race of sorts, as multibillion-dollar investment firms that already were mathematically focused have been signing up the smartest computer scientists and statisticians they can find.
The gambit seems to be working for two of this year’s top-performing hedge funds. Quantitative Investment Management LLC, up 68% this year in its biggest fund, attributes its success to the technique. Teza Capital Management LLC credits machine learning in part for its more than 50% gain so far this year.
Yet instances of parlaying machine learning into investing success over a sustained period are rare. Much of the reason can been seen in the yearslong struggle of Voleon, one of the first investment firms to commit itself fully to the kind of machine learning that is producing many advances in other fields.
One early lesson: Those other advances might not apply to trading, a messier environment where patterns are often muffled. “The idea that we could just take the machine-learning techniques in speech recognition and computer vision to generate better forecasts just didn’t work,” said Mr. Kharitonov, a curly-haired computer scientist who immigrated from Russia with his family when he was 18. “That was our initial idea.”
Mr. Kharitonov, 54 years old, and co-founder Jon McAuliffe, 43, have Ph.D.s. in computer science and statistics, respectively. Both went into finance and became researchers at D.E. Shaw Group, one of the oldest and most successful “quant” hedge funds. Mr. Kharitonov at one point reported to a young Jeff Bezos, before his boss left to found Amazon.com.
For several years, Messrs. Kharitonov and McAuliffe believed the tools of machine learning, which both had studied, were too crude to use in investing. The methodology was sound, but computers weren’t fast enough and there weren’t enough large data sets to comb.
In contrast to the more common case where a scientist has a hypothesis and writes an algorithm for a computer to carry out, in machine learning a person seeds the computer with vast quantities of data and asks it to figure out patterns all by itself.
The computer, in effect, writes its own algorithm, and uses it to make predictions. How it came up with these, it doesn’t say.

By 2007, new data sets and more-powerful computers had persuaded Mr. Kharitonov, who is known as Misha, and Mr. McAuliffe to start their own firm devoted to investing via machine learning. They formed Voleon, picking the name, which has no particular meaning, because the domain for the website was available to register.
Raising money, they faced deeply skeptical institutional investors. Big quant firms can explain fairly well what their algorithms are doing, having built them. In machine learning, only the computer knows why it did what it did.
The inherent mystery of the approach, where computers detect patterns too subtle for humans to easily comprehend, gave the Voleon principals little hope of being able to explain to potential investors why their firm would buy or sell a stock.
“A lot of people simply weren’t interested,” said Mr. Kharitonov. “But we found a few who understood the potential of machine learning.”
Live trading began in fall 2008, the depths of the financial crisis. For the following two full years, the firm lost money, despite the U.S. stock market’s gradual recovery from its low in March 2009. The Voleon founders plowed ahead, believing they were tackling one of machine learning’s hardest problems and it would take time to hone their system enough to earn profits.
The basic problem they faced was that markets are so chaotic. Machine-learning systems have been best applied so far to situations where patterns are more of a repeating nature, and thus easier to discern, such as in playing the ancient game of Go or even guiding a driverless car. The financial markets are “noisier”—continually being affected by new events, the relationships among which are frequently shifting.
The protean nature of the markets also means yesterday’s relationships can vanish as investors figure them out and move to take advantage of them. This isn’t a problem faced by machine learning in other fields, such as converting human speech to text; computer engineers can count on human speech continuing to have the same basic characteristics.
Even though the success of machine learning in other fields was partly what persuaded them to try it in investing, by late 2011 the Voleon founders had thrown out most of the techniques from other applications. Replacing them was the founders’ own, custom-designed system for the unruly markets.
Books with titles such as “Elements of Large-Sample Theory” and “BDA3” lined the bookshelves in the tidy office of Mr. McAuliffe on a recent visit. Mr. Kharitonov’s office sported a circuit board he had taken apart and a jumble of boxes overflowing with papers.
One challenge the two faced was the need to run 15-year simulations of the stock market using every “tick” of the price of every share. This involved terabytes of data. Voleon needed to run the simulations in hours, but they were taking days and weeks.
At the time, the whole company was around 10 or 12 people. The team tried buying more computing power and using special chips built for computer gaming known as graphics processing units, or GPUs. But it still took too long.
Mr. McAuliffe spent months alone in his office working the problem through in a painstaking process, focusing on intricate details. Finally he cracked it, and Voleon was able to launch a second-generation platform in July 2012.
“The brute-force approach didn’t work,” Mr. Kharitonov said. “The standard techniques didn’t work.”
Their new trading system brought in much better profits, and more investor interest. After modest 2011 returns in its flagship fund, Voleon notched gains of 34.9% in 2012 and 46.3% in 2013, according to an investor.
Still, following two more up years, Voleon suffered a loss of more than 9% in 2016, prompting concern among some of its investors.
“Nothing focuses your mind like a drawdown,” Mr. Kharitonov said, referring to the loss. “We learned a lot from last year.”
This year has been better. The firm, which has about $1.8 billion under management, was up about 4.5% through October in its flagship fund, one investor said. Its annualized return since inception is about 10.5%.
The firm’s uneven results, complicated strategy—and Mr. Kharitonov’s habit of occasionally pausing client meetings to hold phone conversations in Russian—haven’t stopped Voleon from growing. It is broadening its investing target beyond U.S. and European stocks and has expanded into another building near the University of California, Berkeley, campus.
Some investment firms that have tried machine learning use it only for limited tasks, among them AQR Capital Management LLC, which is experimenting with the technique to find the optimal time to roll over futures contracts. Voleon not only is fully focused on machine learning— trading more than $1 billion worth of stocks a day using the technique— but stands out for its complete lack of interest in the reason its system buys one stock and sells another.
The more predictive a machine-learning system is, the more difficult it is for people to comprehend what it is up to, according to Mr. Kharitonov. “You can have maximum explainability or maximum predictive power,” he said, paraphrasing the late Berkeley statistician Leo Breiman.
At the root of this, mathematicians say, is that the human mind is set up to understand scenarios with about three dimensions, while dozens or hundreds are within the grasp of machine-learning systems. Their power comes in discovering connections, often nonlinear, among those dimensions.
This “doesn’t mean we don’t think about what’s going on,” Mr. McAuliffe said. Voleon researchers sometimes design what they call “perturbations” to study the importance of various inputs into the prediction system.
This testing also helps them figure out whether, in certain cases, it might be too tuned to historical data to be useful for forecasts. In statistics, this is known as the problem of “overfitting.”
Voleon’s computers look for relationships in not just financial information but also nonfinancial data sets. Broadly, these could include anything from analyses of satellite images and shipping manifests to credit-card receipts and social-media sentiment about particular companies. Successfully analyzing such data is a goal of quants and non-quants alike as they look for a heads-up about changes in the health of an industry or the supply of a commodity.
The Voleon principals won’t reveal what data they feed into their system, to say nothing of how they have trained it to assess the data. Like other quant firms, Voleon guards its techniques and strategies. Frosted glass on its quarters provides privacy. No sign on the property identifies the firm.
Investors uncomfortable with the mystery of it all have “self-selected” out of the firm, Mr. Kharitonov said. While he understands their discomfort, he believes computers make fewer mistakes than people.
“The application of machine learning science to financial prediction is still in its early stage,” he said. “We are just scratching the surface.”