AI-Run Hedge Fund Creates Cryptocurrency To Reward Data Scientists Based on Accuracy of Their Predictions

As artificial intelligence becomes a more prominent factor in organizational decision making, questions arise about which approaches to compensation will optimize outcomes that are not entirely driven by humans. JL

Motherboard reports:

The new currency will be used where data scientists had an incentive to create models that worked disproportionately well with the test data they were provided, but fared less well predicting new information. The amount of money that a data scientist makes will now be a function of how well their model predicts the market so they can earn more money by being honest about the performance they expect.
The world of financial trading is already dominated by data analysis, but new hedge fund Numerai has taken things a step further.
Just giving a description of the company is like a game of buzzword bingo: it makes investments using predictions made by thousands of artificial intelligence models, which have been designed by thousands of anonymous data scientists around the world who receive payouts in bitcoin. Now, not content with just wowing Silicon Valley with its AI hivemind, the company has now announced that it will be launching its own Ethereum-backed cryptocurrency which will be used to incentivize the amount of cooperation and transparency among its data scientists.
A whitepaper published on Monday explains that the new currency, Numeraire, will be used as a way to tackle a situation where data scientists had an incentive to create models that worked disproportionately well with the test data they were provided, but fared less well predicting new information. Roughly speaking, the paper explains that the amount of money that a data scientist makes will now be a function of how well their model predicts the market, and a Numeraire bet they place on how accurate it is—so they can earn more money by being honest about the performance they expect.
In an inexplicably sea punk video, members of the Numerai team and other investors explains that a similar idea underpins the working of the bitcoin network, where miners are incentivized to use their processing power in a way that strengthens the network as a whole rather than working selfishly.
So why do this with cryptocurrency and not some other kind of reward?
"I wanted something that was super elegant, mathematically," says founder Richard Craib in a phone call. "My background is in maths, and they way we do the staking mechanism makes the auction very simple. The way the whitepaper things out makes it very easy if you're a data scientist to reason about how much you should stake: it naturally ties to how confident you are."
The mechanism is designed to promote self-revelation, a concept described as part of auction theory in which a bidder is moved to truthfully expose rather than conceal their inner values. The blog post accompanying the cryptocurrency announcement also links this to the classic game theory archetype of the Prisoner's Dilemma: a situation where two parties in a transaction who do not trust each other are incentivized to behave in a way that's individually rational, but always leads to poor outcomes for the group.
Approaches based on game theory look for ways to change the incentive structure, by adding either a reward for cooperation or a punishment for defection that is known to all parties, so that cooperative behavior becomes the rational choice even in the absence of trust. The Numeraire currency incorporates this by using Ethereum's smart contract functionality to distribute the rewards (or 'burn' the money from inaccurate bets) in a way that can be publicly audited and is immutable once deployed, without any centralized control from even the Numerai team.
But is this combination of AI, cryptocurrency and a thousands-strong team of scientists really necessary to run a hedgefund? In some ways it isn't, Craib says:
"I could have made a hedge fund without any crowdsourcing, or without incentivizing the data scientists, or anything like that... But what we're trying to do is make a long term hedge fund. Many of these hedge funds do very well for two or three years, and then end up not having long-term sustainable network effects. Numerai is something I want to do forever, and so I'm willing to do all these things while they're a little bit ahead of their time."
With the cryptocurrency having just launched, there's no way to evaluate the incentive structure just yet, and Craib won't comment on the hedge fund's overall performance other to say than it is "doing quite well"; but with global markets having been rocked many times by gambles made by traders using bad information, a mechanism that can inject more trust and transparency into the financial world will be worth watching.