When the Robots All Sell

We’ve said repeatedly that AI is powerful, it’s everywhere and it’s here to stay. Still, there are many dangers associated with AI that are being swept under the rug by major frontier AI companies like OpenAI and Anthropic.

One of the dangers in finance is that AI is being used for portfolio management and even to execute specific stock, bond and currency trades.That may produce positive returns initially.

The difficulty is that, as more portfolio managers adopt AI, algorithms trained on overlapping data and built with similar structures could quickly produce herd behavior. This herd behavior could trigger a panic because the algorithms lack human common sense and independent judgment.

Regulators may have to intervene to halt trading until some order can be restored.

In short, AI is the ultimate resonator. In describing the drivers of a potential AI-based market panic, don’t ignore the most powerful component — AI itself can amplify an AI attack.

Many trading and risk management systems used by bankers, brokers and wealth managers have automated or AI functions built into them, whether the user understands them or not.

Most such applications are quite simple and relatively harmless. If a trading system is programmed to detect when a particular stock has dropped 2% in price and warns the user in a soft voice, that’s a kind of automated alert system, but not a dangerous one. It’s up to the human user to decide what to do once the warning is given.

From there, it’s a simple matter to code the system automatically to sell the stock once the 2 percent threshold is hit, without awaiting human instructions. Traders consider such stop-loss features valuable in volatile markets and are relieved not to have to track each stock and make each decision. Systems quickly grow to add new factors, create more complexity, and assume more autonomy.

Some hedge funds are already using AI to make buy and sell decisions without human intervention based on thousands of factors using layered neural networks, deep learning, and millions of pages of training materials. Recent academic research suggests such systems can outperform human analysts and traders.

As such systems proliferate, the danger is that progressively larger percentages of total assets under management are handed over to the robots for pre-programmed trading.

If robots had something like human cognitive diversity even to the point of some robots being more risk averse, trend followers, and still others contrarians, etc. it might be possible to replicate a human-like trading ecosystem in which fear and greed balanced out and a rough equilibrium with bounded volatility could be maintained. This is not the case.

Instead, the input nodes contain mostly the same factors (learned in business schools, economics departments, and Wall Street training programs), edge weights are roughly the same based on regression analysis, the stop-loss limits are the same (as dictated by risk managers using standard tools as ordered by management), and the inputs are identical since we all get the same news at the same time.

Overlapping Strategies

Many training sets overlap, drawing from the same internet pages. Therefore, it should come as no surprise that the outputs are the same. With no top-down coordination and no desire necessarily to achieve this result, all of the robots are singing from the same songbook. The market results can be fearsome.

These and other dangers will emerge at some point. Experts can already describe them in detail. There are documented tests in which frontier models have behaved deceptively or attempted to circumvent safeguards. But that has not stood in the way of the mad dash to launch potentially enormous IPOs for OpenAI and Anthropic in the near future.

One big mistake humans make is to anthropomorphize GPT (and the AI behind it). We start to treat Alexa and Siri and other robots as friends or confidants. We trust them too much. It’s important to realize that there’s no real intelligence behind the facade; it’s all just math.

Along with the overreliance is the fact that these systems are highly error prone. They can perform amazing analytical tasks, but they also make things up (so-called hallucinations) and get some things flat wrong. It’s fine if you’re a subject matter expert yourself and you can spot the errors. It’s more dangerous if the user is not a subject matter expert and is relying on erroneous output to make key decisions.

The Limits of Artificial Intelligence

The worst dangers claimed are to the effect that AI will take over the world and destroy humanity. Fortunately, that’s hyperbole. That won’t happen because superintelligence (or Artificial General Intelligence, AGI) under which humans are to AI what apes are to humans may never be achieved. The AI gurus keep claiming AGI will be achieved and are spending over $1 trillion to prove it, but that money will be wasted.

Computers can’t function at that level. They can find existing knowledge and solve complex problems at lightning speed, but whether they can create new knowledge as humans can remains disputed.

The Law of Conservation of Information in search, developed by my collaborator Bill Dembski and others, argues that AI can redistribute existing information, but it cannot find new information. All AI output comes from existing sources or is smuggled into the model through the query. In my view, only humans can create genuinely new information.

Given these and other problems, it is by no means clear sailing for Anthropic and OpenAI. If their IPOs cannot be launched at the valuations targeted (or if they cannot be launched at all), the reasons may be quite mundane.

These include massive operating losses, negative cash flow and crashing prices for “tokens,” the units of computing power sold by AI companies.

Then there are the mounting legal risks. Lawsuits could arise from harm to other software companies, harm to children and harm to individuals who rely on AI for medical advice, only to suffer serious consequences when that advice proves dangerously flawed.

Another hurdle for the IPOs of Anthropic, OpenAI and other AI aspirants is the competition from China. The Chinese are deploying open-source models, including Kimi, developed by Moonshot AI. This contrasts with the closed, proprietary models favored by leading U.S. AI companies.

China’s Cheap-AI Trap

The Chinese have adopted a clever strategy. They are making their best AI models open source and practically free. That means any company can download the model and run it at prices that can be far cheaper than OpenAI’s offerings. Many corporate technology heads may favor the Chinese models on price alone (and their performance is comparable to the best frontier models from U.S. giants).

But there’s a catch. Downloading open-source model weights does not mean operating them is free. As an enterprise grows, it may face rising infrastructure, support and integration costs, even if those costs remain below U.S. alternatives.

Here’s the hook — any model requires configuration to other systems for interoperability. If Chinese models become deeply embedded in enterprise systems, their developers could gain influence over software ecosystems and operating standards.

It’s like Microsoft from the 1980s onward. In my view, Microsoft software was inferior to UNIX and Macintosh. But Microsoft grew so quickly that the software was locked in at almost every company. You had to keep using successive versions of Windows, all the way to Windows 11, because IT managers did not want to tear down their systems and start over.

Finally, one should consider the leadership of this AI revolution.

Sam Altman’s Empire of Promises

Who is Sam Altman?

We know he’s the CEO of OpenAI, one of the AI giants working on so-called frontier models that push the limits of computer processing power and complex algorithms. That’s an enormously important project that could affect the future of humanity, and he’s in charge of a large part of it. But what do we really know about him?

To begin with, he’s not a scientist or an engineer. He dropped out of college and has no degree. That’s fine; a lot of smart people drop out of college and do quite well in the world. But let’s not assume he has the same technical expertise as the scientists and engineers building the models he promotes.

He’s also not a scholar; he has no peer-reviewed research papers to his credit. That’s OK too. Entrepreneurs may have better things to do than conduct research.

When you peel back the curtain, it turns out that Altman is really just a promoter. In 2005, he started a company called Loopt that failed to gain traction and was sold to another tech company. In 2011, he joined Y Combinator (YC), a highly successful startup investor in hundreds of tech companies. The founder of YC, Paul Graham, later said that Altman was removed as head of YC because of “YC partners’ mistrust.”

Altman went on to co-found OpenAI with Elon Musk and other backers. Peter Thiel was among its early supporters. OpenAI was meant to be a non-profit that would conduct AI research to benefit humanity. Altman pushed it in the direction of a for-profit company earning the distrust of Musk. Musk later sued OpenAI and Altman alleging breach of contract.

So, Altman is a promoter, fundraiser and big talker with a track record of conflicts and alleged broken promises. He’s not the worst person in the world. But it’s not clear he should be in charge of the AI revolution.

On Oct. 8, reports revealed that OpenAI’s annualized revenue was approaching $50 billion, roughly $20 billion below a figure circulated just days earlier. The discrepancy stemmed largely from differences in how AI companies account for sales through cloud partners.

Anthropic and OpenAI are in a race with each other to launch their IPOs. But they’re also in a race with regulators, lawyers and skeptical customers to see if they can even launch the IPOs at all.