Robots vs. Regulators: How we lost control of the market

What's wrong with this picture?

Source: Nanex.

That depends on whom you ask. My Foolish colleague Matt Koppenheffer pointed out that it shows only the increased presence of high-frequency trading, or HFT, on more tickers across various markets over the past five years. No cause for alarm, right? But many commentators chose to look at it differently. Most wanted to use big, ominous, downright apocalyptic words like "warzone" and "overwhelmed" and "terrifying."

The precise impact of HFT on the markets is still hotly debated, but that just highlights the bigger problem — virtually no one can say with any certainty what that impact is. The algorithms in HFT code are so zealously guarded that Goldman Sachs (NYSE: GS) filed criminal charges against a former employee to keep its "special sauce" secret.

Even if some other financial firm got a hold of Goldman's code, they'd still be a step behind unless they also happened to have their trading machinery parked right in a major exchange's data centre for maximum speed. The higher spikes you see as the graphic above creeps closer to the present are one result of this special hosting service, called co-location, that NYSE Euronext (NYSE: NYX) markets as "a competitive edge… [that] provides unsurpassed value." That premium positioning gives HFT computers a few extra milliseconds to place a bid before any competing orders can sneak in. Talk about being in the right place at the right time.

Is any of this terrifying? Well, we fleshy investors have had a long and complicated relationship with machines, thanks in no small part to their wildly varied portrayal in mass media. For every R2-D2 there's a Dalek, and for every helpful giant Autobot in Transformers there's an equally evil giant Decepticon. Most people don't understand technology well enough to feel comfortable when it malfunctions in high-profile ways. When the market gyrates wildly and fingers point to computerised trading, as happened not too long ago with Knight Capital (NYSE: KCG), explaining it as "robots make mistakes, too" isn't particularly reassuring.

Now, what can we do about it? At the moment, not much, but the answer shouldn't be to throw our hands up and wait for another glitch. The rise of the HFT machines is one symptom of a larger problem: a spreading opacity across the public market that makes it harder for ordinary investors to trust the numbers they find.

Now, what's wrong with this picture?

Sources: Morningstar, Goldman Sachs, and SEC annual reports.

Multiply that big orange wave a few dozen times and you might get a better sense of what regulators are up against. Every year, the resources the SEC wields to keep the financial sector on the straight and narrow are dwarfed by the money Goldman Sachs spends on its annual operations. And that's just one major financial firm that happens to have some HFT running on the side. In the past year alone we've seen multiple instances of financial-industry ethical failures, both in and out of Washington.

• The SEC dropped its investigation of Goldman Sachs after failing to find enough evidence of duplicity in Goldman's sale of a subprime mortgage investment package. Goldman itself has been spinning its actions as "puffery" to evade responsibility for misleading investors.
• Multiple major banks have been implicated in the manipulation of LIBOR rates, which are used to set interest rates for trillions of dollars of debt instruments.
• The STOCK Act, passed earlier this year, has a nice big gaping hole that allows the family members of elected representatives to trade in insider information. That renders moot any hope of trading transparency from Congress, as long as its members can simply pass knowledge on to their favourite uncle or sister-in-law.
• The JOBS Act, a real abomination of "reform" legislation, has already led to such laughable situations as a globally popular 134-year-old British soccer club going public as an "emerging high-growth" company.

And then there are other problems simmering under the surface, like "dark pools" of secondary markets that may make up nearly half of all trading, without ever being visible to you and me. Many HFT actions take place in these dark pools, where they may never be tallied or tracked. That has a real impact on ordinary investors, who don't get to see what happens behind the curtain or understand why.

You might have gotten a glimpse of this growing opacity during Facebook's (Nasdaq: FB) disastrous IPO, when a perfect storm of backroom downgrades, technology glitches, and dueling underwriter priorities led a lot of underinformed retail down a rabbit hole of losses that's yet to abate. Maybe it blindsided you in March, when the HFT-friendly BATS Global Markets exchange performed such a comedy of errors during its IPO that it managed to drag down most stocks with A and B tickers, including everyone's favourite iPhone maker. Technological fumbles are not isolated mistakes, but little can be done to control them when the agency in charge of market regulation is so hopelessly outmatched.

High-frequency trading is not the greatest threat to your portfolio, but it is one of many potential threats that we barely understand. Without real reform, and without some effort to drag these shadowy algorithms into a public light, another technological failure will happen in the near future. You can be sure of it.