Markets are giant calculators outputting the price of risk. Those outputs are fed back into our choices ultimately determining where resources flow. I call this appreciative knowledge because it supports one’s understanding of how the system works. I differentiate it from instrumental knowledge because it is not prescriptive. It’s more like the grout in your understanding. It holds the tiles of logic together and it’s aesthetically satisfying
By the end of this little journey, the role of “complete” markets will make more sense at the system level.
📱
Financial innovation and “complete markets”
Voting and markets are both mechanisms for coordinating human behavior and expressing their preferences. Let’s just jump in:
The Problem With Voting
Voting has an egalitarian appeal. It is fair and just. But in the 18th century, French mathematician and philosopher the Marquis de Condorcet showed that it’s outputs can be “incoherent”. 2 centuries later, economist Kenneth Arrow formalized this idea as “The Impossibility Theorem”.
Arrow’s most famous scholarly achievement is his celebrated ‘impossibility theorem’, which lies at the heart of understanding how a government, or other collective decision-making process, can employ individual preferences as inputs from which decisions are determined. Dictatorships can do this with ease: a leader’s preferences determine action. Other cases are far more complex.
Intuitively, democracies aspire to assign equal weight to voters and produce policies that are the ‘will of the people’. But how should differences in preferred outcomes be evaluated? Should everyone have one vote per election, regardless of the intensity of their feelings about the issues at stake? Should voting be first past the post, or should proportional representation be followed? There are many ways to produce collective choices.
We have powerful intuitions as to how collective choice procedures should function. One is that if everyone prefers one policy to another, the latter should never be preferred. Another desideratum is that voting procedures should always produce coherent decisions. But as the Marquis de Condorcet showed in 1785, incoherence can occur in a majority voting system where choices are sequentially considered pairwise: specifically, that it is possible that candidate A defeats B, B defeats C, but C defeats A, thereby failing to produce a coherent notion of a winner.
Condorcet’s remarkable insight leads to the question of whether alternative voting schemes can avoid such outcomes. Arrow’s even more remarkable (1951a) analysis, which was his doctoral dissertation, asked whether there can be any procedure that respects the preferences of all and at the same time always produces coherent decisions? The impossibility theorem proved that the answer is no. Any procedure, no matter how clever, runs the risk of producing cycles in voting or other bizarre outcomes. In other words, it may not be the case that there is always a coherent voice of the people.
Why is this so important? Arrow’s results in no way call into question the intrinsic value of democratic processes. Rather, they demonstrate that no procedure can aggregate individual preferences in a way that meets all objectives in all cases. A perfect voting system or collective action scheme, in this sense, does not exist and any institutional design must recognize this.
The Value Of Markets
Democracies are imperfect ways to express preferences and coordinate behavior. This leaves room for alternative mechanisms including one that, like democracy has been around for thousands of years.
In Dinosaur Markets, I relayed the story of how markets were explained to me.
The Dinosaur Question
Democracies are controlled by votes. All votes are equal. But, markets are not democracies. To understand the difference I’ll recount a lesson I was taught as a trader trainee 20 years ago.
It was explained:
If you poll the population, “Did humans walk the earth at the same time as dinosaurs?”, the responses come back split about 50/50. That’s democracy.
Now imagine there is a contract that trades openly on an exchange that is worth $100 if it is true that dinosaurs and humans co-existed and $0 if that is false. Even though the population is split, this contract is not going to trade for $50. It’s going to zero. Why? Because the small percentage of people and scientists who know the truth are going to see a profit from selling this contract even down to $1 since they know this proposition is false. And if the scientists don’t have enough money, they will be able to convince or get hired by people with more money to back this venture of selling this contract to zero.
That is the value of markets. You get correct answers. While a democratic poll may tell you what people believe or desire, it does not assign the proper truth value to the proposition. Now consider the implications of being correct. You make more money which gives you more resources to continue being more correct. The marginal price in markets is set by the market participants with the most money and as a group, they have the best-calibrated assessment of what fair value is. And these groups are in the minority of the total betting population. Markets are not a democracy.
An idea we understand today is that markets are efficient mechanisms for allocating resources. This is the oft-told and valid story of price discovery and signaling. “Wisdom of crowds” stuff.
But efficient doesn’t mean perfect. And there is a spectrum of efficiency and information value embedded in prices.
Limitations on Markets
Just as an exposition on markets and auctions could fill several lifetimes of reading, the limitations on them can as well. Markets can fail or degrade from “wisdom of crowds” to “madness of crowds”.
Critical conditions must exist for them to function well.
A non-exhaustive list would include:
being trustworthy, unrigged, and uncorrupt
they must be transparent — cheap to access and view
a diversity of opinions to allow noisy/bad predictions to cancel out and increase the possibility of bringing in new, pertinent information
resistant to cartel-like coordination
🚧
A Comment on Market Maximalism
I hear some readers groan.
“Here we go again, some trader weaned on the market-maxi milk of libertarianism, pontificating on how society should be organized”.
I am well aware that “free markets” are as real as leprechauns. Rules are made by people. And according to American civics, corporations are people. I’m not naive to charges of crony capitalism.
(In fact, I’ve been blunt that Georgism, which some would charge as socialism, is actually a purer form of market-based capitalism.)
The evolution of economics suggests that rather than abandoning markets for being imperfect, we chip away at the forces that keep them from being as effective while maintaining the broader perspective that any communal decision algorithm is necessarily designed with trade-offs between efficiency and equality.
Market Innovation
As important as Amazon’s “long tail” model or Uber mobilizing the value of otherwise parked cars, financial innovation is a story of novel technology enabling banking and pseudo-banking functions. Like any technology or tool, it’s a form of leverage. I mean this in the pulley sense as much as I mean it in the lending sense of the word. We can do more with less.
Trade itself is a technology that makes the pie bigger by promoting specialization and harnessing comparative advantage.
The very idea of a corporation and equity shares is a technology. It allows savers who are willing to accept a risky, subordinate position in the cap structure in exchange for higher returns to finance entrepreneurs who cannot self-fund the risks.
Risk is being exchanged at a fixed price today for a floating return tomorrow.
The “invisible hand” of markets ultimately sets these prices. Any innovation that allows risk to be priced and transferred at a transparent price between parties is a form of trade, imbued with the same merits of efficiency and choice via increased liquidity.
In fact, this type of trade precedes the possibility for Amazon, Uber, or maybe any large-scale modern technology to exist or at least to have progressed as quickly as they have.
While it sometimes (and justifiably) gets painted as a villain just as any technology can, financial innovation, on the whole, has been a main character in our story of progress. While laws and competition will determine how the private gains are divvied, its byproducts are public goods — liquidity and efficient signals.
We have a collective interest in improving the usefulness of markets as signals for capital allocation. The straightforward methods for doing this include addressing the forces described above in Limitations on Markets.
But there is another approach that should be used as well.
The problem is the approach suffers from a mix of abstraction, bad marketing, and a host of unsympathetic characters — it’s called:
More derivatives!
Before I lose you, let me remind you how deep you are into this post already. If you’re still here, you must trust me. I won’t betray the generosity. Even if you find yourself disagreeing by the end, you will learn some neat stuff along the way. Plus I’d bet that any impulsive resistance is triggered by misunderstanding my use of the word “derivative”.
Before I can explain what I mean by “more derivatives” and why, we need to take a learning detour.
🚧
A detour into the financial theory of “Complete Markets”
Kenneth Arrow appears again as we define 2 nested concepts:
Arrow-Debreu Markets
From Wikipedia:
In economics, a complete market (aka Arrow-Debreu market) is a market with two conditions:
1. Negligible transaction costs and therefore also perfect information
2. There is a price for every asset in every possible state of the world
In such a market, the complete set of possible bets on future states of the world can be constructed with existing assets without friction. Here, goods are state-contingent; that is, a good includes the time and state of the world in which it is consumed. For instance, an umbrella tomorrow if it rains is a distinct good from an umbrella tomorrow if it is clear.
A pure security or simple contingent claim is a state claim that pays off in only one state. Any state-contingent claim can be regarded as a collection of pure securities.
A system of markets is complete if and only if the number of attainable pure securities equals the number of possible states.
🪙
“State of the world”?
The notion of “state of the world” can is most easily understood in context. For example, “the state of the world where the coin has come up tails” is distinct in time and outcome from a preceding or following “state of the world where the coin has come up heads”.
If you flipped tails twice in a row these are still distinct states of the world because they occur at different times.
“Dynamically Complete” Markets
In order for a market to be complete, it must be possible to instantaneously enter into any position regarding any future state of the market. In contrast, a market is called dynamically complete if it is possible to construct a self-financing trading strategy that will have the same cash-flow.
In other words, a complete market allows you to place all of your bets at once, while a dynamically complete market may require that you execute subsequent trades after making your initial investment. The requirement that the strategy be self-financing means that subsequent trades must be cash-flow neutral (you cannot contribute or withdraw any additional funds).
Any complete market is also dynamically complete.
⚛️
A familiar example: The Black-Scholes Model
A “self-financing trading strategy that will have the same cash-flow” is what arbitrageurs simply call “replication”. Using the volatility trading context, an option is the original strategy and “dynamic hedging” is the replication strategy.
Black-Scholes is a model that says that if you initiate a portfolio of:
cash
shares in proportion to the deltas generated from fair volatility of the asset’s movements
you can create the same cash flows as the option implying that fair volatility.
[For options nerds, the amount of cash + shares depends on the moneyness of the option — aka the distance of the strike from the stock price in units of standard deviation which depends on volatility and time.]
If you held offsetting positions in the option and the dynamic hedging replication strategy you would have no risk, therefore the portfolio’s final payoff should be zero when discounted to present value by the risk-free rate.
It follows that the cost of the replication strategy must be the value of the option!
Extracting risk-neutral probabilities
The probabilities of the stock moving up and down depend on the volatility…putting the logic in reverse — the volatility implies a risk-neutral or arbitrage-free probability.
[For options nerds this is the familiar N(d2) term in the Black Scholes equation.]
A note on Black-Scholes and implied distributions
The model is not useful as an absolute way to value options because it is loaded with highly stylized (bullshit) assumptions. It’s better used in reverse as a thermometer to measure/compare implied volatilities.
Instead, we extract risk-neutral probabilities from option structures such as vertical spreads which involve buying and selling options of different strikes effectively “canceling” out the model assumptions. We are left with structures that mimic over/under bets and we already learned that the odds embedded in such bets imply probabilities.
By computing the butterflies across a single expiry it is possible to draw the implied histogram of prices.
Let's move from the abstract to the concrete.
From these definitions, you can see that the market for a stock is not “complete” since the stock can occupy infinite prices at any point in the future. There isn’t a contract for every possible state. You can open and subsequently close a position at a nearly infinite number of points in time.
But… you can also see how options are a step towards completing the market by allowing you to fine-tune discrete bets that expire at discrete times. Of course, there are not infinite strikes and expiry dates, but the ability to create more specific trade expressions allows an investor to structure bets that map to finer-grained theses.
One glance at an option surface will allow an investor to see if the stock is worth $100 because:
a) it’s 50/50 to be $50 or $150
or
b) 90% to be worth 0 and 10% to be worth $1000
The stock price alone tells us the expected value of the business. The options market allows us to interpolate what the distribution of that expected value looks like.
When I used the word “derivative” I was referring to any financial contract that derives its value from a state of the underlying. Insurance is not usually thought of as a “derivative” but of course, it is — its value is contingent on a spelled-out state of the world.
🧩
Derivatives move markets from less to more complete
In an interview with volatility manager Cem Karsan hints at the financial theory of complete markets when he says “options are the true underlying”:
I’ve said before, and people think I’m crazy, but options are the underlying. It is the full distribution of potential outcomes. Equity values, bond values, asset values are ultimately a summary of much more rich, probabilistic information that lies underneath the surface. And I think that acceptance and understanding are slowly happening not just to institutions, but to human beings. It doesn’t happen overnight. But in a world where we’ve created an ETF and ETN for every style, and every factor, the fact that we’re still betting on up and down, and live in two dimensions is nonsensical. So I very much believe that if you look 20 years in the future, the Option Chains are the underlying.
A nice checkpoint for your understanding is if you read that and it sounds less cryptic than it would have if you read it 20 minutes ago.
Here’s another example.
This time it’s Mike Green being interviewed on the HiddenForces pod. He’s talking about financial derivatives like the VIX but steps back for a moment and drops a beautiful theoretical lesson for context:
This goes back to the underlying theory of markets. The work of Kenneth Arrow highlights this underlying dynamic that ultimately markets need to be complete. They need to have options for, and I don't mean options in the form of derivatives, but they need to have products that allow people to express functionally all possible desires and outcomes.
So, the desire to purchase insurance is a very important one. But inevitably, the participants in an insurance market are going to be smaller than the participants in the actual fundamental underlying. The cost to purchase your house is by necessity much, much greater than the cost to insure your home. And that's to do with probability. That has to be a very small fraction of the underlying value of the house. Otherwise:
1. I can't purchase insurance because it would represent such a sizable fraction of my overall outlay and
2. That would actually retard economic development because if I can't buy the insurance on the house, I really can't take the risk of buying a nice house. I can't put that capital at risk except if I'm extraordinarily wealthy.
That would have an interesting impact on the mortgage market and mortgage financing. All of these innovations that have occurred from a financing standpoint, things like diversifying the risk of mortgages or diversifying the risk in the form of insurance, all of those have facilitated the type of economic activity that we very much take for granted.
When I say we need “more derivatives” what I mean is increasing channels to measure the consensus price for risks across the surface of possible states so we can monitor, study, manage, and respond to it coherently.
We “complete” markets when we loosen the limits to arbitrage. This can mean allowing currencies to float, supporting securities lending markets, and allowing sound risk management to expand the definitions of collateral. It could mean fostering liquidity in prediction markets and generally creating more transparency around risks that are present but not accounted for.
I’m not a huge fan of crypto in practice but at its intellectual best it aspires to what I’m talking about. Grift is not part of theory and we need to actually build in the real world, but if humans just gave up on good ideas because of obstacles you’d be fighting a staph infection right now instead of reading this on a phone.
💡
Examples of this thinking from Nick Kokonas
I’ll close this section with a few ideas from one of my favorite entrepreneurs, former option trader, and cofounder of the Alinea restaurant group Nick Kokonas.
Restaurant reservations
The traditional way to make reservations is a green pasture of unpriced risks and opportunities.
The restaurant has limited inventory and bears all the risk of the customer canceling
Supply of tables is constant but customer demand is variable. Why not borrow a playbook from airlines — charge less for reservations made further in the future? Should a Friday night reservation cost the same as a Tuesday lunch?
Software allows us to experiment with pricing so restaurants and customers can price the risks and make better choices about how much they are willing to pay to mitigate them.
The key input: transparency
In an interview with Tim Ferriss, Nick explains the seed of what would become Tock, the restaurant reservation software:
It was also about this thing as I dug into how Alinea ran, there’s thing that you can fix and control, and there’s things you can’t. This was the one that I would answer phones at Alinea and it was like being a therapist. People would go like, “I would like to make a reservation on my anniversary, seven weeks from now on a Thursday.” And you’re like, “I’m really sorry, sir, it’s totally full.” 100 percent of the time they thought you were lying to them. Consequently, I realized that we were saying “No” to people more than we were saying “Yes.” That even when we said “No” in a nice way and took 10 minutes to do so, people thought that we were lying, or that they weren’t important to us, or any of those things.
I felt that transparency was actually more important than the actual money and the actual yield management. I felt like allowing people to see the entirety of the inventory was the most important thing. I had the same thing with the trading markets. They’ve moved to more and more transparency, more and more speed. This is true across every industry. I just felt like “Wow, this industry I found myself in was so backwards.” I just wanted to make the whole thing more transparent. Even if it didn’t go beyond my own restaurants, I was totally fine with that. In fact, I didn’t do anything with it for four years. I didn’t really intend on making it a software product. I intended to fix my own problems.
Prepaying for Beef
Why am I not surprised that Nick found an overpriced option 60-day option (meaning rapid theta btw), and pasted it?
Nick was paying $34/lb. This embedded an option premium for matching his demand to time.
Now he pre-books Alinea customers, crushing the volatility of his revenue.
He now "sells" the option net of carry costs (interest and freezer storage).
Nick wins because he understands option pricing. However, if there was a liquid forward contract, the “economic” surplus could have been shared more efficiently between the supplier and Nick. And Nick would be okay with this!
Sure, he gets the best of it on this deal, but he understands that a healthy, transparent market for meat would keep the relative bargaining power between supplier and consumer in balance. This is a more stable win-win environment to build a business upon.
Nick’s primary business is delivering world-class experiences to diners — not picking off beef suppliers.
You can dive into the interviews to see how Nick uses the same exact thinking when approaching 2 separate cartel-like industries: book publishing and…truffles!
The recurring theme is the same — the risks are mispriced because there is no transparency. If we shine a light on the data by aggregating scattered phone calls into a centralized futures ladder everyone can make better resource allocation decisions. They can see the price to offset or accept the risks.
💡
A thought: Finance As A Necessary Abstraction
Finance, like code, is an abstraction. It's a symbol manipulation field. While the output of code is easy to see in the wild, it's harder to draw a line from progress in the real world back to financial innovation.
Of course, finance has its share of bad actors, but when those grievances spill over to honest practitioners it can make them feel alienated from their own work.
I'm not going to go all Blankfein on you and invoke “God's Work” but when you see someone toiling in operations (my mother worked in a bank back office most of her career) there's no justification for suffering the indignity of “you contribute nothing to society”.
While traders and bankers seem overly adept at justifying their chosen profession, I still meet many introspective types in those roles who feel conflicted about their jobs, especially since many of them had the ability to answer “more respectable” callings.
I’ve covered this before in Finance Guilt but don’t worry you aren’t evil or…special. You’re just a plain ol’ worker bee in the hive of human advancement.
[For the paradox-sniffers: it's true your individual job doesn't matter but collectively all the people doing your job matter. It's similar to the paradox of one vote having a 0% probability of mattering but voting still matters or the famous paradox of thrift that acknowledges that while it’s wise for individuals to save, if we all try to increase our savings simultaneously it would tank the economy]
