The Old Fee Market is Broken, Long Live the New Fee Market

What the wrong fee mechanism does to your coin.

Almost all cryptocurrencies today require their users to attach fees to their transactions [1]. The miners then collate transactions paying the highest fees into the blockchain, and derive an income stream. This mechanism is superficially appealing, and has led some to push hard for a blockchain vision, dubbed the "fee market," driven almost entirely by such fees.

Much of the BTC/BCH split stemmed from a difference of vision around this central point, where BTC developers wanted to steer Bitcoin away from a reliance on block rewards and towards higher transaction fees. In contrast, BCH developers reacted strongly to high fees and wanted to keep the economics of miner compensation centered primarily around block rewards. Both sides make good points: block rewards are not sustainable, because the number of coins outstanding is fixed and therefore the rewards must diminish over time. At the same time, high fees lead to terrible user experiences, where some users paid as much as $55 to send transactions, while others complained bitterly of stuck transactions.

In this post, we describe why the predominant fee paradigm used in cryptocurrencies is broken. We describe the reasons why the so-called "fee market" will not lead to a stable, predictable user experience.

In addition, we provide an alternative way to charge fees that yields a sensible fee market that has more stable, and therefore more predictable, pricing. This novel mechanism also provides lower variance, and therefore higher predictability of returns, for miners.

An analysis of how our mechanism would behave in Bitcoin shows that it could have saved users over $272 million dollars over December 2017, and could have reduced the variance of miners' fee revenues by a factor of 7.4.

Problem with the Current Fee Mechanism

Under the current fee paradigm, a user wishing to submit a transaction must figure out an appropriate fee. This turns out to be a very difficult task.

This seems straightforward, but there are problems.

Fees and Cognitive Load

The first problem is the cognitive load on the user: it is difficult to decide exactly how much to bid, whether to go over or under. Surely, if the value of the transaction to the user is X, then her bids will be capped at the utility of the transaction to that user (say, X mBTC). But between 0 and X, there are many options, and picking the right one depends on a lot of other factors. Exactly how important is this transaction to me right now? How full is the mempool? What are the competing bids? Given that there will be, in expectation, another 10 minutes of transactions streaming in to the miners before the next block is discovered, how low a fee can she get away with while still getting her transaction included in the blockchain? These are clearly difficult questions. Doing the right thing involves watching the chain closely and monitoring the transaction until it is included, an act that detracts from the hassle-free use of one's money. One may be tempted to just go through a middleman, such as an exchange, to handle it all, which creates centralization and simply recreates the current banking system except with unregulated exchanges as centralized custodians. This has, historically, led to a stream of exit scams and other SFYL-events [2].

Broken Attempts

But the real problem is much more fundamental, and it stems from the fact that the fee mechanism in Bitcoin and other currencies is implemented as a pay-what-you-bid, or multi-unit first-price, auction. This fee behavior will lead to "sticky" and unnecessarily high fees, followed by sudden fee collapses, just as we have seen over the course of the last few years.

To see why, imagine a universe where everyone is using a simple historical fee estimator. Specifically, imagine a fee estimator that looks back on historical transactions and suggests a fee based on what happened on average in the past. If transactions were paying an average of 100 satoshis per byte in the recent past, then a user will simply attach a fee of 100 or more satoshis per byte.

This approach is completely broken. During times of congestion, the fees to get included will naturally go up, as they should. If the transactions momentarily arrive faster than blocks are found, the fees attached will go up. But they will remain high even after the congestion has ended. If, for instance, the rate of arrival for transactions is exactly equal to the rate at which blocks clear them, the system should be able to support 0-fees. And yet this approach will force the users to pay fees as if they were operating at the height of congestion. The fee structure that arises during congestion is ensconced in the system even though the conditions have changed, an artifact of bad mechanism design.

One can imagine other fee estimation heuristics, such as underbidding on purpose to explore if one can get away with paying less, that would yield better results. But it all is up to the vagaries of other people's choices of fee estimators that would determine the prices. A smart user who notices that the fee estimators are broken would have little recourse except to pay the prevailing fees. The system would provide no mechanism by which optimal choices are made. And of course, the mavericks who do try to explore paying lower fees, just to see if the entire ensemble of users could move to a lower price point, will have to deal with stuck transactions and transaction delays.

Blockchains are not the only place where similar auction mechanisms have led to poor user experiences. When ad placement in search engines was based on first price auctions, researchers observed similar patterns. Advertisers would compete with each other in order to get a better placement, driving the fees sky high. This would be followed by many participants quitting the game, which would cause a precipitous drop, only to restart the unstable cycle all over again.

A Better Fee Market

In a new paper, we propose a new mechanism to charge for transactions. This mechanism is only a slight code change away from the old mechanism, but it has the potential to yield a much more stable fee market, a much better user experience as well as a big savings in fees, and a more predictable revenue stream for miners.

Our proposed mechanism is fairly simple: transactions specify a fee, just like before, and miners place transactions in a block, just like before. Except, instead of charging each transaction the fee it bid, we charge each transaction the lowest fee charged to any transaction in that block. Any surplus fees a transaction offered are returned to that user, to a designated address they specify. Hence, a transaction in effect says "I am willing to pay up to $30 for this transaction," but is charged only $5 if the lowest fee transaction in that block paid $5. The remaining $25 are returned to the user.

Our proposed mechanism brings insights from multi-unit second price auctions to the world of cryptocurrenices where currently multi-unit first price auctions are the norm. Whereas before, fee selection was a stressful and difficult task, with our mechanism, users can simply attach to their transaction the true maximum value they would be happy to pay. This is because they are not going to be charged that value: instead, they are charged whatever the minimum was to get into that block. In essence, the lowest paying transaction establishes just how little it took to get into that block, and everyone within that block is charged the same amount per byte. This is not only equitable and fair, but it takes away pressure to play games with fee selection. As an additional bonus, the benefit from playing such games decreases as the blockchain gets more popular, further disincentivizing strategic fee selection. Note also that it picks the highest-paying transactions, just like first-price auctions, though it charges them strictly less than what they would be willing to bear.

Our proposal couples this idea with three other mechanisms to provide a comprehensive solution that prevents malicious behavior by miners. First, if a miner fails to fill a block, they cannot charge any fees. So a miner cannot take a high-paying transaction, ignore the rest of the mempool, and collect the entirety of the fee paid by that transaction while refusing to fill a block. They are, of course, free to fill the rest of the block with their own synthetic transactions, but they will have to pay out of their own pockets for those (and the next point addresses why the miner cannot just pay those additional fees directly to himself). Second, a miner is rewarded the average fee collected from the last B blocks, not just the fees from block they themselves mined. This ensures that miners also have little incentive to act strategically. Instead their best interests are aligned with maximizing the number of high-value transactions cleared per second. Finally, we propose that every block reserves some space, around 20%, which are exempt from this mechanism. This enables a miner to include transactions of high importance to themselves, such as those used for pool rewards, without affecting the fee mechanism and without being penalized. This addresses the miners' needs and provides a simple migration path from the current state of affairs.

Our mechanism brings order to the chaotic fee market of today. During the dramatic Bitcoin price increase in December 2017, we estimate that users would have saved over $272 million in transaction fees and miners would have a much more predictable fee revenue, reducing their daily fee variance an average of 7.4 times. These gains are not surprising -- our mechanism enables transaction fees to correspond to the true demand that users have for block space. This will make the fee markets more predictable. As a result, users can just bid their true value and know that they are not going to be overpaying for block space. Thus, this removes a painful strategic element to simply using cryptocurrencies today.

If you are designing new cryptocurrencies or are an active user of existing currencies, we highly encourage you to push for this fee mechanism instead of unstable first-price auctions.


[1]Cryptocurrencies that provide fee-free transactions for all are horribly and trivially broken, as they are open to simple, flooding-based denial-of-service attacks.
[2]SFYL: Sorry For Your Loss.
Share on Linkedin
Share on Reddit
comments powered by Disqus