Welcome to another AllCoreDevs update... better late than never 🙈!
So much has happened since January that I've struggled to find the time to write it down! Here are the highlights:
Following Kintsugi, the Kiln testnet was recently launched. It incorporated changes to The Merge specifications based on edge cases discovered on Kintsugi, as well as some renamings. While the specs for The Merge now seem mostly final, running through the transition on Kiln surfaced implementation issues in several clients. Teams are now doubling down on testing to ensure that all implementations are safe and stable. Danny covered this in his latest Finalized post.
Additionally, we're now asking the broader developer community to use Kiln and make sure their products work as expected. Shoutout to Kurtosis, Tenderly, Lido, Uniswap, EthStaker, Infura and Blockdaemon who've each taken a stab at it!
Assuming no critical issues are found, Kiln is expected to be the last new public testnet to be launched. The next step, once we are comfortable with client implementations and infrastructure/tooling readiness, will be to run The Merge on existing testnets, such as Ropsten, Goerli, Sepolia, etc.
As with every upgrade, we'll monitor the testnets after the upgrade to ensure they are stable. Once we are confident testnets are running as expected, we will schedule the transition for the Ethereum mainnet!
While we are close, and this is an incredibly exciting moment for the entire community, the safety of transition, more than any target date, is the #1 priority for The Merge. This is, by far, the most complex upgrade to Ethereum ever done. We want to get it right 😁.
When decided, testnet and mainnet upgrade timelines will be communicated across community publications such as Week in Ethereum, What's New in Eth2, the EF blog and many others. Anything which currently advertises a target date is wrong, as no date has been set. Be extra vigilant for potential scams/fake announcements over the coming months!
The difficulty bomb, which was pushed back in the Arrow Glacier upgrade last year, is expected to be felt on the network around June. Its progression is being tracked here. While it would be ideal to transition to PoS before we need to push back the bomb, three things are worth noting:
As mentioned in the previous update, with the specs for The Merge mostly frozen, planning efforts have started around Shanghai. A specification for the upgrade is available here. Three major changes are tentatively planned for the upgrade, along with some small wins. Let's dive into them!
For years, researchers and client developers have struggled with improving the EVM without breaking existing contracts. Last year, the Ipsilon team came up with a clever solution: provide new functionality to contracts which are deployed with a specific identifier, but leave existing contracts executing as is. This is now know as EVM Object Format, or EOF for short.
In the London upgrade, we reserved part of this identifier by disabling the deployment of new contracts starting with the 0xEF byte. A few contracts starting with this byte were deployed before London went live, but now that it no longer is possible to do so, we can add a second byte (called the Magic Byte 🪄) to the 0xEF prefix and get a sequence which we can guarantee isn't used by any contract.
EIP-3540 describes this in detail and highlights the first tangible benefit of the approach: the separation of code and data, which is beneficial for on-chain code validation. This could also pave the way toward introducing new contract code sections types that could help enable currently complicated features, such as Account Abstraction, control flow in the EVM and EIP-3074.
A companion EIP to 3540, EIP-3670 will enable code validation for EOF contracts upon deployment.
Another major feature for Shanghai is the activation of Beacon Chain withdrawals. After several proposals, we've arrived at a design that client teams are satisfied with: EIP-4895: Beacon chain push withdrawals as operations.
A meta-spec outlines how the entire process works. At a high level, in each slot, the Beacon Chain can process a set number of full or partial withdrawals. These withdrawals are tracked in receipts which contain the amount, destination address and unique index of each withdrawal. As part of the block creation and validation process, these withdrawals are then credited on the execution layer, similarly to how proof-of-work issuance is credited to miners today.
A tracking issue for the various changes required at the consensus layer to enable this is available in the consensus-specs repository. The option for partial withdrawals will allow validators to withdraw their accumulated rewards while maintaining the 32 ETH stake required to remain a validator and continue earning rewards.
The last large thing we hope to include in Shanghai is a reduction of fees on layer 2. Because L2s post their transaction data (and/or proofs) on L1, a large part of end users' transaction fees come from the amortized gas costs of L1 data storage. Sharding offers a cheaper alternative for L2s to post their data, but while the specification seems to have settled, a full sharding implementation isn't ready yet.
In the meantime, two options are available to reduce these costs: either a CALLDATA cost reduction on mainnet, or a "proto-sharding" implementation, possible with the introduction of a new transaction type on Ethereum, called Shard Blob Transactions.
CALLDATA Cost ReductionThe simplest way to reduce transaction fees on L2s is to lower the cost of storing data on L1. EIP-4488 proposes to do this, reducing the cost from 16 gas per byte of CALLDATA to 3 gas/byte. This reduced storage cost would translate to lower L2 fees [1].
While the gas cost reduction itself is a simple change, it does have some second order effects. First, increasing the CALLDATA in blocks leads to larger block sizes. To balance this, the EIP proposes a cap to the maximum amount of CALLDATA in a block. Second, even with this cap, this EIP would increase the rate at which historical chain data grows on the execution layer. To address this, out of band historical data retrieval needs to be developed, and guarantees about historical data on the Ethereum P2P network need to be changed, as proposed in EIP-4444.
While the increase in historical chain data would happen gradually, including this EIP implies we will need to deal with the issue with increased urgency after it goes live. Additionally, little from this EIP would be reused in a full sharding implementation. It would primarily be a temporary solution. That said, the EIP is a relatively simple change to implement and does provide significant fee reductions on L2s.
Another proposal, which gets us closer to a full sharding deployment, is EIP-4844 [2], which introduces Shard Blob Transactions. As with Beacon Chain withdrawals, this proposal has a meta spec linking to consensus layer specifications and other resources.
At a high-level, this new type of transaction would include commitments to blobs of data which are gossiped on the Beacon Chain. This proposal can be thought of as a "mini-sharding", where instead of relying on data availability sampling, every node on the network needs to validate all the data in blobs. Like in the full sharding context, these blobs of data are only guaranteed to be made available for some time on the network, rather than stored forever. To keep node requirements manageable, blob data is limited to 1MB/slot compared to 16mb/slot in full sharding.
EIP-4844 would lay the groundwork needed for a full sharding implementation. Notably, all future changes would be limited to the consensus layer. From the execution layer's perspective, sharding would be up and running!
The Optimism team, which has been working on the EIP, has launched a website which provides an overview of the EIP, aggregates various spec links and highlights positive community reactions to the effort.
[1] Due to other components involved in pricing L2 transactions, the reduction wouldn't be a full 5x. Optimism has a good explanation of L2 fee components here. Also, ZK rollups wouldn't benefit to the same extent as Optimistic rollups.
[2] EIP-4488 (CALLDATA cost reduction) and EIP-4844 (Shard Blob Transactions) have frustratingly similar EIP numbers for competing proposals 😅
In addition to these three large changes, a handful of small improvements are also being considered for Shanghai, namely:
COINBASE address, fixing an oversight in EIP-2929initcode and introduces gas metering for the fieldPUSH0, that, as you'd expect, pushes 0 onto the EVM stackIn addition, there are several other EIPs proposed for the upgrade (see a rough list here). EOF, Withdrawals and Layer 2 fee reductions would already make Shanghai one of the largest upgrades to date, so we now need to be very diligent in prioritizing what we include.
Once we start implementing and testing the various EIPs, we will have a better feel for whether we have any extra capacity. Of course, we still need to get The Merge done before that!
As you may have noticed above, several proposals for Shanghai now span both the execution and consensus layers. Historically, we've used different processes to introduce changes on each layer.
On the EL, Core EIPs contain changes' specifications. The Ethereum Yellow Paper is the reference specification for the network, but is often updated only after an upgrade is live, sometimes with a significant delay. This means the effective specification for the EL is often "the Yellow Paper + EIPs X, Y, Z".
On the CL, an executable specification is used as a reference, and changes are specified directly in it. The specification can then be used to generate tests for changes.
So, while the EL process is well understood by the community (and provides an easy to reference description of changes), from a technical perspective, it isn't ideal. Conversely, while the CL process is cleaner technically, it is harder for the broader community to follow. Luckily, work has begun on EELS: an executable spec for Ethereum's EL!
Having an executable specification across both the EL and CL should allow us to harmonize the change process on both layers. There are still many kinks to work out, but conversations have started about how to best do the migration. An Ethereum Magicians thread is now dedicated to the topic. While EELS is still under development, we may be able to use it during the Shanghai upgrade in parallel to the current process.
Hopefully, merging the EL & CL processes is simpler than merging the actual CL and EL 😅.
Last but not least, I want to touch on Protocol Guild, which now has a full explainer website. Compensation for protocol maintainers has been a hot topic lately and PG hopes to be part of the solution. Full disclosure: I am a PG member and would receive funds from it.
You can think of compensation in three buckets: base pay, aligned incentives, and potential upside. Currently, base pay for client developers and researchers is handled through their respective employer. While some of these may provide incentives in the form of equity, the Ethereum Foundation announced its 39,000 ETH Client Incentive Program last year to ensure all client teams have a significant stake in Ethereum.
PG differs from both base compensation and the incentive program in that it aims to provide its members with exposure to a variety of tokens from projects building on ETH, rather than ETH itself. The Guild is composed of protocol engineers, researchers and a handful of folks coordinating protocol work, like myself. It currently sits at about 100 members.
In short, the Guild allows sponsors to donate tokens, which then vest over time to recipients. The recipient set can be updated, which allows for new contributors to be added and stale contributors to be removed periodically.
The Guild is still an early experiment, but if successful, could be a base-layer-focused complement to initiatives such as Gitcoin and Retroactive Public Goods Funding.
After a successful Gitcoin grant, the next step for PG is testing the smart contract architecture. In parallel to this, outreach for initial donors will begin. The plan is to run PG for ~1 year with a limited amount of donations to ensure that both the technical and governance components work smoothly. Hopefully, this pilot proves that we can create new mechanisms to coordinate public goods funding on Ethereum!
Our main priority remains The Merge, with a renewed focus on testing. Over the next month, we are hoping to finalize implementations, run multiple short-lived devnets, and gather feedback from application, infrastructure and tooling providers. Everything else (Shanghai, execution specs, Protocol Guild) should also keep moving along in the background.
Expect another update in a month or two. In the meantime, we'll also have the chance to discuss all of this face to face at Devconnect — see you in Amsterdam 👋🏻🇳🇱!