Welcome to another AllCoreDevs update - the last one of 2022 🫡
While these started out as a monthly series, the cadence is slowly trending towards quarterly. Think of these updates as a digest of everything big that’s going on around AllCoreDevs. If you want a higher resolution, I recommend the more frequent ones by Christine Kim, the CL call notes by Ben Edgington or my ACD tweetstorms.
With that said, let’s get into it!
TL;DR 👀
-
The contents of the Shanghai/Capella upgrade has been finalized: withdrawals, EOF, and a handful of minor changes… so long as they don’t delay withdrawals ❌
-
Blobspace is coming: EIP-4844 will be at the centre of Ethereum’s next upgrade, with its summoning ceremony starting soon 🕯️
-
An effort is underway to harmonize the technical side of the EL & CL upgrade processes. We’re also seeing active discussion about better incorporating the community’s input in the process 📣
-
Protocol Guild published a mid-pilot report, and a rough plan to scale and better support L1 maintainers in 2023 ⛓️🛡️
Shanghai / Capella 🌃
On the last AllCoreDevs, client teams agreed to a finalized scope for the Shanghai / Capella upgrade. While the name of the upgrade might still be up for debate, teams are clear about its scope. The main feature of the upgrade is the introduction of Beacon Chain withdrawals for stakers. Getting this out ASAP is something client teams don’t want to compromise on, so other features in the upgrade need to be ready at the same time, or may be dropped.
The Shanghai EL spec lists all included EIPs:
While the list is long, it can be broken into three distinct buckets: minor improvements, EVM Object Format and Withdrawals. Let’s go through each of these:
Minor Improvements ⚙️
EIP-3651: Warm COINBASE
This EIP fixes an oversight in EIP-2929, which changed the gas costs of accessing certain data fields based on whether clients already have them in memory (WARM) or need to retrieve them from disk (COLD).
EIP-2929 sets two pieces of data which clients have in memory to WARM at the start of each transaction: the sending and receiving addresses. EIP-3651 adds a third address to this list, the COINBASE address (a.k.a. feeRecipient), given it’s also an address clients have in memory when processing a block’s transactions.
EIP-3855: PUSH0 instruction
As the name implies, EIP-3855 introduces an opcode to push the value 0 to the stack. Pushing 0 is frequently used to pad values in the EVM, and this opcode will offer a more efficient, and cheaper, way to do this.
EIP-3860: Limit and meter initcode
This EIP adds a maximum size for initcode and introduces gas metering based on its length. The size limit adds an invariant to the EVM which will make it easier to reason about and propose changes to.
A 2 gas/32 byte cost for initcode is introduced to account for the jumpdest-analysis clients must do prior to execution, which previously was not accounted for in the gas schedule.
EVM Object Format 🧰
The majority of EIPs included in Shanghai are all part of a single feature: EVM Object Format (EOF). The work was broken down into 5 distinct EIPs to help client developers reason about each change in isolation, but to give a higher level overview, a unified spec was released this week. The 5 EOF EIPs are:
It’s worth noting that the first step for EOF was taken in the London upgrade, with EIP-3541, which reserved the 0xEF00 prefix for EOF contracts. Over the past few months, the scope of EOF for Shanghai also evolved.
In February, client teams agreed to consider two minimal EOF EIPs for Shanghai: EIPs 3540 & 3670. These would act as building blocks but wouldn’t provide the full functionality without the introduction of EIPs 4200, 4750 and 5450. While it is possible to extend EOF, backwards incompatible extensions may require a version bump. Because pre-EOF or EOF contracts with a specific version must always be executable, each new EOF version means client developers must maintain a new set of EVM execution rules in parallel to legacy ones.
Before EOF, clients only maintained one set of EVM rules at a time. The codebases also support historical EVM rules which can change every network upgrade, but once they get to the tip of the chain, only the most recent rules must be applied. With EOF, clients will maintain two parallel set of EVM rules, so they can execute code in both EOF and non-EOF contracts. In other words, EOF version bumps increase the number of parallel, rather than sequential, EVM rulesets that must be maintained.
For this reason, over the past few months, clients teams began preferring a “big EOF” approach. This way, while they must implement a larger change set, the EOF version will stay fixed longer, reducing the number of “parallel EVMs” to maintain. Hence, “Big EOF” is what was considered, and eventually included, in the Shanghai upgrade.
That said, bigger features are obviously trickier to implement and test and teams don’t want to see EOF significantly delaying Beacon Chain withdrawals. Therefore, clients agreed to remove EOF from Shanghai if, come January, implementations are not complete and quickly interoperating with each other.
With that context, let’s now briefly go over the various EOF EIPs:
EIP-3540: EVM Object Format (EOF) v1
This EIP introduces the “container” for EOF contracts. It adds markers which discriminate code and data sections in contracts, and prevents EOF contracts who don’t comply with the format to be deployed. This provides the assurance that any on-chain EOF contract would follow a valid layout, which simplifies both interactions with these contracts as well as static analysis of them.
EIP-3670: EOF - Code Validation
Building on the container introduced by 3540, EIP-3670 ensures that the code in EOF an contract is valid, or prevents it from being deployed.
This means undefined opcodes cannot be deployed in EOF contracts, which has the added benefit of reducing the number of EOF version bumps required. If a new opcode is added, validation rules can simply be changed to enable it, and there is a guarantee that no EOF deployed contract references it in its code section.
EIP-4200: EOF - Static relative jumps
EIP-4200 introduces the first EOF-exclusive opcodes: RJUMP, RJUMPI and RJUMPV, which encode destinations as signed immediate values. These new JUMP opcodes can be used by compilers to optimize gas costs, as they remove the need for runtime jumpdest-analysis that is required with the existing JUMP & JUMPI opcodes.
EIP-4750: EOF - Functions
EIP-4750 takes features from 4200 one step further: it disallows the JUMP & JUMPI opcodes, and adds an alternative for functionality that can’t be replicated with RJUMP, RJUMPI and RJUMPV. It does this by introducing specific function sections in EOF bytecode, which can be jumped to, called and returned from with new JUMPF, CALLF and RETF opcodes respectively.
EIP-5450: EOF - Stack Validation
Finally, EIP-5450 adds yet another validation check to EOF contracts, this time around the stack. The EIP prevents EOF contracts from deploying code which can result in a stack underflows, as well as some cases of overflows. With this, clients can reduce the number of validation checks they make during execution of EOF contracts because they have better assurances around stack-related exceptions with them.
As a non EVM expert, leaning heavily on the EIPs themselves, that’s as far as I can take you! If you want to dig into EOF more, I recommend Geth’s lightclients or Solidity’s Leo’s threads.
Beacon Chain Withdrawals 💸
Last but not least, the main component of “Shapella” is Beacon Chain withdrawals. The change is specified across both the consensus layer specs and EIP-4895. A now-slightly-out-of-date meta-spec ties these together.
At a high level, here is how withdrawals will work:
-
When proposing a block, validators scan linearly through validator indices to find the first 16 validators with
0x01credentials who either:-
Have a balance above 32 ETH (i.e. have accrued validator rewards)
-
Are
withdrawable(i.e. have fully exited the validator set)
-
-
From these, the validator will create a list of withdrawals to be included in their
ExecutionPayload. Each item in that list contains the following:-
WithdrawalIndex: index among all withdrawals ever made — this helps differentiate between withdrawals for the same amount from the same validator to the same address -
ValidatorIndex: the index of the validator whose balance is being withdrawn -
ExecutionAddress: the ETH address on the execution layer where the withdrawal should be sent -
Amount: the amount, in gwei (not wei), to be sent toExecutionAddress
-
-
When building or processing a block, EL clients will apply these withdrawals after transaction execution. In other words, processing withdrawals is similar to how proof-of-work rewards were credited, and does not compete with user transactions for block space.
A few more details worth noting:
-
When processing withdrawals, there is no difference between a “full” vs. “partial” withdrawal in terms of priority/ordering. Full withdrawals happen once a validator has left the exit queue, and partial withdrawals happen periodically when the linear scan over the validator set reaches a validator’s index.
-
For withdrawals to be processsed, validators must use
0x01credentials, which represent an ETH address. At the launch of the beacon chain, only0x00credentials, which used BLS keypairs, were allowed. In order to enable withdrawals, validators with0x00credentials will need to sign aBLSToExecutionChangemessage. These will be enabled as part of the Capella upgrade. Validators can expect support and tutorials for multiple tools to sign this message. -
The scan over the validator set is bounded each block. If, after scanning a subset of the validator set, there aren’t 16 withdrawals to process, the validator will stop scanning and the next one will pick up from the last validator index scanned.
As always, there will be several devnets and testnets (potentially even some new ones!) for validators to run through the process and iron out any issues before things go live on mainnet.
Shanghai/Capella isn’t the only upgrade on which progress was made, though! Teams have also been looking ahead to the next one.
Cancun 🏝️
With Shanghai full, but many CFI’d EIPs not making it in, client teams began discussing what should be considered for the next upgrade: Cancun (CL name TBD) 🏝️
On the CL side, EIP-4844 had already moved towards being spec’ed as the first post-Capella upgrade. The EL doesn’t (yet!) have a spec which enables such a layout, but EL teams agreed to follow a similar path and center the next upgrade around EIP-4844.
Following the convention of using devcon city names for upgrades, cancun.md was created, with EIP-4844 being formally included in the upgrade.
This decision happened in the final minutes of the last 2022 AllCoreDevs call, so there wasn’t time to address other proposals. EIPs that were CFI’d for Shanghai but didn’t make it in were moved over to Cancun’s CFI list, and an Ethereum Magicians thread was opened to discuss Cancun candidate EIPs. Early next year, deliberations around Cancun’s scope should start in earnest.
KZG Ceremony 🕯️
One other Cancun-related thing to expect then is the KZG Ceremony 🕯️, which is a requirement for EIP-4844.
The Ceremony will generate the randomness needed to verify the validity of blob data. For it to be considered secure, only a single participant must have participated honestly. In other words, if all but one participant collude, then the entire process is cryptographically secure.
Starting in January, the ceremony will be open to everyone for several months. With a goal of getting 10,000 contributors, this is planned to be the largest such ceremony to date! If you want to be sure not to miss it, give Trent Van Epps a follow!
Post-Merge Upgrade Process 📜
As mentioned in previous updates, one big TODO with The Merge behind us is harmonizing the upgrade process for Ethereum across the EL & CL. At a high level the EL uses the Yellow Paper & EIPs to specify changes, while the CL uses an executable Python specification.
The EL process has the upside of EIPs being well known by the community and formatted in a way which clearly presents the reasoning behind a proposal. The mix of the math-heavy Yellow Paper overlaid with EIPs, and need to put specifications in context within EIPs make the execution layer specifications both hard to understand and extend.
The CL side has the inverse problem: it has a clear and approachable spec, living within a single repository, but changes aren’t specifically identifiable, and proposals get burried among the other open PRs on the repo.
With the introduction of the Ethereum Execution Layer Specification, we can hopefully bridge the gap from the EL side. And, with some process wrangling, we might be able to get EIPs introduced as part of the CL process…!
That said, as the Shanghai upgrade scope was being debated and finalized, it became clear that another “piece” may be missing from this process: somewhere for the community to voice their relative preferences about changes, engage in discussions about the overall scope of an upgrade (rather than individual EIPs), and have that be incorporated as part of decision making in the AllCoreDevs & CL calls.
It’s not quite clear what this looks like yet - and I’d love suggestions! - but with the number of stakeholders being actively involved in protocol changes and the number of domains L1 changes affect both growing, it’s apparent that something is needed.
Luckily, we’re not starting from scratch: Ethereum Magicians has been around for years now, and its IRL gatherings, as well as ad-hoc breakout rooms or community calls may be good starting points to extend.
Expect more updates on this front early 2023!
Protocol Guild Update ⛓️🛡️
With the Protocol Guild pilot halfway done, the collective published a report examining how things are going and what the planned next steps for the project are.
As a reminder, PG is a permisionless funding mechanism for Ethereum L1 client developers, protocol researchers, and supporting contributors, such as yours truly.
The mechanism is centered around individuals, not organizations. In short, every member is eligible for a share of tokens vesting in the guild, weighted by the duration of their contributions to Ethereum. Member additions and removals are done, in true Ethereum fashion, by reaching rough consensus within PG, based on a set of criteria. This list is then put on chain using 0xSplit’s split contract. Donors can then send funds either directly to the split, or to a vesting contract that feed to it.
The mid-pilot report was summarized on Twitter. Here are some highlights ✨
-
The pilot raised 9.7m$, from many orgs such as Lido, Uniswap, ENS, NounsDAO and MolochDAO, as well as some recurring individual donors (s/o Tetranode 🦈!) — thank you all for making this possible❤️🔥!
-
PG now has 128 members, up from 90 at launch, and 5m$ has been distributed among them already 💸
-
On average, each member received 39,000$, with a min of 13k$ and max of 79k$ 🧧
-
PG’s architecture is evolving to support L2s and remove the need for a multisig to update weights 🙅
These early results show PG working as intended: a mechanism which distributes a basket of tokens to a self-curating, growing, set of protocol contributors. Proving this wouldn’t have been possible without the generous support from pilot donors.
Looking forward, it’s now time to scale PG to its full potential: providing competitive risk-adjusted compensation for maintainers of Ethereum. The lowest hanging fruit here is for projects to contribute to PG from Day 1, as described in Danny Ryan’s PG inception tweet.
Donations for the pilot mostly came from large projects with significant treasuries. If Protocol Guild can convince projects to contribute from Day 1, when their token is still effectively worth “nothing”, then Ethereum maintainers can benefit from the entire upside trajectory of successful projects.
With enough projects participating, incentives can shift towards keeping the best people towards maintaining the protocol, rather than pulling them away from it.
To support this, as well as many other contribution types, PG will need a technical overhaul. The next version should work across both L1 and L2s, and reduce its on-chain governance footprint even more.
If you’re a project looking to contribute to Protocol Guild, please reach out — my DMs are open 📭!
Next Steps ✅
And that’s a wrap for 2022… what a year! Three months ago, we hadn’t even merged! Now, Ethereum is silently running proof-of-stake in the background and focus has shifted to what’s on the horizon.
As everyone gets back from the holidays in January, you can expect:
-
Shanghai/Capella devnets and shadow forks 👻
-
The KZG Ceremony going live 🕯️
-
Discussions around Cancun, and how the network upgrade process should evolve to better capture the preferences of the community 🏝️
-
Protocol Guild’s pilot to wrap up and post-pilot architecture to be announced ⛓️🛡️
Thanks for reading! And thank you to everyone who spent time trying to improve Ethereum over the past year - we got a lot done. Hope you all take some well-deserved holidays 🎄.
See you in 2023 👋!
Thank you to lightclients, Alex Stokes and Joe Schweitzer for reviewing drafts of this update and making sure I get the technical details right!