This PR tracks the overall progress of aligning current beacon chain implementation to spec version 0.6. Please comment on the issue before you pick up an item to work on.

• [x] 💎 Update constant names and values 💎

• [x] 💎 Update Protobuf data structures 💎

  • [x] Update Crosslink | #2313
  • [x] Update Eth1Data | #2315
  • [x] Update AttestationData | #2316
  • [x] Update AttestationDataAndCustodyBit | #2316
  • [x] Update IndexedAttestation | #2316
  • [x] Update DepositData | #2318
  • [x] Update BeaconBlockHeader | #2327
  • [x] Update Validator | #2318
  • [x] Update HistoricalBatch | #2327
  • [x] Update ProposerSlashing | #2328
  • [x] Update AttesterSlashing | #2328
  • [x] Update Attestation | #2328
  • [x] Update Deposit | #2327
  • [x] Update VoluntaryExit | #2328
  • [x] Update Transfer | #2328
  • [x] Update BeaconBlockBody | #2340
  • [x] Update BeaconBlock | #2340
  • [x] Update BeaconState | #2341
  • [x] Update Fork | #2545

• [x] 💎 Update general helper functions 💎

  • [x] Update slot related helpers | #2345
  • [x] Update validator related helpers | #2352
  • [x] Update balance related helpers | #2348
  • [x] Update permuted index helpers | #2350
  • [x] Update splitoffset helpers | #2365
  • [x] Update committee related helpers | #2354
  • [x] Update shard delta helpers | #2497
  • [x] Update epoch start shard helpers | #2497
  • [x] Update block root related helpers | #2355
  • [x] Update state root related helpers | #2355
  • [x] Update randao mix helpers | #2357
  • [x] Update active index root helpers | #2369
  • [x] Update generate seed helpers | #2371
  • [x] Update beacon proposer index helpers | #2381
  • [x] Update attesting indices helpers | #2449
  • [x] Update balance related helpers | #2356
  • [x] Update bitfield related helpers | #2446 (bug fix)
  • [x] Update verify indexed attestation helpers | #2385
  • [x] Update churn limit helpers | #2364
  • [x] Update convert to indexed | #2519
  • [x] update get domain | #2545

• [x] 💎 Update routines for changing validator status** 💎

  • [x] Update initiate validator exit | #2366
  • [x] Update slash validator (relies on balance and BeaconProposerIndex changes) | #2391

• [x] 💎 Verify and update chain start logic 💎

  • [x] Update genesis functions | #2465
  • [x] Update Deposit Contract to new spec.

• [x] 💎 Verify and update SSZ 💎

  • [x] 🔹 Implement self-signed containers🔹

• [x] 💎 Update beacon chain state transition function 💎

  • [x] 🔹 Update per epoch processing🔹
    • [x] Update general helper functions | #2379
    • [x] Update justification and finalization functions | #2448
    • [x] Update crosslink function | #2460
    • [x] Update rewards and penalties functions | #2517
    • [x] Update registry updates functions | #2521
    • [x] Update slashings function
    • [x] Update final updates functions
  • [x] 🔹 Update per slot processing🔹
  • [x] 🔹 Update per block processing🔹
    • [x] Implement process block header | #2534
    • [x] Update process randao | #2614
    • [x] Update process eth1 data
    • [x] Update proposer slashing | #2603
    • [x] Update attester slashing
    • [x] Update attestation processing
    • [x] Update deposits processing
    • [x] Update exits processing
    • [x] Implement transfers processing
    • [x] Update state root verification

• [x] 💎 **Verify and update BLS

  • [x] 🔹 add bls verify

• [x] align RPC endpoint eth1data to spec | #2733

Write/improve test for TestBroadcast such that a broadcast is tested to be received by a p2p peer. This involves looking into the libp2p library to determine how to appropriately create hosts in memory for a full test of our broadcast p2p logic.

See: https://github.com/prysmaticlabs/prysm/blob/master/shared/p2p/service_test.go

🐞 Bug Report

I am running v2.1.4 of beaconchain and validator. The validator can't proceed because the beaconchain is not syncing properly.

Description

On restart of the beaconchain, it seems to be stuck trying to complete the initial-sync portion. It keeps repeating this in the log:

Aug 20 09:29:58 nuc4 beacon-chain[1945]: time="2022-08-20 09:29:58" level=info msg="Connected to new endpoint: http://localhost:8551" prefix=powchain Aug 20 09:30:05 nuc4 beacon-chain[1945]: time="2022-08-20 09:30:05" level=info msg="Processing block batch of size 57 starting from 0x99d6671e... 3707969/3708750 - estimated time remaining 4m34s" blocksPerSecond=2.9 peers=45 prefix=initial-sync Aug 20 09:30:05 nuc4 beacon-chain[1945]: time="2022-08-20 09:30:05" level=warning msg="Batch is not processed" error="could not process block in batch: could not process block: could not process block header: parent root 0x5e918c1757a5ae990d67d869f909a344199fe5e0448999782049e5bdf382eb18 does not match the latest block header signing root in state 0x9ef78c510be027cb8abebfe3910c4a3b82f21ce5446d9c541903b7ba25398369" prefix=initial-sync Aug 20 09:30:05 nuc4 beacon-chain[1945]: time="2022-08-20 09:30:05" level=info msg="Processing block batch of size 58 starting from 0x90aa636a... 3708033/3708750 - estimated time remaining 2m4s" blocksPerSecond=5.8 peers=45 prefix=initial-sync ... Aug 20 09:30:06 nuc4 beacon-chain[1945]: time="2022-08-20 09:30:06" level=error msg="Could not get reconstruct full bellatrix block from blinded body" error="could not fetch execution block with txs by hash 0x01d1075f83868e6eb900e4e255d83903d0c869748e858f2b809500f9d563c204: timeout from http.Client" prefix=sync

(see attached for the complete log fragment)

This seems similar to another issue with the same title - but that was for ropsten, but this one is for praeter/goerli. Maybe I should have combined issues? (Maybe the same issue as https://github.com/prysmaticlabs/prysm/issues/11270)

Has this worked before in a previous version?

Yes, this was working before. I was using 2.1.4-rc1 going into the merge. Last night, I stopped and restarted and from there, it kept repeating this in the log. I upgraded to 2.1.4 but it did not improve the situation.

🔬 Minimal Reproduction

🔥 Error

🌍 Your Environment

Operating System: NAME="Ubuntu" VERSION="20.04.4 LTS (Focal Fossa)" x86_64

What version of Prysm are you running? (Which release)

2.1.4

Anything else relevant (validator index / public key)?

Log Fragment: repeated-log.txt

$ geth version Geth Version: 1.10.21-stable Git Commit: 671094279e8d27f4b4c3c94bf8b636c26b473976 Architecture: amd64 Go Version: go1.18.4 Operating System: linux GOPATH= GOROOT=go

geth --goerli --http --cache=2048 --datadir --http.api eth,net,engine,admin --authrpc.vhosts="localhost" --authrpc.jwtsecret=/jwt.hex

/usr/local/bin/beacon-chain --datadir=

go_abi_library(
  name = "my_contract_go_library",
  srcs = [
     "my_contract.sol",
   ],
   # other options
)

# then this can be used as a go_library 
go_library(
  name = "go_default_library",
  srcs = ["library.go"],
  deps = [":my_contract_go_library"],
)

This involves writing the skylark rule which leverages the go-ethereum abigen and solidity compiler in the bazel toolchain to generate the go bindings and expose them as a go_library.

https://docs.bazel.build/versions/master/skylark/spec.html

The benefit here is that users will no longer need to regenerate the go-bindings when updating contracts and we can conform to the same version of solidity (solc) for this project.

since 3 hours my node has stopped working.

Latest Prysm version is v1.3.11. geth VERSION: 1.10.4-stable-aa637fd3

prysm.sh[2165]: time="2021-07-07 11:17:49" level=warning msg="Could not determine if beacon chain started: could not setup beacon chain ChainStart streaming client: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:4000: connect: connection refused": could not connect" prefix=validator

beacon-chain.service - eth2 beacon chain service Loaded: loaded (/etc/systemd/system/beacon-chain.service; enabled; vendor preset: enabled) Active: active (running) since Wed 2021-07-07 11:08:54 CEST; 13min ago Main PID: 2161 (beacon-chain-v1) Tasks: 11 (limit: 36052) Memory: 13.8G CGroup: /system.slice/beacon-chain.service └─2161 /home/eth/prysm/prysm.sh --mainnet --p2p-max-peers=75 --http-web3provider=http://127.0.0.1:8545 --accept-terms-of-use

The ActiveValidatorIndices function is supposed to return the indices of the currently active validators. But when appending to the indices array it appends the index of the validator in the array returned by activeIndicesCache.ActiveIndicesInEpoch(epoch) and not the actual index of the validator.

See: https://github.com/prysmaticlabs/prysm/blob/0326be86b5781b8e9d40d8f003d28a895e71203f/beacon-chain/core/helpers/validators.go#L82-L86

That is why a validator with an index greater than the number of active validators will never get any slot proposal or attestation assignments.

Unfortunately the type v1alpha1.Validator that is returned by activeIndicesCache.ActiveIndicesInEpoch(epoch) also does not contain the validator index.

Opening this issue to discussion memory efficient strategy to allow more cached values for active validator indices. One easy optimization for now is to store the list of inactive validators. Data structure is easy with a map, if an index is not present in the map then it is active. Another suggestion is a radix tree

Create and add a p2p middleware adapter to increment monitoring metrics counters via prometheus.

Some of the requirements to close this issue:

• [ ] Brief design on what metrics are interesting to monitor
• [ ] An easily reusable p2p middleware adapter to monitor incoming and outgoing p2p messages
• [ ] Exposing metrics to prometheus via HTTP server (preferrably something like :8080/metrics). This port should be configurable via flag --monitoring-port.
• [ ] Add flag to disable metrics monitoring --disable-monitoring.

If there are more requirements discovered in the design, please add them to the issue in a comment or editing this description.

🚀 Feature Request

Description

In the event of having a corrupted validator.db an easy and cheap safeguard would be to grab attestations from the last 3 epochs from the beacon-node and check if the validating indexes appear there. This would prevent having a double attestation in most cases. This test can be run only at launchtime and if the validator.db does not show any activity for the last/current epoch. So that the impact will be minimal in long running validators.

I'd like to see what the team think of possible design patterns before submitting a PR, but I think this can be as easy as pulling the last 3 epochs blocks from the node, and comparing the attestations for the indexes of the validating accounts. And stopping if something is found on-chain that is not found on the validator.db with a possible flag --disable-launchtime-check or whatever to override this behavior.

Additional Info

Discussion in Discord: https://discord.com/channels/476244492043812875/730447046930071672/782041664327909417

🐞 Bug Report

Description

beacon-chain does not start with a private node ETH1 (Docker)

Has this worked before in a previous version?

🔬 Minimal Reproduction

🔥 Error

time="2020-11-23 23:31:55" level=error msg="Could not connect to powchain endpoint" error="could not dial eth1 nodes: eth1 node using incorrect chain id, 0 != 1" prefix=powchain

time="2020-11-23 23:31:54" level=warning msg="Running on ETH2 Mainnet" prefix=flags
time="2020-11-23 23:31:54" level=info msg="Using "max_cover" strategy on attestation aggregation" prefix=flags
time="2020-11-23 23:31:54" level=info msg="Checking DB" database-path="/data/beaconchaindata" prefix=node
time="2020-11-23 23:31:55" level=warning msg="No bootstrap addresses supplied" prefix=p2p
time="2020-11-23 23:31:55" level=info msg="Deposit contract: 0x00000000219ab540356cbb839cbe05303d7705fa" prefix=node
time="2020-11-23 23:31:55" level=info msg="Waiting for state to be initialized" prefix=initial-sync
time="2020-11-23 23:31:55" level=info msg="Starting beacon node" prefix=node version="Prysm/v1.0.0-beta.2/f361450e8da18033f5458b018e3e6784295f50ca. Built at: 2020-11-14 02:35:28+00:00"
time="2020-11-23 23:31:55" level=info msg="Waiting to reach the validator deposit threshold to start the beacon chain..." prefix=blockchain
time="2020-11-23 23:31:55" level=info msg="gRPC server listening on port" address="127.0.0.1:4000" prefix=rpc
time="2020-11-23 23:31:55" level=warning msg="You are using an insecure gRPC server. If you are running your beacon node and validator on the same machines, you can ignore this message. If you want to know how to enable secure connections, see: https://docs.prylabs.network/docs/prysm-usage/secure-grpc" prefix=rpc
time="2020-11-23 23:31:55" level=info msg="Starting JSON-HTTP API" address="127.0.0.1:3500" prefix=gateway
time="2020-11-23 23:31:55" level=error msg="Could not connect to powchain endpoint" error="could not dial eth1 nodes: eth1 node using incorrect chain id, 0 != 1" prefix=powchain

🌍 Your Environment

Operating System:

  
   docker -v
Docker version 19.03.8, build afacb8b7f0

  

Eth1 Node: (Docker):

  
docker run -p 127.0.0.1:8545:8545 -p 30303:30303 -v $(HOME)/eth2/node:/root ethereum/client-go --http -http.addr "0.0.0.0"

--or-
docker run -p 0.0.0.0:8545:8545 -p 30303:30303 -v $(HOME)/eth2/node:/root ethereum/client-go --http -http.addr "0.0.0.0"

  

Beacon: (Docker):

  

docker run -it -v$(HOME)/eth2/beacon/.eth2:/data -p 4000:4000 -p 13000:13000 -p 12000:12000/udp --name beacon-node \
  gcr.io/prysmaticlabs/prysm/beacon-chain:stable \
  --datadir=/data \
  --rpc-host=127.0.0.1 \
  --http-web3provider="http://192.168.178.170:8545"

  

What version of Prysm are you running? (Which release)

  
gcr.io/prysmaticlabs/prysm/beacon-chain:stable
Prysm/v1.0.0-beta.2/f361450e8da18033f5458b018e3e6784295f50ca
  

Anything else relevant (validator index / public key)?

🚀 Feature Request

Description

When using configuration file, it would be nice to be able to set validators external signer public keys as a list.

Exemple of wished configuration file:

accept-terms-of-use: true
log-format: json
prater: true
<... other configuration ...>
validators-external-signer-public-keys:
- key1
- key2
- key3
- ...

Teku does support it this way.

When I try to do it right now, I have this kind of error:

time="2022-07-08T07:49:25Z" level=error msg="Mismatched type for flag 'validators-external-signer-public-keys'. Expected 'string' but actual is ''" prefix=main

What type of PR is this?

Syncing Improvement

What does this PR do? Why is it needed?

In smaller devnets, it came up that prysm was waiting an excessive period of time for peers to replenish their limits for our leaky bucket. We modify our bandwidth waiting method, to more accurately gauge when it is the right time to correctly request blocks from those particular peers again. This also adds a unit test for this particular case

Which issues(s) does this PR fix?

N.A

Other notes for review

Hi For past few weeks we are getting these intermittent warning logs from consensus node

Attestation is too old to broadcast, discarding it. Current Slot: xxxx , Attestation Slot: xxx

May i know what could be the reason behind this issue & is it something that is fixed in newer version(3.2.0) ?

Sometimes this logs is flooding & there is not much info about this log over there to fix this.

Versions currently running for Consensus client: beacon-chain-v3.1.1-linux-amd64 Execution client : geth-linux-amd64-1.10.26-e5eb32ac

What type of PR is this?

Feature

What does this PR do? Why is it needed?

It adds Capella support for the ProposeBeaconBlock and GetBeaconBlock endpoints of the validator's REST API.

I connected to promethus and grafana. Metrics shows no loss of att and prop. But when I checked the beaconNode exporter, I found the miss att、prop.

There is another question. Why does it happen Sync Participation is 0%

This PR implements https://github.com/ethereum/consensus-specs/pull/2649, which adds historical summaries to the beacon state and updates them for epoch processing. This is part of the Capella update