Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why run your node? How to do dockerless #844

Merged
merged 6 commits into from
Aug 1, 2023
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion src/.vuepress/config.js
Original file line number Diff line number Diff line change
Expand Up @@ -337,7 +337,6 @@ module.exports = {
'/docs/developers/build/transaction-fees.md',
'/docs/developers/build/system-contracts.md',
'/docs/developers/build/dev-node.md',
'/docs/developers/build/run-a-node.md',
'/docs/developers/build/differences.md',
'/docs/developers/build/json-rpc.md',
'/docs/developers/build/testing-dapps.md',
Expand All @@ -355,6 +354,13 @@ module.exports = {
],
collapsable: true,
},
{
title: "Operating OP nodes",
children: [
'/docs/developers/node-oper/which-node.md',
'/docs/developers/node-oper/dockerless.md'
]
},
'/docs/developers/known-issues.md',
{
title: "Useful Tools",
Expand Down
1 change: 1 addition & 0 deletions src/docs/developers/node-oper/README.md
Original file line number Diff line number Diff line change
@@ -1,71 +1,43 @@
---
title: Running an OP Mainnet or testnet node
title: Running a node without Docker
lang: en-US
---

If you're looking to build an app on OP Mainnet you'll need access to an OP Mainnet node. You have two options - use a hosted node from providers like Alchemy or run your own.

## Hosted node providers

You can get a free, hosted one from [any of these providers](../../useful-tools/providers.md) to get up and building quickly. Of them, [Alchemy](https://www.alchemy.com/optimism) is our preferred node provider, and is used to power our [public endpoint](../../useful-tools/networks.md).

However, you might be interested in running your very own node.
Here we'll go over the process of running an OP Mainnet or testnet node for yourself.

## Upgrades

If you run a node you need to subscribe to [an update feed](../releases.md) (either [the mailing list](https://groups.google.com/a/optimism.io/g/optimism-announce) or [the RSS feed](https://changelog.optimism.io/feed.xml)) to know when to upgrade.
Otherwise, your node will eventually stop working.

## Configuration choices

### Hardware requirements

Replicas need to store the transaction history of OP Mainnet (or the relevant OP testnet) and to run Geth.
They need to be relatively powerful machines (real or virtual).
We recommend at least 16 GB RAM, and an SSD drive with at least 500 GB free (for OP Mainnet).

### Source of synchronization



[The `op-geth` component](../bedrock/explainer.md#execution-client) synchronizes from both other OP Mainnet (or testnet) nodes (https://github.com/ethereum-optimism/optimism/blob/65ec61dde94ffa93342728d324fecf474d228e1f/specs/exec-engine.md#happy-path-sync), meaning L2, [and Ethereum (or the appropriate L1 testnet)](https://github.com/ethereum-optimism/optimism/blob/65ec61dde94ffa93342728d324fecf474d228e1f/specs/exec-engine.md#worst-case-sync) if necessary.

To synchronize only from L1, you edit the [op-node configuration](https://github.com/ethereum-optimism/optimism/blob/65ec61dde94ffa93342728d324fecf474d228e1f/specs/rollup-node.md) to set `OP_NODE_P2P_DISABLE` to `true`.

When you use RPC to get block information (https://github.com/ethereum-optimism/optimism/blob/65ec61dde94ffa93342728d324fecf474d228e1f/specs/rollup-node.md#l2-output-rpc-method), you can specify one of four options for `blockNumber`:

- an actual block number
- **pending**: Latest L2 block
- **latest**: Latest block written to L1
- **finalized**: Latest block fully finalized on L1 (a process that takes 12 minutes with Proof of Stake)



## Docker configuration
Here are the instructions if you want to build you own read-only replica without relying on our images.
These instructions were generated on an Ubuntu 20.04 LTS box, but they should work with other systems too.

The recommended method to create a replica is to use [Docker](https://www.docker.com/) and the Docker images we provide for [`op-geth`](https://github.com/ethereum-optimism/op-geth/releases/latest) and [`op-node`](https://github.com/ethereum-optimism/optimism/releases/).
For `op-node` you need to scroll down to the latest release that has `op-node`.
**Note:** This is *not* the recommended configuration.
While we did QA on these instructions and they work, the QA that the docker images undergo is much more extensive.

They include all the configuration settings.
This is the recommended method because it is what we for our own systems.
As such, the docker images go through a lot more tests than any other configuration.
## Prerequisites

### Configuring and running the node
You’ll need the following software installed to follow this tutorial:

Follow [these instructions](https://github.com/smartcontracts/simple-optimism-node) to build and run the node.
- [Git](https://git-scm.com/)
- [Go](https://go.dev/)
- [Node](https://nodejs.org/en/)
- [Pnpm](https://classic.yarnpkg.com/lang/en/docs/install/)
- [Foundry](https://github.com/foundry-rs/foundry#installation)
- [Make](https://linux.die.net/man/1/make)
- [jq](https://github.com/jqlang/jq)
- [direnv](https://direnv.net)
- [zstd](http://facebook.github.io/zstd/)

This tutorial was checked on:

## Non-docker configuration
| Software | Version | Installation command(s) |
| -------- | ---------- | - |
| Ubuntu | 20.04 LTS | |
| git, curl, jq, make, and zstd | OS default | `sudo apt update` <br> `sudo apt install -y git curl make jq zstd`|
| Go | 1.20 | `wget https://go.dev/dl/go1.20.linux-amd64.tar.gz` <br> `tar xvzf go1.20.linux-amd64.tar.gz` <br> `sudo cp go/bin/go /usr/bin/go` <br> `sudo mv go /usr/lib` <br> `echo export GOROOT=/usr/lib/go >> ~/.bashrc`
| Node | 16.19.0 | `curl -fsSL https://deb.nodesource.com/setup_16.x | sudo -E bash -` <br> `sudo apt-get install -y nodejs`
| pnpm | 8.5.6 | `sudo npm install -g pnpm`
| yarn | 1.22.19 | `sudo npm install -g yarn`
| Foundry | 0.2.0 | `curl -L https://foundry.paradigm.xyz | bash` <br> `. ~/.bashrc` <br> `foundryup`

Here are the instructions if you want to build you own read-only replica without relying on our images.
These instructions were generated on an Ubuntu 20.04 box, but they should work with other systems too.

**Note:** This is *not* the recommended configuration.
While we did QA on these instructions and they work, the QA that the docker images undergo is much more extensive.


### Build the Optimism Monorepo
## Build the Optimism Monorepo

1. Clone the [Optimism Monorepo](https://github.com/ethereum-optimism/optimism).

Expand All @@ -79,17 +51,17 @@ While we did QA on these instructions and they work, the QA that the docker imag

```bash
cd optimism
yarn install
pnpm install
```

1. Build the various packages inside of the Optimism Monorepo.
1. Build the necessary packages inside of the Optimism Monorepo.

```bash
make op-node
yarn build
pnpm build
```

### Build op-geth
## Build op-geth

1. Clone [`op-geth`](https://github.com/ethereum-optimism/op-geth):

Expand All @@ -108,7 +80,7 @@ While we did QA on these instructions and they work, the QA that the docker imag



### Get the data dir
## Get the data dir

The next step is to download the data directory for `op-geth`.

Expand All @@ -135,12 +107,12 @@ The next step is to download the data directory for `op-geth`.
cp jwt.txt ~/optimism/op-node
```

### Scripts to start the different components
## Scripts to start the different components

#### `op-geth`
### `op-geth`

This is the script for OP Goerli.
For OP Mainnet (or other OP networks in the future, [get the sequencer URL here](../../useful-tools/networks.md)).
For OP Mainnet (or other OP networks in the future), [get the sequencer URL here](../../useful-tools/networks.md)).

```
#! /usr/bin/bash
Expand Down Expand Up @@ -181,7 +153,7 @@ Later, for regular usage, you can remove that option to improve geth database in

:::

#### `op-node`
### `op-node`

- Change `<< URL to L1 >>` to a service provider's URL for the L1 network (either L1 Ethereum or Goerli).
- Set `L1KIND` to the network provider you are using (alchemy, infura, etc.).
Expand All @@ -198,7 +170,7 @@ NET=goerli
cd ~/optimism/op-node

./bin/op-node \
--l1=$L1UL \
--l1=$L1URL \
--l1.rpckind=$L1KIND \
--l2=http://localhost:8551 \
--l2.jwt-secret=./jwt.txt \
Expand All @@ -211,7 +183,7 @@ cd ~/optimism/op-node



### The initial synchornization
## The initial synchornization

The datadir provided by Optimism is not updated continuously, so before you can use the node you need a to synchronize it.

Expand Down Expand Up @@ -241,18 +213,33 @@ INFO [06-26|14:02:12.976] Chain head was updated number=4,068,
INFO [06-26|14:02:12.982] Starting work on payload id=0x5542117d680dbd4e
```

#### How long will the synchronization take?
### How long will the synchronization take?

To estimate how long the synchronization will take, you need to first find out how many blocks you synchronize in a minute.

You can use this script, which uses [Foundry](https://book.getfoundry.sh/). and the UNIX Note that this script is for OP Goerli.
For OP Mainnet substitute `https://mainnet.optimism.io`
You can use this script, which uses [Foundry](https://book.getfoundry.sh/).



```sh
#! /usr/bin/bash

export ETH_RPC_URL=http://localhost:8545
T0=`cast block latest number` ; sleep 60 ; T1=`cast block latest number`
CHAIN_ID=`cast chain-id`
echo Chain ID: $CHAIN_ID
echo Please wait

if [ $CHAIN_ID -eq 10 ]; then
L2_URL=https://mainnet.optimism.io
fi


if [ $CHAIN_ID -eq 420 ]; then
L2_URL=https://goerli.optimism.io
fi


T0=`cast block-number` ; sleep 60 ; T1=`cast block-number`
PER_MIN=`expr $T1 - $T0`
echo Blocks per minute: $PER_MIN

Expand All @@ -268,8 +255,8 @@ echo Progress per minute: $PROGRESS_PER_MIN


# How many more blocks do we need?
HEAD=`cast block --rpc-url https://goerli.optimism.io latest number`
BEHIND=`expr $HEAD - $T1`
HEAD=`cast block-number --rpc-url $L2_URL`
BEHIND=`expr $HEAD - $T1`
MINUTES=`expr $BEHIND / $PROGRESS_PER_MIN`
HOURS=`expr $MINUTES / 60`
echo Hours until sync completed: $HOURS
Expand All @@ -281,6 +268,6 @@ fi
```


### Operations
## Operations

It is best to start `op-geth` first and shut it down last.
53 changes: 53 additions & 0 deletions src/docs/developers/node-oper/which-node.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
---
title: Should you run your own node?
qbzzt marked this conversation as resolved.
Show resolved Hide resolved
lang: en-US
---

If you're looking to build an app on OP Mainnet you'll need access to an OP Mainnet node. You have two options - use a hosted node from providers like Alchemy or run your own.


## Hosted node providers

You can get a free, hosted node from [any of these providers](../../useful-tools/providers.md) to get up and building quickly.
Of them, [Alchemy](https://www.alchemy.com/optimism) is our preferred node provider, and is used to power our [public endpoint](../../useful-tools/networks.md).

Many applications include a server component that needs to communicate directly with the blockchain, if only to query for information.
Once an application's needs exceed the free tier, you can use one of the paid tiers to continue to use the hosted node provider.


## Running your own node
qbzzt marked this conversation as resolved.
Show resolved Hide resolved

There are several advantages to running your own node:

- **Cost**. If your application runs a lot of queries ([`eth_call`](https://docs.alchemy.com/reference/eth-call-optimism), [`eth_estimateGas`](https://docs.alchemy.com/reference/eth-estimategas-optimism), etc.) it might be cheaper to run your own node instead of using a service provider.
- **Verification**. Some types of application need to *know* that the state commitments submitted by the sequencer are accurate.
For example, bridges can allow fast withdrawals because they know that the assets deposited with them on L2 really exist, and belong to the user depositing them.
Bridges achieve this, before the


Of course, there are some additional considerations and costs to running a node on your own:

### Hardware

Replicas need to store the transaction history of OP Mainnet (or the relevant OP testnet) and to run both `op-geth` and `op-node`.
They need to be relatively powerful machines (real or virtual).
We recommend at least 16 GB RAM, and an SSD drive with at least 500 GB free (for OP Mainnet).
Note that during installation time you'll need more storage because you temporarily need space for a compressed copy of the latest snapshot as well as the decompressed `op-geth` data directory.


### Synchronization

[The `op-node` component](../bedrock/explainer.md#rollup-node) synchronizes from L1.
You need to get L1 Ethereum access from somewhere, either a node provider or a node that your organization runs.

[The `op-geth` component](../bedrock/explainer.md#execution-client) synchronizes from both other OP Mainnet (or testnet) nodes (https://github.com/ethereum-optimism/optimism/blob/65ec61dde94ffa93342728d324fecf474d228e1f/specs/exec-engine.md#happy-path-sync), meaning L2, [and Ethereum (or the appropriate L1 testnet)](https://github.com/ethereum-optimism/optimism/blob/65ec61dde94ffa93342728d324fecf474d228e1f/specs/exec-engine.md#worst-case-sync) if necessary.

To synchronize only from L1, you edit the [op-node configuration](https://github.com/ethereum-optimism/optimism/blob/65ec61dde94ffa93342728d324fecf474d228e1f/specs/rollup-node.md) to set `OP_NODE_P2P_DISABLE` to `true`.




### Upgrades

If you run a node you need to subscribe to either [the mailing list](https://groups.google.com/a/optimism.io/g/optimism-announce) or [the RSS feed](https://changelog.optimism.io/feed.xml) to know when to upgrade.
qbzzt marked this conversation as resolved.
Show resolved Hide resolved
Otherwise, your node will eventually stop working.
5 changes: 5 additions & 0 deletions src/x
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@

- [OP Goerli](https://datadirs.optimism.io/goerli-bedrock.tar.zst)
This is the script for OP Goerli.
You can use this script, which uses [Foundry](https://book.getfoundry.sh/). and the UNIX Note that this script is for OP Goerli.
./docs/developers/node-oper/dockerless.md
Loading