Blurt full RPC setup guide (late 2022)

in rpc •  2 years ago 

blurtrpc guide.png

Valid as of Blurt v0.8.

Hardware requirements

  • At least 4 core CPU, preferably scores >1200 single-core and >4000 multi-core on Geekbench 5
  • 8GB RAM
  • 100GB SSD free space, preferably NVMe
  • No GPU is required
  • 10Mbps bandwidth

If running on residential internet connection behind CGNAT, a nearby VPS is required for a public node. Anything $5/month or less should work as long as it has sufficient bandwidth.

Dependency setup

For this guide, we will be using Ubuntu Server 22.04 LTS.

Essentials

First of all, install some essential packages:

sudo apt-get update && sudo apt-get upgrade
sudo apt-get install wget curl iotop iftop htop jq

Python 3.8

As it comes with Python 3.10 by default, you would need to install Python 3.8 until Nexus gets updated to support the latest Python version. You may find the original tutorial here, but to summarize it in a few commands:

sudo apt-get install software-properties-common -y
sudo add-apt-repository ppa:deadsnakes/ppa -y
sudo apt-get update
sudo apt install -y \
    python3.8 \
    python3.8-dev \
    python3.8-venv \
    python3.8-distutils \
    python3.8-lib2to3 \
    python3.8-gdbm
wget https://bootstrap.pypa.io/get-pip.py
python3.8 get-pip.py
python3.8 -m pip install --upgrade pip

You may also need to replace every instance of python3 Nexus commands in this guide with python3.8 to use the above installation.

blurtd and Postgresql

sudo apt-get install -y \
    autoconf \
    automake \
    cmake \
    g++ \
    git \
    zlib1g-dev \
    libbz2-dev \
    libsnappy-dev \
    libssl-dev \
    libtool \
    make \
    pkg-config \
    doxygen \
    libncurses5-dev \
    libreadline-dev \
    perl \
    python3 \
    python3-jinja2 \
    libboost-all-dev \
    postgresql

Postgresql setup

Data directory

By default, the postgresql database is stored in the boot drive. To store it in another directory, create the required folder structure with the appropriate permissions:

cd /otherdrive
mkdir -p db/14/main
sudo chown -R postgres:postgres db
sudo chmod -R 700 db

Set the data directory in /etc/postgresql/14/main/postgresql.conf:

data_directory = '/otherdrive/db/14/main'

And tune the database as such in the same config file (below values are for 8GB RAM, scale according to what you have):

effective_cache_size = 6GB
shared_buffers = 1GB
work_mem = 64MB
maintenance_work_mem = 256MB
synchronous_commit = off
checkpoint_timeout = 20min
max_wal_size = 1GB

Restart postgresql to reflect the changes.

sudo service postgresql restart

Compile blurtd

git clone https://gitlab.com/blurt/blurt
cd blurt
git submodule update --init --recursive
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release -DCLEAR_VOTES=OFF -DENABLE_MIRA=OFF -DLOW_MEMORY_NODE=OFF -DSKIP_BY_TX_ID=OFF ..
make -j$(nproc)
sudo make install

Setup Nexus db

Enter the psql shell as postgres user:

sudo -i -u postgres psql

Create the db with permissions and users:

# Create required role for user
CREATE ROLE user LOGIN PASSWORD 'userpass';

# Set password if needed
ALTER ROLE user WITH PASSWORD 'rolepass';

# Database used by hived
CREATE DATABASE nexus;

Authentication

By default, Postgresql allows only for peer authentication. You may enable password authentication by appending the following line in /etc/postgresql/14/main/pg_hba.conf:

local all all peer # after this line
local all all md5 # add this

For more details, refer to the documentation here.

Block Log

Download a copy of the Blurt block_log file into ~/.blurtd/blockchain:

wget -c https://blurtdev.techcoderx.com/block_log

Genesis snapshot

Download genesis snapshot file into blurtd data dir (default ~/.blurtd):

wget https://cloudflare-ipfs.com/ipfs/QmPrwVpwe4Ya46CN9LXNnrUdWvaDLMwFetMUdpcdpjFbyu -O snapshot.json

Create ramdisk

98% of the time the replay is bottlenecked by disk I/O if the shared_memory.bin file is located on disk. Move it to ramdisk to reduce SSD wear and significantly better performance, hence the memory requirements above. Replace 10G with the size of the ramdisk desired.

cd ~
mkdir ramdisk
sudo mount -t tmpfs -o rw,size=10G tmpfs ~/ramdisk

Alternatively, you may use /dev/shm directory which by default is allocated with half of total system memory.

Configure hived

Generate a blurtd config file if not already, by starting blurtd and immediately stopping it with Ctrl+C:

blurtd

Then edit the config.ini file in the data directory as follows (find the key and modify its value, add them if it is not there):

Plugins

# Basic
plugin = witness webserver p2p json_rpc database_api network_broadcast_api condenser_api block_api

# Get accounts/witness
plugin = account_by_key account_by_key_api

# Transaction status
plugin = transaction_status transaction_status_api

# Account history
plugin = account_history_rocksdb account_history_api

Shared file directory

Specify path to folder containing shared_memory.bin:

shared-file-dir = "/home/<username>/ramdisk"

Shared file size

At least 6GB is needed (currently 4.8GB), this value will increase over time.

shared-file-size = 6G

Spam accounts

Add some spam accounts:

spam-accounts = social

Endpoints

Configure the IP and port of the endpoints to be made available for seed and API calls.

p2p-endpoint = 0.0.0.0:2001
webserver-http-endpoint = 0.0.0.0:8091

Replay!

Finally we are ready to replay the blockchain. Start a screen or tmux session and run the following:

blurtd --replay-blockchain

If your blurtd data directory is not the default ~/.blurtd, specify it in the above command with -d /path/to/data/dir.

Nexus

Installation

git clone https://gitlab.com/blurt/openblurt/nexus.git
cd nexus
pip3.8 install -e .[test]

Configure

At the end of the ~/.bashrc file, define the env vars:

export DATABASE_URL=postgresql://<username>:<password>@localhost:5432/nexus
export HTTP_SERVER_PORT=8092
export STEEMD_URL=http://blurtdurl:8091

If the snapshot.json file is in a non-default data dir, specify path to that file:

export SNAPSHOT_PATH=/path/to/snapshot.json

Sync the db

In screen or tmux session:

hive sync

Start server

In screen or tmux session:

hive server

Reverse proxy

At this point, you should have the required pieces to serve a public Blurt node. To put them together into a single endpoint, we need a reverse proxy such as jussi or rpc-proxy.

If your node is on residential internet behind CGNAT, setup the VPN and forward the ports (blurtd and nexus server) here.

For those who need a rpc-proxy config (installation guide here), you can find them here. Replace target URLs with the actual URL to the respective services.

At this point, you should now have a Blurt API node accessible at port 5002 which can be put behind nginx and be accessible by the public.

Server resource statistics

blurtd (v0.8.2, all plugins)

block_log file size: 13 GB
shared_memory.bin file size (stored in /dev/shm): 4.8 GB
account-history-rocksdb-storage folder size: 16 GB
blockchain folder size: 28 GB

nexus

Output of SELECT pg_size_pretty( pg_database_size('hive') );
Database size: 11 GB
RAM usage: 2 GB (probably can be run with 1GB shared_buffers)


Hive version of this guide can be found here.

Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE BLURT!
Sort Order:  

Congratulations!

You have recieved a coconutty upvote! 🥥 Thank you for contributing to the Blurt Blockchain!

Keep up the great work!

Curated by @outofthematrix!

Please take a moment to vote for my witness. 🗳️ https://blurtwallet.com/~witnesses?highlight=outofthematrix


curationcoconut.jpg

  ·  2 years ago  ·  

Nice one... this teases me enough to put one full node behind Cloudflare.

What is the nexus for?

  ·  2 years ago  ·  

Serves anything non-consensus (i.e. communities, rankings, blog etc)

  ·  2 years ago  ·  

Do you run all of that on one machine with the specs you gave at the top of the post?

I was going to use separate VPS machines to run each, but if it can all run on one machine, I'll just add Nexus and RPC proxy to each of our current cluster of 4 RPCs.

  ·  2 years ago  ·  

Everything can be run within one machine which is what most setups should do, but it is also possible to run them on separate machines.

  ·  2 years ago  ·   (edited)

Awesome… well done. Thanks for all your support on Dtube also.

I just Voted your Witness. Would be great to have thousands of Blurt nodes.

CFDA6A0F-9E0B-4541-AF52-F89DA4D733CC.jpeg



Posted from http://BlurtFree.com

  ·  2 years ago  ·  

Thanks for the guide. I will setup this on my server next week