Categories
Substrate

The Rise of Rust and Blockchain

The Rust programming language is redefining the blockchain space very rapidly. It is my goal to learn this language through an evaluation of blockchain protocols, peer-to-peer networking layers and to understand how the concepts of ownership and modules will drive forward the industry as a whole.

This is a very, very good programming language for safety and performance.

At a high level I understand there are crates and cargo, analogous to npm and node for running packages of software written in Rust. The blockchain industry is increasingly demanding development in the Rust programming language. Here is my overview of Substrate and Libra and how to code is similar or dissimilar in a particular fashion: In learning more about Rust and WASM, WASI and Blockchains I have come to understand that the systems level programming or high performance blockchain will be running in the browser. The Rust WASM Blockchain combination enables powerful applications in the browser. Applications that today I don’t think one would want to have in the browser due to performance; but it will be a combination of having the React and Javascript interface with a WASM based blob that is binary from a program written in Rust. In learning more about Rust and specifically the cargo build system it becomes clear that the development experience will be to leverage the set of libraries across these different projects to effectively create a new ultra-performant enterprise platform that is written in rust and can run in the browser. This could be a number of enterprise applications that are actually using the browser to do the high performant computation and taking the most extensive components of the enterprise application and rewriting them so they can be a WASM blob and run in the browser on desktop and mobile

For an enterprise platform this becomes very important in that it is not just for system of record but for computational tasks such a a blockchain network or ledger across multiple network participants in the browser or for more computational advanced processes that need extra guarantees around security and scalability.

Repos

I am looking into these repos that have been written in Rust and comparing what I see across them:

Starting with Substrate and looking into the pool of the blockchain. The pool is the pool of incoming extrinsics; that are things such as transactions and inherents such as timestamps that are put into a mempool and propagated on the network.

The Rise of Substrate: A multi-blockchain universe

Substrate is the antithesis of a one distributed actor framework smart contract world computer. There will be thousands of application specific blockchains built on this framework. In this framework the blockchain is more of a living organism that is in constant change through an ongoing set of state transitions driven through extrinsics.

Substrate Specifications

Runtime architecture WebAssembly
Implementation language Rust
Component technologies provided with Substrate
Here are some of the technologies bundled with Substrate.

You can swap out any of these components for your own alternative:

Networking LibP2P
Consensus algorithm Hybrid PBFT/Aurand
Randomness beacon Collective coin flipping
Authentication algorithm Edwards-Curve Ed25519
Hashing function Blake2b
Address format Versioned Base-58 + Blake2b checksum

Here is a link to the Substrate Developer Hub

A few commands to get going:

substrate-node-new <node-name> <author>

To create a runtime module:

substrate-module-new <module-name>

First the RPCs –

Author

Hash, BlockHash

Substrate authoring RPC API

submit_extrinsic,
pending_extrinsic,
watch_extrinsic,
unwatch_extrinisic

new client: pool: subscriptions)

Chain

Substrate blockchain API

Hash,
Header,
Number,
SignedBlock

Relay Chain and the Canonical Chain

Subscribe and Unsubscribe -> Block heads

header, block, block_hash, finalised_head, subscribe_new_head, unsubscribe_new_head, subscribe_finalised_heads, unsubscribe_finalised_heads.

State

Substrate state API

Hash,

call

storage

storage_hash

storage_size

runtime_version

query_storage

State Subscribe and Unsubscribe -> Runtime Version

Subscribe and Unsubscribe -> Storage

call(method: String, data: Bytes, block:), storage(key: StorageKey, block:)
storage_hash(key: StorageKey, block:), 
storage_size(key: StorageKey, block:), metadata(block:), query_storage(key: StorageKey, from: Block::Hash, to:),
subscribe_storage, unsubscribe_storage, runtime_version, subscribe_runtime_version)

System

Substrate system API

system_name,
system_version,
system_chain,
system_properties,
system_health


Rust provides a Keystore and the runtime modules.
Abstract block format
crypto database agnostic

 

For interacting with the chain using the RPCs go to https://polkadot.js.org/apps/#/toolbox

 

– base-16 modified merkle trie (aka Ethereum) Patricia Merkle Tree – Sparse Merkle Trees
– Binary Merkle Trie

Wasm “execute_block” function
Extensible Networking, CLI, RPC
Roll your own Consensus
Blockchain PBFT
Probablistic Finality
Consensus API

What do I get with Substrate?

  • Shared ancestry finality tool
    grandpa
  • Hot-swapple, pluggable Consensus
  • Hot-upgradeable, pluggable
  • STFLight client
  • Chain synchronisation
  • Pub/Sub WebSocket JSON-RPC
  • Transaction queue
  • Pervasive, secure networking
  • JS implementation
  • Modular SRML if you want

Interchain connectivity via Polkadot
execute_block function

networking, block authoring and transaction queue
CORE SUBSTRATE

  • RPCs sync, databases, crypto, networking, storage
  • Telemetry
  • Light client
  • Change tracking
  • Pluggable consensus
  • Address formats
  • Low level JS utils

SRML SUBSTRATE

High level JS helpers
Front-end GUI infrastrucuture
Block authoring and transaction queue
JSON config
Chain explorer
event tracking

SOLO CHAIN | SOLO CHAIN + BRIDGE | PARACHAIN

Architected on industry-standard WebAssembly
Highly extensible Libp2p networking
Rust-based primary implementation for speed and reliability
Javascript secondary implementation for developability
Wasm WebAssembly interpreter, written in Rust

Substrate is a blockchain platform with a completely generic State Transition Function ( STF ) and modular components for consensus, networking and configuration.

Despite being “completely generic”, it comes with both standards and conventions (particularly with the Substrate Runtime Module Library ( SRML
) regarding the underlying data-structures which power the STF, thereby making rapid blockchain development a reality.

Each of these datatypes corresponds to a Rust trait. They are:

Hash, a type which encodes a cryptographic digest of some data. Typically just a 256-bit quantity.

BlockNumber, a type which encodes the total number of ancestors any valid block has. Typically a 32-bit quantity.

Digest, basically just a series of DigestItems, this encodes all information that is relevant for a light-client to have at hand within the block.

DigestItem, a type which must be able to encode one of a number of “hard-wired” alternatives relevant to consensus and change-tracking as well as any number of “soft-coded” variants, relevant to specific modules within the runtime.

Header, a type which is representative (cryptographically or otherwise) of all information relevant to a block. It includes the parent hash, the storage root and the extrinsics trie root, the digest and a block number.

Extrinsic, a type to represent a single piece of data external to the blockchain that is recognised by the blockchain. This typically involves one or more signatures, and some sort of encoded instruction (e.g. for transferring ownership of funds or calling into a smart contract).

Block, essentially just a combination of Header and a series of Extrinsics, together with a specification of the hashing algorithm to be used

impl system::Trait for Runtime {
/// The identifier used to distinguish between accounts.
type AccountId = AccountId;
/// The index type for storing how many extrinsics an account has signed.
type Index = Nonce;
/// The index type for blocks.
type BlockNumber = BlockNumber;
/// The type for hashing blocks and tries.
type Hash = Hash;
/// The hashing algorithm used.
type Hashing = BlakeTwo256;
/// The header digest type.
type Digest = generic::Digest<Log>;
/// The header type.
type Header = generic::Header<BlockNumber, BlakeTwo256, Log>;
/// The ubiquitous event type.
type Event = Event;
/// The ubiquitous log type.
type Log = Log;
/// The ubiquitous origin type.
type Origin = Origin;
}

 

Substrate Runtime Architecture

2019-08-26 18_50_37-Architecture of a Runtime · Substrate Developer Hub

 

Runtime Proxy Functions

Have to implement these three:

  • what version
  • what client
  • how to execute a block
// Implement our runtime API endpoints. This is just a bunch of proxying.


impl_runtime_apis! {

    impl runtime_api::Core<Block> for Runtime {

        fn version() -> RuntimeVersion {

            VERSION

        }

        fn execute_block(block: Block) {

            Executive::execute_block(block)

        }

        fn initialize_block(header: &<Block as BlockT>::Header) {

            Executive::initialize_block(header)

        }

        fn authorities() -> Vec<AuthorityId> {

            panic!("Deprecated, please use `AuthoritiesApi`.")

        }

    }

    impl runtime_api::Metadata<Block> for Runtime {

        fn metadata() -> OpaqueMetadata {

            Runtime::metadata().into()

        }

    }

    impl block_builder_api::BlockBuilder<Block> for Runtime {

        fn apply_extrinsic(extrinsic: <Block as BlockT>::Extrinsic) -> ApplyResult {

            Executive::apply_extrinsic(extrinsic)

        }

        fn finalize_block() -> <Block as BlockT>::Header {

            Executive::finalize_block()

        }

        fn inherent_extrinsics(data: InherentData) -> Vec<<Block as BlockT>::Extrinsic> {

            data.create_extrinsics()

        }

        fn check_inherents(block: Block, data: InherentData) -> CheckInherentsResult {

            data.check_extrinsics(&block)

        }

        fn random_seed() -> <Block as BlockT>::Hash {

            System::random_seed()

        }

    }

    impl runtime_api::TaggedTransactionQueue<Block> for Runtime {

        fn validate_transaction(tx: <Block as BlockT>::Extrinsic) -> TransactionValidity {

            Executive::validate_transaction(tx)

        }

    }

    impl consensus_aura::AuraApi<Block> for Runtime {

        fn slot_duration() -> u64 {

            Aura::slot_duration()

        }

    }

    impl offchain_primitives::OffchainWorkerApi<Block> for Runtime {

        fn offchain_worker(n: NumberFor<Block>) {

            Executive::offchain_worker(n)

        }

    }

    impl consensus_authorities::AuthoritiesApi<Block> for Runtime {

        fn authorities() -> Vec<AuthorityId> {

            Consensus::authorities()

        }

    }

}

Runtime Module Template

Module’s configutation trait, storage, declartion and event.

/// A runtime module template with necessary imports

/// Feel free to remove or edit this file as needed.

/// If you change the name of this file, make sure to update its references in runtime/src/lib.rs

/// If you remove this file, you can remove those references

/// For more guidance on Substrate modules, see the example module

/// https://github.com/paritytech/substrate/blob/master/srml/example/src/lib.rs

use support::{decl_module, decl_storage, decl_event, StorageValue, dispatch::Result};

use system::ensure_signed;

/// The module's configuration trait.

pub trait Trait: system::Trait {

    // TODO: Add other types and constants required configure this module.

    /// The overarching event type.

    type Event: From<Event<Self>> + Into<<Self as system::Trait>::Event>;

}

/// This module's storage items.

decl_storage! {

    trait Store for Module<T: Trait> as TemplateModule {

        // Just a dummy storage item.

        // Here we are declaring a StorageValue, `Something` as a Option<u32>

        // `get(something)` is the default getter which returns either the stored `u32` or `None` if nothing stored

        Something get(something): Option<u32>;

    }

}

decl_module! {

    /// The module declaration.

    pub struct Module<T: Trait> for enum Call where origin: T::Origin {

        // Initializing events

        // this is needed only if you are using events in your module

        fn deposit_event<T>() = default;

        // Just a dummy entry point.

        // function that can be called by the external world as an extrinsics call

        // takes a parameter of the type `AccountId`, stores it and emits an event

        pub fn do_something(origin, something: u32) -> Result {

            // TODO: You only need this if you want to check it was signed.

            let who = ensure_signed(origin)?;

            // TODO: Code to execute when something calls this.

            // For example: the following line stores the passed in u32 in the storage

            <Something<T>>::put(something);

            // here we are raising the Something event

            Self::deposit_event(RawEvent::SomethingStored(something, who));

            Ok(())

        }

    }

}

decl_event!(

    pub enum Event<T> where AccountId = <T as system::Trait>::AccountId {

        // Just a dummy event.

        // Event `Something` is declared with a parameter of the type `u32` and `AccountId`

        // To emit this event, we call the deposit function, from our runtime funtions

        SomethingStored(u32, AccountId),

    }

);

Runtime Module Key Value Example

This is simple runtime module that stores a key value map.

use srml_support::{StorageMap, dispatch::Result};

pub trait Trait: system::Trait {}

decl_module! {

    pub struct Module<T: Trait> for enum Call where origin: T::Origin {

        fn set_mapping(_origin, key: u32, value: u32) -> Result {

            <Value<T>>::insert(key, value);

            Ok(())

        }

    }

}

decl_storage! {

    trait Store for Module<T: Trait> as RuntimeExampleStorage {

        Value: map u32 => u32;

    }

}

Libra

Networking level and substrate based on libp2p core.

Substrate based on:

use libp2p::core::

Libra based on:

parity_multiaddr::Multiaddr;
fn listen_on(&self, addr: Multiaddr) -> Result<(Self::Listener, Multiaddr)

– chain level

I was able to get Libra working on my Linux Subsystem using the following commands:

git clone https://github.com/libra/libra

cd libra

./scripts/dev_setup.sh

source /home/<user>/.cargo/env

./scripts/cli/start_cli_testnet.sh

Libra Blockchain API

account_state_sets

fn put_account_state_set(
    store: &StateStore,
    account_state_set: Vec<(AccountAddress, AccountStateBlob)>,
    version: Version,
    root_hash: HashValue,
    expected_nodes_created: usize,
    expected_nodes_retired: usize,
    expected_blobs_retired: usize,
) -> HashValue

use rocksdb

pub type Read Options = rocksdb::ReadOptions;


A simple tx script in Move IR that transfers a coin from address to another:


public main(payee: address, amount: u64) {
  let coin: 0x0.Currency.Coin = 0x0
    .Currency
    .withdraw_from_sender(copy(amount));

  0x0.Currency.deposit(copy(payee), move(coin));
}

Oasis: Blockchain and WASM

By leveraging compiler support and tools built for Wasm and WASI, the blockchain becomes a powerful tool for high-integrity — and even confidential — general-purpose “cloud” computation.

use oasis_std::Context;
use statesets_std::Context;

#[derive(oasis_std::Service)]
struct Quickstart;
impl Quickstart { 
pub fn new(_ctx: &Context) -> Self {

Self

}

pub fn say_hello(&mut self, ctx: &Context) -> String {

format!("Hello, {}!", ctx.sender())

}

}

fn main() {

oasis_std::service!(Quickstart);

}

#[cfg(test)]

mod tests {

extern crate oasis_test;

use super::*;

#[test]

fn test() {

let sender = oasis_test::create_account(1);

let ctx = Context::default().with_sender(sender);

let mut client = Quickstart::new(&ctx);

println!("{}", client.say_hello(&ctx));

}

}

From https://medium.com/oasislabs/blockchain-flavored-wasi-50e3612b8eba:

Notes on struct, impl, and pub fn.

Becomes one main.rs file as opposed to separated. The compiler checks and the gives wasm blog which is deployed on chain with rpc endpoints getting, changing and setting states.

define and implement Oasis service RPCs in Rust.

fn main() { oasis_std::service!(Ballot); }

Compiles to WASM. Deploy to the platform and setup client to call it.

use oasis_std::Context;

use map_vec::Map; use oasis_std::{Address, Context};

[derive(oasis_std::Service)]

derive [Serialize, Deserialize, Service]

pub struct X {

fields: <String>,

''

}
pub fn new(ctx: &Context, description: String, candidates: Vec<String>) -> Self { Ok(Self { description, tally: vec![0; candidates.len()], candidates, accepting_votes: true, admin: ctx.sender(), voters: Map::new(), }) }

/// Returns the candidates being voted upon.

pub fn candidates(&self, _ctx: &Context) -> Vec<&str> { self.candidates.iter().map(String::as_ref).collect() }

/// Returns the description of this ballot.

pub fn description(&self, _ctx: &Context) -> &str { &self.description }

/// Returns whether voting is still open. pub fn voting_open(&self, _ctx: &Context) -> bool { self.accepting_votes }

we have access to the state of the service, as provided by a reference to self.

For state changes:

you’ll see that &self has changed to &mut self, but this is just Rust’s way to know that you want a mutable reference.

Then define Getter Fns to Get State from the Stores

similar to the hyperledger composer model

define a data model in cto file declare assets, transactions and events

define javascript functions for logic with refernece to namespace is same as impl with struct (service state object) RPCs that are defined as pub fn’s.

instead of seperate REST API to make calls to on the model.

A service is created on chain with RPC endpoints.

Clients can call the service endpoint and add listeners for events from the service.

Blockchain WASI

While cloud computing has long brought cost and ease of use, switching from an on-prem solution to cloud has traditionally come with its own inherent risks including a degradation in security and a lack of auditability. These are areas that have the potential to be solved with new emerging technologies including Web Assembly, the Web Assembly System Interface, and blockchain.

We propose a mechanism for trustworthy, uncensorable, and autonomous cloud computation based on the combination of three emerging technologies: Web Assembly, the Web Assembly System Interface, and blockchain.

Oasis Network

The Oasis Network uses 3 main protocols for communication:

Confidentiality is achieved in the Oasis Network by relying on trusted execution environments (TEEs) to secure the execution of any given smart contract. Initially, the Oasis Network will utilize Intel SGX. As more TEE technologies mature, we expect to support more than TEEs than Intel SGX.

from https://docs.oasis.dev/operators/architectural-overview.html#modular-architecture

libp2p

lib-p2p is a modularized and extensible network stack to overcome the networking challenges faced when doing peer-to-peer applications. It is very well defined, composable, swappable; a modular system of protocols, specifications, and libraries that enable the development of peer-to-peer network applications.

It ultimately is a collection of peer-to-peer protocols for finding peers, connecting to them for finding content, and transferring it to them.

Here is a sample ping example from rust-libp2p:

use futures::{prelude::*, future};

use libp2p::{ identity, PeerId, ping::{Ping, PingConfig}, Swarm };

use std::env;

fn main() {

    env_logger::init();

    // Create a random PeerId.

    let id_keys = identity::Keypair::generate_ed25519();

    let peer_id = PeerId::from(id_keys.public());

    println!("Local peer id: {:?}", peer_id);

    // Create a transport.

    let transport = libp2p::build_development_transport(id_keys);

    // Create a ping network behaviour.

    //

    // For illustrative purposes, the ping protocol is configured to

    // keep the connection alive, so a continuous sequence of pings

    // can be observed.

    let behaviour = Ping::new(PingConfig::new().with_keep_alive(true));

    // Create a Swarm that establishes connections through the given transport

    // and applies the ping behaviour on each connection.

    let mut swarm = Swarm::new(transport, behaviour, peer_id);

    // Dial the peer identified by the multi-address given as the second

    // command-line argument, if any.

    if let Some(addr) = env::args().nth(1) {

        let remote_addr = addr.clone();

        match addr.parse() {

            Ok(remote) => {

                match Swarm::dial_addr(&mut swarm, remote) {

                    Ok(()) => println!("Dialed {:?}", remote_addr),

                    Err(e) => println!("Dialing {:?} failed with: {:?}", remote_addr, e)

                }

            },

            Err(err) => println!("Failed to parse address to dial: {:?}", err),

        }

    }

    // Tell the swarm to listen on all interfaces and a random, OS-assigned port.

    Swarm::listen_on(&mut swarm, "/ip4/0.0.0.0/tcp/0".parse().unwrap()).unwrap();

    // Use tokio to drive the `Swarm`.

    let mut listening = false;

    tokio::run(future::poll_fn(move || -> Result<_, ()> {

        loop {

            match swarm.poll().expect("Error while polling swarm") {

                Async::Ready(Some(e)) => println!("{:?}", e),

                Async::Ready(None) | Async::NotReady => {

                    if !listening {

                        if let Some(a) = Swarm::listeners(&swarm).next() {

                            println!("Listening on {:?}", a);

                            listening = true;

                        }

                    }

                    return Ok(Async::NotReady)

                }

            }

        }

    }));

}

1 reply on “The Rise of Rust and Blockchain”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s