The Rust programming language is redefining the blockchain space very rapidly. It is my goal to learn this language through an evaluation of blockchain protocols, peer-to-peer networking layers and to understand how the concepts of ownership and modules will drive forward the industry as a whole.
This is a very, very good programming language for safety and performance.
At a high level I understand there are crates and cargo, analogous to npm and node for running packages of software written in Rust. The blockchain industry is increasingly demanding development in the Rust programming language. Here is my overview of Substrate and Libra and how to code is similar or dissimilar in a particular fashion: In learning more about Rust and WASM, WASI and Blockchains I have come to understand that the systems level programming or high performance blockchain will be running in the browser. The Rust WASM Blockchain combination enables powerful applications in the browser. Applications that today I don’t think one would want to have in the browser due to performance; but it will be a combination of having the React and Javascript interface with a WASM based blob that is binary from a program written in Rust. In learning more about Rust and specifically the cargo build system it becomes clear that the development experience will be to leverage the set of libraries across these different projects to effectively create a new ultra-performant enterprise platform that is written in rust and can run in the browser. This could be a number of enterprise applications that are actually using the browser to do the high performant computation and taking the most extensive components of the enterprise application and rewriting them so they can be a WASM blob and run in the browser on desktop and mobile
For an enterprise platform this becomes very important in that it is not just for system of record but for computational tasks such a a blockchain network or ledger across multiple network participants in the browser or for more computational advanced processes that need extra guarantees around security and scalability.
Repos
I am looking into these repos that have been written in Rust and comparing what I see across them:
Starting with Substrate and looking into the pool of the blockchain. The pool is the pool of incoming extrinsics; that are things such as transactions and inherents such as timestamps that are put into a mempool and propagated on the network.
The Rise of Substrate: A multi-blockchain universe
Substrate is the antithesis of a one distributed actor framework smart contract world computer. There will be thousands of application specific blockchains built on this framework. In this framework the blockchain is more of a living organism that is in constant change through an ongoing set of state transitions driven through extrinsics.
Substrate Specifications
Runtime architecture WebAssembly
Implementation language Rust
Component technologies provided with Substrate
Here are some of the technologies bundled with Substrate.
You can swap out any of these components for your own alternative:
Networking LibP2P
Consensus algorithm Hybrid PBFT/Aurand
Randomness beacon Collective coin flipping
Authentication algorithm Edwards-Curve Ed25519
Hashing function Blake2b
Address format Versioned Base-58 + Blake2b checksum
Here is a link to the Substrate Developer Hub
A few commands to get going:
substrate-node-new <node-name> <author>
To create a runtime module:
substrate-module-new <module-name>
First the RPCs –
Author
Hash, BlockHash
Substrate authoring RPC API
submit_extrinsic,
pending_extrinsic,
watch_extrinsic,
unwatch_extrinisic
new client: pool: subscriptions)
Chain
Substrate blockchain API
Hash,
Header,
Number,
SignedBlock
Relay Chain and the Canonical Chain
Subscribe and Unsubscribe -> Block heads
header, block, block_hash, finalised_head, subscribe_new_head, unsubscribe_new_head, subscribe_finalised_heads, unsubscribe_finalised_heads.
State
Substrate state API
Hash,
call
storage
storage_hash
storage_size
runtime_version
query_storage
State Subscribe and Unsubscribe -> Runtime Version
Subscribe and Unsubscribe -> Storage
call(method: String, data: Bytes, block:), storage(key: StorageKey, block:)
storage_hash(key: StorageKey, block:),
storage_size(key: StorageKey, block:), metadata(block:), query_storage(key: StorageKey, from: Block::Hash, to:),
subscribe_storage, unsubscribe_storage, runtime_version, subscribe_runtime_version)
System
Substrate system API
system_name,
system_version,
system_chain,
system_properties,
system_health
Rust provides a Keystore and the runtime modules.
Abstract block format
crypto database agnostic
For interacting with the chain using the RPCs go to https://polkadot.js.org/apps/#/toolbox
– base-16 modified merkle trie (aka Ethereum) Patricia Merkle Tree – Sparse Merkle Trees
– Binary Merkle Trie
Wasm “execute_block” function
Extensible Networking, CLI, RPC
Roll your own Consensus
Blockchain PBFT
Probablistic Finality
Consensus API
What do I get with Substrate?
- Shared ancestry finality tool
grandpa
- Hot-swapple, pluggable Consensus
- Hot-upgradeable, pluggable
- STFLight client
- Chain synchronisation
- Pub/Sub WebSocket JSON-RPC
- Transaction queue
- Pervasive, secure networking
- JS implementation
- Modular SRML if you want
Interchain connectivity via Polkadot
execute_block function
networking, block authoring and transaction queue
CORE SUBSTRATE
- RPCs sync, databases, crypto, networking, storage
- Telemetry
- Light client
- Change tracking
- Pluggable consensus
- Address formats
- Low level JS utils
SRML SUBSTRATE
High level JS helpers
Front-end GUI infrastrucuture
Block authoring and transaction queue
JSON config
Chain explorer
event tracking
SOLO CHAIN | SOLO CHAIN + BRIDGE | PARACHAIN
Architected on industry-standard WebAssembly
Highly extensible Libp2p networking
Rust-based primary implementation for speed and reliability
Javascript secondary implementation for developability
Wasm WebAssembly interpreter, written in Rust
Substrate is a blockchain platform with a completely generic State Transition Function ( STF ) and modular components for consensus, networking and configuration.
Despite being “completely generic”, it comes with both standards and conventions (particularly with the Substrate Runtime Module Library ( SRML
) regarding the underlying data-structures which power the STF, thereby making rapid blockchain development a reality.
Each of these datatypes corresponds to a Rust trait. They are:
Hash, a type which encodes a cryptographic digest of some data. Typically just a 256-bit quantity.
BlockNumber, a type which encodes the total number of ancestors any valid block has. Typically a 32-bit quantity.
Digest, basically just a series of DigestItems, this encodes all information that is relevant for a light-client to have at hand within the block.
DigestItem, a type which must be able to encode one of a number of “hard-wired” alternatives relevant to consensus and change-tracking as well as any number of “soft-coded” variants, relevant to specific modules within the runtime.
Header, a type which is representative (cryptographically or otherwise) of all information relevant to a block. It includes the parent hash, the storage root and the extrinsics trie root, the digest and a block number.
Extrinsic, a type to represent a single piece of data external to the blockchain that is recognised by the blockchain. This typically involves one or more signatures, and some sort of encoded instruction (e.g. for transferring ownership of funds or calling into a smart contract).
Block, essentially just a combination of Header and a series of Extrinsics, together with a specification of the hashing algorithm to be used
impl system::Trait for Runtime {
/// The identifier used to distinguish between accounts.
type AccountId = AccountId;
/// The index type for storing how many extrinsics an account has signed.
type Index = Nonce;
/// The index type for blocks.
type BlockNumber = BlockNumber;
/// The type for hashing blocks and tries.
type Hash = Hash;
/// The hashing algorithm used.
type Hashing = BlakeTwo256;
/// The header digest type.
type Digest = generic::Digest<Log>;
/// The header type.
type Header = generic::Header<BlockNumber, BlakeTwo256, Log>;
/// The ubiquitous event type.
type Event = Event;
/// The ubiquitous log type.
type Log = Log;
/// The ubiquitous origin type.
type Origin = Origin;
}
Substrate Runtime Architecture
Runtime Proxy Functions
Have to implement these three:
- what version
- what client
- how to execute a block
// Implement our runtime API endpoints. This is just a bunch of proxying.
impl_runtime_apis! {
impl runtime_api::Core<Block> for Runtime {
fn version() -> RuntimeVersion {
VERSION
}
fn execute_block(block: Block) {
Executive::execute_block(block)
}
fn initialize_block(header: &<Block as BlockT>::Header) {
Executive::initialize_block(header)
}
fn authorities() -> Vec<AuthorityId> {
panic!("Deprecated, please use `AuthoritiesApi`.")
}
}
impl runtime_api::Metadata<Block> for Runtime {
fn metadata() -> OpaqueMetadata {
Runtime::metadata().into()
}
}
impl block_builder_api::BlockBuilder<Block> for Runtime {
fn apply_extrinsic(extrinsic: <Block as BlockT>::Extrinsic) -> ApplyResult {
Executive::apply_extrinsic(extrinsic)
}
fn finalize_block() -> <Block as BlockT>::Header {
Executive::finalize_block()
}
fn inherent_extrinsics(data: InherentData) -> Vec<<Block as BlockT>::Extrinsic> {
data.create_extrinsics()
}
fn check_inherents(block: Block, data: InherentData) -> CheckInherentsResult {
data.check_extrinsics(&block)
}
fn random_seed() -> <Block as BlockT>::Hash {
System::random_seed()
}
}
impl runtime_api::TaggedTransactionQueue<Block> for Runtime {
fn validate_transaction(tx: <Block as BlockT>::Extrinsic) -> TransactionValidity {
Executive::validate_transaction(tx)
}
}
impl consensus_aura::AuraApi<Block> for Runtime {
fn slot_duration() -> u64 {
Aura::slot_duration()
}
}
impl offchain_primitives::OffchainWorkerApi<Block> for Runtime {
fn offchain_worker(n: NumberFor<Block>) {
Executive::offchain_worker(n)
}
}
impl consensus_authorities::AuthoritiesApi<Block> for Runtime {
fn authorities() -> Vec<AuthorityId> {
Consensus::authorities()
}
}
}
Runtime Module Template
Module’s configutation trait, storage, declartion and event.
/// A runtime module template with necessary imports
/// Feel free to remove or edit this file as needed.
/// If you change the name of this file, make sure to update its references in runtime/src/lib.rs
/// If you remove this file, you can remove those references
/// For more guidance on Substrate modules, see the example module
/// https://github.com/paritytech/substrate/blob/master/srml/example/src/lib.rs
use support::{decl_module, decl_storage, decl_event, StorageValue, dispatch::Result};
use system::ensure_signed;
/// The module's configuration trait.
pub trait Trait: system::Trait {
// TODO: Add other types and constants required configure this module.
/// The overarching event type.
type Event: From<Event<Self>> + Into<<Self as system::Trait>::Event>;
}
/// This module's storage items.
decl_storage! {
trait Store for Module<T: Trait> as TemplateModule {
// Just a dummy storage item.
// Here we are declaring a StorageValue, `Something` as a Option<u32>
// `get(something)` is the default getter which returns either the stored `u32` or `None` if nothing stored
Something get(something): Option<u32>;
}
}
decl_module! {
/// The module declaration.
pub struct Module<T: Trait> for enum Call where origin: T::Origin {
// Initializing events
// this is needed only if you are using events in your module
fn deposit_event<T>() = default;
// Just a dummy entry point.
// function that can be called by the external world as an extrinsics call
// takes a parameter of the type `AccountId`, stores it and emits an event
pub fn do_something(origin, something: u32) -> Result {
// TODO: You only need this if you want to check it was signed.
let who = ensure_signed(origin)?;
// TODO: Code to execute when something calls this.
// For example: the following line stores the passed in u32 in the storage
<Something<T>>::put(something);
// here we are raising the Something event
Self::deposit_event(RawEvent::SomethingStored(something, who));
Ok(())
}
}
}
decl_event!(
pub enum Event<T> where AccountId = <T as system::Trait>::AccountId {
// Just a dummy event.
// Event `Something` is declared with a parameter of the type `u32` and `AccountId`
// To emit this event, we call the deposit function, from our runtime funtions
SomethingStored(u32, AccountId),
}
);
Runtime Module Key Value Example
This is simple runtime module that stores a key value map.
use srml_support::{StorageMap, dispatch::Result};
pub trait Trait: system::Trait {}
decl_module! {
pub struct Module<T: Trait> for enum Call where origin: T::Origin {
fn set_mapping(_origin, key: u32, value: u32) -> Result {
<Value<T>>::insert(key, value);
Ok(())
}
}
}
decl_storage! {
trait Store for Module<T: Trait> as RuntimeExampleStorage {
Value: map u32 => u32;
}
}
Libra
Networking level and substrate based on libp2p core.
Substrate based on:
use libp2p::core::
Libra based on:
parity_multiaddr::Multiaddr;
fn listen_on(&self, addr: Multiaddr) -> Result<(Self::Listener, Multiaddr)
– chain level
I was able to get Libra working on my Linux Subsystem using the following commands:
git clone https://github.com/libra/libra
cd libra
./scripts/dev_setup.sh
source /home/<user>/.cargo/env
./scripts/cli/start_cli_testnet.sh
Libra Blockchain API
account_state_sets
fn put_account_state_set(
store: &StateStore,
account_state_set: Vec<(AccountAddress, AccountStateBlob)>,
version: Version,
root_hash: HashValue,
expected_nodes_created: usize,
expected_nodes_retired: usize,
expected_blobs_retired: usize,
) -> HashValue
use rocksdb
pub type Read Options = rocksdb::ReadOptions;
A simple tx script in Move IR that transfers a coin from address to another:
public main(payee: address, amount: u64) {
let coin: 0x0.Currency.Coin = 0x0
.Currency
.withdraw_from_sender(copy(amount));
0x0.Currency.deposit(copy(payee), move(coin));
}
Oasis: Blockchain and WASM
By leveraging compiler support and tools built for Wasm and WASI, the blockchain becomes a powerful tool for high-integrity — and even confidential — general-purpose “cloud” computation.
use oasis_std::Context;
use statesets_std::Context;
#[derive(oasis_std::Service)]
struct Quickstart;
impl Quickstart {
pub fn new(_ctx: &Context) -> Self {
Self
}
pub fn say_hello(&mut self, ctx: &Context) -> String {
format!("Hello, {}!", ctx.sender())
}
}
fn main() {
oasis_std::service!(Quickstart);
}
#[cfg(test)]
mod tests {
extern crate oasis_test;
use super::*;
#[test]
fn test() {
let sender = oasis_test::create_account(1);
let ctx = Context::default().with_sender(sender);
let mut client = Quickstart::new(&ctx);
println!("{}", client.say_hello(&ctx));
}
}
From https://medium.com/oasislabs/blockchain-flavored-wasi-50e3612b8eba:
Notes on struct, impl, and pub fn.
Becomes one main.rs file as opposed to separated. The compiler checks and the gives wasm blog which is deployed on chain with rpc endpoints getting, changing and setting states.
define and implement Oasis service RPCs in Rust.
fn main() { oasis_std::service!(Ballot); }
Compiles to WASM. Deploy to the platform and setup client to call it.
use oasis_std::Context;
use map_vec::Map; use oasis_std::{Address, Context};
[derive(oasis_std::Service)]
derive [Serialize, Deserialize, Service]
pub struct X {
fields: <String>,
''
}
pub fn new(ctx: &Context, description: String, candidates: Vec<String>) -> Self { Ok(Self { description, tally: vec![0; candidates.len()], candidates, accepting_votes: true, admin: ctx.sender(), voters: Map::new(), }) }
/// Returns the candidates being voted upon.
pub fn candidates(&self, _ctx: &Context) -> Vec<&str> { self.candidates.iter().map(String::as_ref).collect() }
/// Returns the description of this ballot.
pub fn description(&self, _ctx: &Context) -> &str { &self.description }
/// Returns whether voting is still open. pub fn voting_open(&self, _ctx: &Context) -> bool { self.accepting_votes }
we have access to the state of the service, as provided by a reference to self.
For state changes:
you’ll see that &self has changed to &mut self, but this is just Rust’s way to know that you want a mutable reference.
Then define Getter Fns to Get State from the Stores
similar to the hyperledger composer model
define a data model in cto file declare assets, transactions and events
define javascript functions for logic with refernece to namespace is same as impl with struct (service state object) RPCs that are defined as pub fn’s.
instead of seperate REST API to make calls to on the model.
A service is created on chain with RPC endpoints.
Clients can call the service endpoint and add listeners for events from the service.
Blockchain WASI
While cloud computing has long brought cost and ease of use, switching from an on-prem solution to cloud has traditionally come with its own inherent risks including a degradation in security and a lack of auditability. These are areas that have the potential to be solved with new emerging technologies including Web Assembly, the Web Assembly System Interface, and blockchain.
We propose a mechanism for trustworthy, uncensorable, and autonomous cloud computation based on the combination of three emerging technologies: Web Assembly, the Web Assembly System Interface, and blockchain.
Oasis Network
The Oasis Network uses 3 main protocols for communication:
Confidentiality is achieved in the Oasis Network by relying on trusted execution environments (TEEs) to secure the execution of any given smart contract. Initially, the Oasis Network will utilize Intel SGX. As more TEE technologies mature, we expect to support more than TEEs than Intel SGX.
from https://docs.oasis.dev/operators/architectural-overview.html#modular-architecture
libp2p
lib-p2p is a modularized and extensible network stack to overcome the networking challenges faced when doing peer-to-peer applications. It is very well defined, composable, swappable; a modular system of protocols, specifications, and libraries that enable the development of peer-to-peer network applications.
It ultimately is a collection of peer-to-peer protocols for finding peers, connecting to them for finding content, and transferring it to them.
Here is a sample ping example from rust-libp2p:
use futures::{prelude::*, future};
use libp2p::{ identity, PeerId, ping::{Ping, PingConfig}, Swarm };
use std::env;
fn main() {
env_logger::init();
// Create a random PeerId.
let id_keys = identity::Keypair::generate_ed25519();
let peer_id = PeerId::from(id_keys.public());
println!("Local peer id: {:?}", peer_id);
// Create a transport.
let transport = libp2p::build_development_transport(id_keys);
// Create a ping network behaviour.
//
// For illustrative purposes, the ping protocol is configured to
// keep the connection alive, so a continuous sequence of pings
// can be observed.
let behaviour = Ping::new(PingConfig::new().with_keep_alive(true));
// Create a Swarm that establishes connections through the given transport
// and applies the ping behaviour on each connection.
let mut swarm = Swarm::new(transport, behaviour, peer_id);
// Dial the peer identified by the multi-address given as the second
// command-line argument, if any.
if let Some(addr) = env::args().nth(1) {
let remote_addr = addr.clone();
match addr.parse() {
Ok(remote) => {
match Swarm::dial_addr(&mut swarm, remote) {
Ok(()) => println!("Dialed {:?}", remote_addr),
Err(e) => println!("Dialing {:?} failed with: {:?}", remote_addr, e)
}
},
Err(err) => println!("Failed to parse address to dial: {:?}", err),
}
}
// Tell the swarm to listen on all interfaces and a random, OS-assigned port.
Swarm::listen_on(&mut swarm, "/ip4/0.0.0.0/tcp/0".parse().unwrap()).unwrap();
// Use tokio to drive the `Swarm`.
let mut listening = false;
tokio::run(future::poll_fn(move || -> Result<_, ()> {
loop {
match swarm.poll().expect("Error while polling swarm") {
Async::Ready(Some(e)) => println!("{:?}", e),
Async::Ready(None) | Async::NotReady => {
if !listening {
if let Some(a) = Swarm::listeners(&swarm).next() {
println!("Listening on {:?}", a);
listening = true;
}
}
return Ok(Async::NotReady)
}
}
}
}));
}
