Welcome to Fabric¶

Asset Transfer with SDK¶

The individual javascript programs will exercise the SDK APIs to register and enroll the client with the provisioned Certificate Authority. Once the client is properly authenticated, the programs will demonstrate basic chaincode functionalities - deploy, invoke, and query. Make sure you are in the working directory where you pulled the source code before proceeding.

Upon success of each node program, you will receive a “200” response in the terminal.

Register/enroll & deploy chaincode (Linux or OSX):

# Deploy initializes key value pairs of "a","100" & "b","200".
GOPATH=$PWD node deploy.js

Register/enroll & deploy chaincode (Windows):

# Deploy initializes key value pairs of "a","100" & "b","200".
SET GOPATH=%cd%
node deploy.js

Issue an invoke. Move units 100 from “a” to “b”:

node invoke.js

Query against key value “b”:

# this should return a value of 300
node query.js

Explore the various node.js programs, along with example_cc.go to better understand the SDK and APIs.

Asset transfer with CLI¶

Use the cli container to manually exercise the create channel and join channel APIs.

Channel - myc1 already exists, so let’s create a new channel named myc2.

Exec into the cli container:

docker exec -it cli bash

If successful, you should see the following in your terminal:

/opt/gopath/src/github.com/hyperledger/fabric/peer #

Send createChannel API to Ordering Service:

CORE_PEER_COMMITTER_LEDGER_ORDERER=orderer:7050 peer channel create -c myc2

This will return a genesis block - myc2.block - that you can issue join commands with. Next, send a joinChannel API to peer0 and pass in the genesis block as an argument. The channel is defined within the genesis block:

CORE_PEER_COMMITTER_LEDGER_ORDERER=orderer:7050 CORE_PEER_ADDRESS=peer0:7051 peer channel join -b myc2.block

To join the other peers to the channel, simply reissue the above command with peer1 or peer2 specified. For example:

CORE_PEER_COMMITTER_LEDGER_ORDERER=orderer:7050 CORE_PEER_ADDRESS=peer1:7051 peer channel join -b myc2.block

Once the peers have all joined the channel, you are able to issues queries against any peer without having to deploy chaincode to each of them.

Deploy, invoke and query

Run the deploy command. This command is deploying a chaincode named mycc to peer0 on the Channel ID myc2. The constructor message is initializing a and b with values of 100 and 200 respectively.

CORE_PEER_ADDRESS=peer0:7051 CORE_PEER_COMMITTER_LEDGER_ORDERER=orderer:7050 peer chaincode deploy -C myc2 -n mycc -p github.com/hyperledger/fabric/examples -c '{"Args":["init","a","100","b","200"]}'

Run the invoke command. This invocation is moving 10 units from a to b.

CORE_PEER_ADDRESS=peer0:7051 CORE_PEER_COMMITTER_LEDGER_ORDERER=orderer:7050 peer chaincode invoke -C myc2 -n mycc -c '{"function":"invoke","Args":["move","a","b","10"]}'

Run the query command. The invocation transferred 10 units from a to b, therefore a query against a should return the value 90.

CORE_PEER_ADDRESS=peer0:7051 CORE_PEER_COMMITTER_LEDGER_ORDERER=orderer:7050 peer chaincode query -C myc2 -n mycc -c '{"function":"invoke","Args":["query","a"]}'

You can issue an exit command at any time to exit the cli container.

Create the initial channel

If you want to manually create the initial channel through the cli container, you will need to edit the Docker Compose file. Use an editor to open docker-compose-gettingstarted.yml and comment out the channel_test.sh command in your cli image. Simply place a # to the left of the command. (Recall that this script is executing the create and join channel APIs when you run docker-compose up) For example:

cli:
  container_name: cli
  <CONTENT REMOVED FOR BREVITY>
  working_dir: /opt/gopath/src/github.com/hyperledger/fabric/peer
#  command: sh -c './channel_test.sh; sleep 1000'
#  command: /bin/sh

Then use the cli commands from above.

Troubleshooting¶

If you have existing containers running, you may receive an error indicating that a port is already occupied. If this occurs, you will need to kill the container that is using said port.

If a file cannot be located, make sure your curl commands executed successfully and make sure you are in the directory where you pulled the source code.

If you are receiving timeout or GRPC communication errors, make sure you have the correct version of Docker installed - v1.13.0. Then try restarting your failing docker process. For example:

docker stop peer0

Then:

docker start peer0

Another approach to GRPC and DNS errors (peer failing to resolve with orderer and vice versa) is to hardcode the IP addresses for each. You will know if there is a DNS issue, because a more results.txt command within the cli container will display something similar to:

ERROR CREATING CHANNEL
PEER0 ERROR JOINING CHANNEL

Issue a docker inspect <container_name> to ascertain the IP address. For example:

docker inspect peer0 | grep IPAddress

AND

docker inspect orderer | grep IPAddress

Take these values and hard code them into your cli commands. For example:

CORE_PEER_COMMITTER_LEDGER_ORDERER=172.21.0.2:7050 peer channel create -c myc1

AND THEN

CORE_PEER_COMMITTER_LEDGER_ORDERER=<IP_ADDRESS> CORE_PEER_ADDRESS=<IP_ADDRESS> peer channel join -b myc1.block

If you are seeing errors while using the node SDK, make sure you have the correct versions of node.js and npm installed on your machine. You want node v6.9.5 and npm v3.10.10.

If you ran through the automated channel create/join process (i.e. did not comment out channel_test.sh in the docker-compose-gettingstarted.yml), then channel - myc1 - and genesis block - myc1.block - have already been created and exist on your machine. As a result, if you proceed to execute the manual steps in your cli container:

CORE_PEER_COMMITTER_LEDGER_ORDERER=orderer:7050 peer channel create -c myc1

Then you will run into an error similar to:

<EXACT_TIMESTAMP>       UTC [msp] Sign -> DEBU 064 Sign: digest: 5ABA6805B3CDBAF16C6D0DCD6DC439F92793D55C82DB130206E35791BCF18E5F
Error: Got unexpected status: BAD_REQUEST
Usage:
  peer channel create [flags]

This occurs because you are attempting to create a channel named myc1, and this channel already exists! There are two options. Try issuing the peer channel create command with a different channel name - myc2. For example:

CORE_PEER_COMMITTER_LEDGER_ORDERER=orderer:7050 peer channel create -c myc2

Then join:

CORE_PEER_COMMITTER_LEDGER_ORDERER=orderer:7050 CORE_PEER_ADDRESS=peer0:7051 peer channel join -b myc2.block

If you do choose to create a new channel, and want to run deploy/invoke/query with the node.js programs, you also need to edit the “channelID” parameter in the config.json file to match the new channel’s name. For example:

{
   "chainName":"fabric-client1",
   "chaincodeID":"mycc",
   "channelID":"myc2",
   "goPath":"../../test/fixtures",
   "chaincodePath":"github.com/example_cc",

OR, if you want your channel called - myc1 -, remove your docker containers and then follow the same commands in the Manually create and join peers to a new channel topic.

Clean up

Shut down your containers:

docker-compose -f docker-compose-gettingstarted.yml down

Helpful Docker tips

Remove a specific docker container:

docker rm <containerID>

Force removal:

docker rm -f <containerID>

Remove all docker containers:

docker rm -f $(docker ps -aq)

This will merely kill docker containers (i.e. stop the process). You will not lose any images.

Remove an image:

docker rmi <imageID>

Forcibly remove:

docker rmi -f <imageID>

Remove all images:

docker rmi -f $(docker images -q)

Assets¶

Assets can range from the tangible (real estate and hardware) to the intangible (contracts and intellectual property). You can easily define Assets in client-side javascript and use them in your Fabric application using the included Fabric Composer tool.

Fabric supports the ability to exchange assets using unspent transaction outputs as the inputs for subsequent transactions. Assets (and asset registries) live in Fabric as a collection of key-value pairs, with state changes recorded as transactions on a Channel ledger. Fabric allows for any asset to be represented in binary or JSON format.

Chaincode¶

Chaincode is software defining an asset or assets, and the transaction instructions for modifying the asset(s). In other words, it’s the business logic. Chaincode enforces the rules for reading or altering key value pairs or other state database information. Chaincode functions execute against the ledger current state database and are initiated through a transaction proposal. Chaincode execution results in a set of key value writes (write set) that can be submitted to the network and applied to the ledger on all peers.

Ledger Features¶

The ledger is the sequenced, tamper-resistant record of all state transitions in the fabric. State transitions are a result of chaincode invocations (‘transactions’) submitted by participating parties. Each transaction results in a set of asset key-value pairs that are committed to the ledger as creates, updates, or deletes.

The ledger is comprised of a blockchain (‘chain’) to store the immutable, sequenced record in blocks, as well as a state database to maintain current fabric state. There is one ledger per channel. Each peer maintains a copy of the ledger for each channel of which they are a member.

• Query and update ledger using key-based lookups, range queries, and composite key queries
• Read-only queries using a rich query language (if using CouchDB as state database)
• Read-only history queries - Query ledger history for a key, enabling data provenance scenarios
• Transactions consist of the versions of keys/values that were read in chaincode (read set) and keys/values that were written in chaincode (write set)
• Transactions contain signatures of every endorsing peer and are submitted to ordering service
• Transactions are ordered into blocks and are “delivered” from an ordering service to peers on a channel
• Peers validate transactions against endorsement policies and enforce the policies
• Prior to appending a block, a versioning check is performed to ensure that states for assets that were read have not changed since chaincode execution time
• There is immutability once a transaction is validated and committed
• A channel’s ledger contains a configuration block defining policies, access control lists, and other pertinent information
• Channel’s contain :ref:`MSP`s allowing crypto materials to be derived from different certificate authorities

See the Ledger topic for a deeper dive on the databases, storage structure, and “query-ability.”

Privacy through Channels¶

Fabric employs an immutable ledger on a per-channel basis, as well as chaincodes that can manipulate and modify the current state of assets (i.e. update key value pairs). A ledger exists in the scope of a channel - it can be shared across the entire network (assuming every participant is operating on one common channel) - or it can be privatized to only include a specific set of participants.

In the latter scenario, these participants would create a separate channel and thereby isolate/segregate their transactions and ledger. Fabric even solves scenarios that want to bridge the gap between total transparency and privacy. Chaincode gets installed only on peers that need to access the asset states to perform reads and writes (in other words, if a chaincode is not installed on a peer, it will not be able to properly interface with the ledger). To further obfuscate the data, values within chaincode can be encrypted (in part or in total) using common cryptographic algorithms such as SHA0-256, etc. before appending to the ledger.

Security & Membership Services¶

Hyperledger Fabric underpins a transactional network where all participants have known identities. Public Key Infrastructure is used to generate cryptographic certificates which are tied to organizations, network components, and end users or client applications. As a result, data access control can be manipulated and governed on the broader network and on channel levels. This “permissioned” notion of Fabric, coupled with the existence and capabilities of channels, helps address scenarios where privacy and confidentiality are paramount concerns.

See the Fabric CA section to better understand cryptographic implementations, and the sign, verify, authenticate approach used in Fabric.

Consensus¶

In distributed ledger technology, consensus has recently become synonymous with a specific algorithm, within a single function. However, consensus encompasses more than simply agreeing upon the order of transactions, and this differentiation is highlighted in Hyperledger Fabric through its fundamental role in the entire transaction flow, from proposal and endorsement, to ordering, validation and commitment. In a nutshell, consensus is defined as the full-circle verification of the correctness of a set of transactions comprising a block.

Consensus is ultimately achieved when the order and results of a block’s transactions have met the explicit policy criteria checks. These checks and balances take place during the lifecycle of a transaction, and include the usage of endorsement policies to dictate which specific members must endorse a certain transaction class, as well as system chaincodes to ensure that these policies are enforced and upheld. Prior to commitment, the peers will employ these system chaincodes to make sure that enough endorsements are present, and that they were derived from the appropriate entities. Moreover, a versioning check will take place during which the current state of the ledger is agreed or consented upon, before any blocks containing transactions are appended to the ledger. This final check provides protection against double spend operations and other threats that might compromise data integrity, and allows for functions to be executed against non-static variables.

In addition to the multitude of endorsement, validity and versioning checks that take place, there are also ongoing identity verifications happening in all directions of the transaction flow. Access control lists are implemented on hierarchal layers of the network (ordering service down to channels), and payloads are repeatedly signed, verified and authenticated as a transaction proposal passes through the different architectural components. To conclude, consensus is not merely limited to the agreed upon order of a batch of transactions, but rather, it is an overarching characterization that is achieved as a byproduct of the ongoing verifications that take place during a transaction’s journey from proposal to commitment.

Check out the Transaction Flow diagram for a visual representation of consensus.

B2B Contract¶

Business contracts can be codified to allow two or more parties to automate contractual agreements in a trusted way. Although information on blockchain is naturally “public”, B2B contracts may require privacy control to protect sensitive business information from being disclosed to outside parties that also have access to the ledger.

While confidential agreements are a key business case, there are many scenarios where contracts can and should be easily discoverable by all parties on a ledger. For example, a ledger used to create offers (asks) seeking bids, by definition, requires access without restriction. This type of contract may need to be standardized so that bidders can easily find them, effectively creating an electronic trading platform with smart contracts (aka chaincode).

Manufacturing Supply Chain¶

Final assemblers, such as automobile manufacturers, can create a supply chain network managed by its peers and suppliers so that a final assembler can better manage its suppliers and be more responsive to events that would require vehicle recalls (possibly triggered by faulty parts provided by a supplier). The blockchain fabric must provide a standard protocol to allow every participant on a supply chain network to input and track numbered parts that are produced and used on a specific vehicle.

Why is this specific example an abstract use case? Because while all blockchain cases store immutable information, and some add the need for transfer of assets between parties, this case emphasizes the need to provide deep searchability backwards through as many as 5-10 transaction layers. This backwards search capability is the core of establishing provenance of any manufactured good that is made up of other component goods and supplies.

Asset Depository¶

Assets such as financial securities must be able to be dematerialized on a blockchain network so that all stakeholders of an asset type will have direct access to that asset, allowing them to initiate trades and acquire information on an asset without going through layers of intermediaries. Trades should be settled in near real time and all stakeholders must be able to access asset information in near real time. A stakeholder should be able to add business rules on any given asset type, as one example of using automation logic to further reduce operating costs.

One Trade, One Contract¶

From the time that a trade is captured by the front office until the trade is finally settled, only one contract that specifies the trade will be created and used by all participants. The middle office will enrich the same electronic contract submitted by the front office, and that same contract will then be used by counter parties to confirm and affirm the trade. Finally, securities depository will settle the trade by executing the trading instructions specified on the contract. When dealing with bulk trades, the original contract can be broken down into sub-contracts that are always linked to the original parent contract.

Direct Communication¶

Company A announces its intention to raise 2 Billion USD by way of rights issue. Because this is a voluntary action, Company A needs to ensure that complete details of the offer are sent to shareholders in real time, regardless of how many intermediaries are involved in the process (such as receiving/paying agents, CSD, ICSD, local/global custodian banks, asset management firms, etc). Once a shareholder has made a decision, that decision will also be processed and settled (including the new issuance of shares) in real time. If a shareholder sold its rights to a third party, the securities depository must be able to record the new shares under the name of their new rightful owner.

Separation of Asset Ownership and Custodian’s Duties¶

Assets should always be owned by their actual owners, and asset owners must be able to allow third-party professionals to manage their assets without having to pass legal ownership of assets to third parties (such as nominee or street name entities). If issuers need to send messages or payments to asset owners (for example, listed share holders), issuers send them directly to asset owners. Third-party asset managers and/or custodians can always buy, sell, and lend assets on behalf of their owners. Under this arrangement, asset custodians can focus on providing value-add services to shareowners, without worrying about asset ownership duties such as managing and redirecting payments from issuers to shareowners.

Interoperability of Assets¶

If an organization requires 20,000 units of asset B, but instead owns 10,000 units of asset A, it needs a way to exchange asset A for asset B. Though the current market might not offer enough liquidity to fulfill this trade quickly, there might be plenty of liquidity available between asset A and asset C, and also between asset C and asset B. Instead of settling for market limits on direct trading (A for B) in this case, a chain network connects buyers with “buried” sellers, finds the best match (which could be buried under several layers of assets), and executes the transaction.

Art Auction¶

[WIP] ...coming soon

Shows the provenance, attestation, and ownership of a piece of artwork and the ensuing interaction of the various stakeholders. Not yet stable with v1 codebase.

Learn more about the components here

Learn more about the client-side application here

Marbles¶

[WIP] ...coming soon

The marbles chaincode application demonstrates the ability to create assets (marbles) with unique attributes - size, color, owner, etc... and trade these assets with fellow participants in a blockchain network. It is not yet stable with v1 codebase.

Learn more about the marbles chaincode and client-side application here

Overview¶

Logging in the peer application and in the shim interface to chaincodes is programmed using facilities provided by the github.com/op/go-logging package. This package supports

• Logging control based on the severity of the message
• Logging control based on the software module generating the message
• Different pretty-printing options based on the severity of the message

All logs are currently directed to stderr, and the pretty-printing is currently fixed. However global and module-level control of logging by severity is provided for both users and developers. There are currently no formalized rules for the types of information provided at each severity level, however when submitting bug reports the developers may want to see full logs down to the DEBUG level.

In pretty-printed logs the logging level is indicated both by color and by a 4-character code, e.g, “ERRO” for ERROR, “DEBU” for DEBUG, etc. In the logging context a module is an arbitrary name (string) given by developers to groups of related messages. In the pretty-printed example below, the logging modules “peer”, “rest” and “main” are generating logs.

16:47:09.634 [peer] GetLocalAddress -> INFO 033 Auto detected peer address: 9.3.158.178:7051
16:47:09.635 [rest] StartOpenchainRESTServer -> INFO 035 Initializing the REST service...
16:47:09.635 [main] serve -> INFO 036 Starting peer with id=name:"vp1" , network id=dev, address=9.3.158.178:7051, discovery.rootnode=, validator=true

An arbitrary number of logging modules can be created at runtime, therefore there is no “master list” of modules, and logging control constructs can not check whether logging modules actually do or will exist. Also note that the logging module system does not understand hierarchy or wildcarding: You may see module names like “foo/bar” in the code, but the logging system only sees a flat string. It doesn’t understand that “foo/bar” is related to “foo” in any way, or that “foo/*” might indicate all “submodules” of foo.

peer¶

The logging level of the peer command can be controlled from the command line for each invocation using the --logging-level flag, for example

peer node start --logging-level=debug

The default logging level for each individual peer subcommand can also be set in the core.yaml file. For example the key logging.node sets the default level for the node subcommmand. Comments in the file also explain how the logging level can be overridden in various ways by using environment varaibles.

Logging severity levels are specified using case-insensitive strings chosen from

CRITICAL | ERROR | WARNING | NOTICE | INFO | DEBUG

The full logging level specification for the peer is of the form

[<module>[,<module>...]=]<level>[:[<module>[,<module>...]=]<level>...]

A logging level by itself is taken as the overall default. Otherwise, overrides for individual or groups of modules can be specified using the

<module>[,<module>...]=<level>

syntax. Examples of specifications (valid for all of --logging-level, environment variable and core.yaml settings):

info                                       - Set default to INFO
warning:main,db=debug:chaincode=info       - Default WARNING; Override for main,db,chaincode
chaincode=info:main=debug:db=debug:warning - Same as above

Go chaincodes¶

The standard mechanism to log within a chaincode application is to integrate with the logging transport exposed to each chaincode instance via the peer. The chaincode shim package provides APIs that allow a chaincode to create and manage logging objects whose logs will be formatted and interleaved consistently with the shim logs.

As independently executed programs, user-provided chaincodes may technically also produce output on stdout/stderr. While naturally useful for “devmode”, these channels are normally disabled on a production network to mitigate abuse from broken or malicious code. However, it is possible to enable this output even for peer-managed containers (e.g. “netmode”) on a per-peer basis via the CORE_VM_DOCKER_ATTACHSTDOUT=true configuration option.

Once enabled, each chaincode will receive its own logging channel keyed by its container-id. Any output written to either stdout or stderr will be integrated with the peer’s log on a per-line basis. It is not recommended to enable this for production.

LogDebug, LogInfo, LogNotice, LogWarning, LogError, LogCritical

DEBUG, INFO, NOTICE, WARNING, ERROR, CRITICAL

(c *ChaincodeLogger) Debug(args ...interface{})
(c *ChaincodeLogger) Info(args ...interface{})
(c *ChaincodeLogger) Notice(args ...interface{})
(c *ChaincodeLogger) Warning(args ...interface{})
(c *ChaincodeLogger) Error(args ...interface{})
(c *ChaincodeLogger) Critical(args ...interface{})

(c *ChaincodeLogger) Debugf(format string, args ...interface{})
(c *ChaincodeLogger) Infof(format string, args ...interface{})
(c *ChaincodeLogger) Noticef(format string, args ...interface{})
(c *ChaincodeLogger) Warningf(format string, args ...interface{})
(c *ChaincodeLogger) Errorf(format string, args ...interface{})
(c *ChaincodeLogger) Criticalf(format string, args ...interface{})

var logger = shim.NewLogger("myChaincode")

func main() {

    logger.SetLevel(shim.LogInfo)

    logLevel, _ := shim.LogLevel(os.Getenv("SHIM_LOGGING_LEVEL"))
    shim.SetLoggingLevel(logLevel)
    ...
}

1. System architecture¶

The blockchain is a distributed system consisting of many nodes that communicate with each other. The blockchain runs programs called chaincode, holds state and ledger data, and executes transactions. The chaincode is the central element as transactions are operations invoked on the chaincode. Transactions have to be “endorsed” and only endorsed transactions may be committed and have an effect on the state. There may exist one or more special chaincodes for management functions and parameters, collectively called system chaincodes.

2. Basic workflow of transaction endorsement¶

In the following we outline the high-level request flow for a transaction.

Remark: Notice that the following protocol *does not assume that all transactions are deterministic, i.e., it allows for non-deterministic transactions.*

2.4. The ordering service delivers a transactions to the peers¶

When an event deliver(seqno, prevhash, blob) occurs and a peer has applied all state updates for blobs with sequence number lower than seqno, a peer does the following:

• It checks that the blob.endorsement is valid according to the policy of the chaincode (blob.tran-proposal.chaincodeID) to which it refers.
• In a typical case, it also verifies that the dependencies (blob.endorsement.tran-proposal.readset) have not been violated meanwhile. In more complex use cases, tran-proposal fields in endorsement may differ and in this case endorsement policy (Section 3) specifies how the state evolves.

Verification of dependencies can be implemented in different ways, according to a consistency property or “isolation guarantee” that is chosen for the state updates. Serializability is a default isolation guarantee, unless chaincode endorsement policy specifies a different one. Serializability can be provided by requiring the version associated with every key in the readset to be equal to that key’s version in the state, and rejecting transactions that do not satisfy this requirement.

• If all these checks pass, the transaction is deemed valid or committed. In this case, the peer marks the transaction with 1 in the bitmask of the PeerLedger, applies blob.endorsement.tran-proposal.writeset to blockchain state (if tran-proposals are the same, otherwise endorsement policy logic defines the function that takes blob.endorsement).
• If the endorsement policy verification of blob.endorsement fails, the transaction is invalid and the peer marks the transaction with 0 in the bitmask of the PeerLedger. It is important to note that invalid transactions do not change the state.

Note that this is sufficient to have all (correct) peers have the same state after processing a deliver event (block) with a given sequence number. Namely, by the guarantees of the ordering service, all correct peers will receive an identical sequence of deliver(seqno, prevhash, blob) events. As the evaluation of the endorsement policy and evaluation of version dependencies in readset are deterministic, all correct peers will also come to the same conclusion whether a transaction contained in a blob is valid. Hence, all peers commit and apply the same sequence of transactions and update their state in the same way.

Illustration of the transaction flow (common-case path).

Figure 1. Illustration of one possible transaction flow (common-case path).

3. Endorsement policies¶

4 (post-v1). Validated ledger and PeerLedger checkpointing (pruning)¶

Table of Contents¶

1. Overview
2. Getting Started
   1. Prerequisites
   2. Install
   3. Explore the Fabric CA CLI
3. File Formats
   1. Fabric CA server’s configuration file format
   2. Fabric CA client’s configuration file format
4. Configuration Settings Precedence
5. Fabric CA Server
   1. Initializing the server
   2. Starting the server
   3. Configuring the database
   4. Configuring LDAP
   5. Setting up a cluster
6. Fabric CA Client
   1. Enrolling the bootstrap user
   2. Registering a new identity
   3. Enrolling a peer identity
   4. Reenrolling an identity
   5. Revoking a certificate or identity
   6. Enabling TLS
7. Appendix

Overview¶

The diagram below illustrates how the Fabric CA server fits into the overall Hyperledger Fabric architecture.

There are two ways of interacting with a Fabric CA server: via the Fabric CA client or through one of the Fabric SDKs. All communication to the Fabric CA server is via REST APIs. See fabric-ca/swagger/swagger-fabric-ca.json for the swagger documentation for these REST APIs.

The Fabric CA client or SDK may connect to a server in a cluster of Fabric CA servers. This is illustrated in the top right section of the diagram. The client routes to an HA Proxy endpoint which load balances traffic to one of the fabric-ca-server cluster members. All Fabric CA servers in a cluster share the same database for keeping track of users and certificates. If LDAP is configured, the user information is kept in LDAP rather than the database.

Getting Started¶

# sudo apt install libtool libltdl-dev

# go get -u github.com/hyperledger/fabric-ca/cmd/...

# fabric-ca-server start -b admin:adminpw

# cd $GOPATH/src/github.com/hyperledger/fabric-ca
# make docker
# cd docker/server
# docker-compose up -d

Hyperledger Fabric Certificate Authority Server

Usage:
  fabric-ca-server [command]

Available Commands:
  init        Initialize the fabric-ca server
  start       Start the fabric-ca server

Flags:
      --address string                  Listening address of fabric-ca-server (default "0.0.0.0")
  -b, --boot string                     The user:pass for bootstrap admin which is required to build default config file
      --ca.certfile string              PEM-encoded CA certificate file (default "ca-cert.pem")
      --ca.keyfile string               PEM-encoded CA key file (default "ca-key.pem")
  -c, --config string                   Configuration file (default "fabric-ca-server-config.yaml")
      --csr.cn string                   The common name field of the certificate signing request to a parent fabric-ca-server
      --csr.serialnumber string         The serial number in a certificate signing request to a parent fabric-ca-server
      --db.datasource string            Data source which is database specific (default "fabric-ca-server.db")
      --db.tls.certfiles string         PEM-encoded comma separated list of trusted certificate files (e.g. root1.pem, root2.pem)
      --db.tls.client.certfile string   PEM-encoded certificate file when mutual authenticate is enabled
      --db.tls.client.keyfile string    PEM-encoded key file when mutual authentication is enabled
      --db.tls.enabled                  Enable TLS for client connection
      --db.type string                  Type of database; one of: sqlite3, postgres, mysql (default "sqlite3")
  -d, --debug                           Enable debug level logging
      --ldap.enabled                    Enable the LDAP client for authentication and attributes
      --ldap.groupfilter string         The LDAP group filter for a single affiliation group (default "(memberUid=%s)")
      --ldap.url string                 LDAP client URL of form ldap://adminDN:adminPassword@host[:port]/base
      --ldap.userfilter string          The LDAP user filter to use when searching for users (default "(uid=%s)")
  -p, --port int                        Listening port of fabric-ca-server (default 7054)
      --registry.maxenrollments int     Maximum number of enrollments; valid if LDAP not enabled
      --tls.certfile string             PEM-encoded TLS certificate file for server's listening port (default "ca-cert.pem")
      --tls.enabled                     Enable TLS on the listening port
      --tls.keyfile string              PEM-encoded TLS key for server's listening port (default "ca-key.pem")
  -u, --url string                      URL of the parent fabric-ca-server

Use "fabric-ca-server [command] --help" for more information about a command.

# fabric-ca-client
Hyperledger Fabric Certificate Authority Client

Usage:
  fabric-ca-client [command]

Available Commands:
  enroll      Enroll user
  reenroll    Reenroll user
  register    Register user
  revoke      Revoke user

Flags:
  -c, --config string                Configuration file (default "/Users/saadkarim/.fabric-ca-client/fabric-ca-client-config.yaml")
      --csr.cn string                The common name field of the certificate signing request to a parent fabric-ca-server
      --csr.serialnumber string      The serial number in a certificate signing request to a parent fabric-ca-server
  -d, --debug                        Enable debug logging
      --enrollment.hosts string      Comma-separated host list
      --enrollment.label string      Label to use in HSM operations
      --enrollment.profile string    Name of the signing profile to use in issuing the certificate
      --id.affiliation string        Name associated with the identity
      --id.attr string               Attributes associated with this identity (e.g. hf.revoker=true)
      --id.maxenrollments int        MaxEnrollments is the maximum number of times the secret can be reused to enroll.
      --id.name string               Unique name of the identity
      --id.secret string             Secret is an optional password. If not specified, a random secret is generated.
      --id.type string               Type of identity being registered (e.g. 'peer, app, user')
  -m, --myhost string                Hostname to include in the certificate signing request during enrollment (default "saads-mbp.raleigh.ibm.com")
      --tls.certfiles string         PEM-encoded comma separated list of trusted certificate files (e.g. root1.pem, root2.pem)
      --tls.client.certfile string   PEM-encoded certificate file when mutual authenticate is enabled
      --tls.client.keyfile string    PEM-encoded key file when mutual authentication is enabled
      --tls.enabled                  Enable TLS for client connection
  -u, --url string                   URL of fabric-ca-server (default "http://localhost:7054")

Use "fabric-ca-client [command] --help" for more information about a command.

File Formats¶

# Server's listening port (default: 7054)
port: 7054

# Enables debug logging (default: false)
debug: false

#############################################################################
#  TLS section for the server's listening port
#############################################################################
tls:
  # Enable TLS (default: false)
  enabled: false
  # TLS for the server's listening port (default: false)
  certfile: ca-cert.pem
  keyfile: ca-key.pem

#############################################################################
#  The CA section contains the key and certificate files used when
#  issuing enrollment certificates (ECerts) and transaction
#  certificates (TCerts).
#############################################################################
ca:
  # Certificate file (default: ca-cert.pem)
  certfile: ca-cert.pem
  # Key file (default: ca-key.pem)
  keyfile: ca-key.pem

#############################################################################
#  The registry section controls how the fabric-ca-server does two things:
#  1) authenticates enrollment requests which contain a username and password
#     (also known as an enrollment ID and secret).
#  2) once authenticated, retrieves the identity's attribute names and
#     values which the fabric-ca-server optionally puts into TCerts
#     which it issues for transacting on the Hyperledger Fabric blockchain.
#     These attributes are useful for making access control decisions in
#     chaincode.
#  There are two main configuration options:
#  1) The fabric-ca-server is the registry
#  2) An LDAP server is the registry, in which case the fabric-ca-server
#     calls the LDAP server to perform these tasks.
#############################################################################
registry:
  # Maximum number of times a password/secret can be reused for enrollment
  # (default: 0, which means there is no limit)
  maxEnrollments: 0

  # Contains user information which is used when LDAP is disabled
  identities:
     - name: <<<ADMIN>>>
       pass: <<<ADMINPW>>>
       type: client
       affiliation: ""
       attrs:
          hf.Registrar.Roles: "client,user,peer,validator,auditor,ca"
          hf.Registrar.DelegateRoles: "client,user,validator,auditor"
          hf.Revoker: true
          hf.IntermediateCA: true

#############################################################################
#  Database section
#  Supported types are: "sqlite3", "postgres", and "mysql".
#  The datasource value depends on the type.
#  If the type is "sqlite3", the datasource value is a file name to use
#  as the database store.  Since "sqlite3" is an embedded database, it
#  may not be used if you want to run the fabric-ca-server in a cluster.
#  To run the fabric-ca-server in a cluster, you must choose "postgres"
#  or "mysql".
#############################################################################
database:
  type: sqlite3
  datasource: fabric-ca-server.db
  tls:
      enabled: false
      certfiles: db-server-cert.pem
      client:
        certfile: db-client-cert.pem
        keyfile: db-client-key.pem

#############################################################################
#  LDAP section
#  If LDAP is enabled, the fabric-ca-server calls LDAP to:
#  1) authenticate enrollment ID and secret (i.e. username and password)
#     for enrollment requests;
#  2) To retrieve identity attributes
#############################################################################
ldap:
   # Enables or disables the LDAP client (default: false)
   enabled: false
   # The URL of the LDAP server
   url: ldap://<adminDN>:<adminPassword>@<host>:<port>/<base>
   tls:
      certfiles: ldap-server-cert.pem
      client:
         certfile: ldap-client-cert.pem
         keyfile: ldap-client-key.pem

#############################################################################
#  Affiliation section
#############################################################################
affiliations:
   org1:
      - department1
      - department2
   org2:
      - department1

#############################################################################
#  Signing section
#############################################################################
signing:
    profiles:
      ca:
         usage:
           - cert sign
         expiry: 8000h
         caconstraint:
           isca: true
    default:
      usage:
        - cert sign
      expiry: 8000h

###########################################################################
#  Certificate Signing Request section for generating the CA certificate
###########################################################################
csr:
   cn: fabric-ca-server
   names:
      - C: US
        ST: "North Carolina"
        L:
        O: Hyperledger
        OU: Fabric
   hosts:
     - <<<MYHOST>>>
   ca:
      pathlen:
      pathlenzero:
      expiry:

#############################################################################
#  Crypto section configures the crypto primitives used for all
#############################################################################
crypto:
  software:
     hash_family: SHA2
     security_level: 256
     ephemeral: false
     key_store_dir: keys

#############################################################################
# Client Configuration
#############################################################################

# URL of the fabric-ca-server (default: http://localhost:7054)
URL: http://localhost:7054

#############################################################################
#    TLS section for the client's listenting port
#############################################################################
tls:
  # Enable TLS (default: false)
  enabled: false

  # TLS for the client's listenting port (default: false)
  certfiles:   # Comma Separated (e.g. root.pem, root2.pem)
  client:
    certfile:
    keyfile:

#############################################################################
#  Certificate Signing Request section for generating the CSR for
#  an enrollment certificate (ECert)
#############################################################################
csr:
  cn: <<<ENROLLMENT_ID>>>
  names:
    - C: US
      ST: "North Carolina"
      L:
      O: Hyperledger
      OU: Fabric
  hosts:
   - <<<MYHOST>>>
  ca:
    pathlen:
    pathlenzero:
    expiry:

#############################################################################
#  Registration section used to register a new user with fabric-ca server
#############################################################################
id:
  name:
  type:
  affiliation:
  attributes:
    - name:
      value:

#############################################################################
#  Enrollment section used to enroll a user with fabric-ca server
#############################################################################
enrollment:
  hosts:
  profile:
  label:

Configuration Settings Precedence¶

The Fabric CA provides 3 way to configure settings on the fabric-ca-server and fabric-ca-client. The precedence order is defined below:

1. CLI flags
2. Environment variables
3. Configuration file

In the remainder of this document, we refer to making changes to configuration files. However, configuration file changes can be overridden through environment variables or CLI flags.

For example, if we have the following in the client configuration file:

tls:
  # Enable TLS (default: false)
  enabled: false

  # TLS for the client's listenting port (default: false)
  certfiles:   # Comma Separated (e.g. root.pem, root2.pem)
  client:
    certfile: cert.pem
    keyfile:

The following environment variable may be used to override the cert.pem setting in the configuration file:

export FABRIC_CA_CLIENT_TLS_CLIENT_CERTFILE=cert2.pem

If we wanted to override both the environment variable and configuration file, we can use a command line flag.

fabric-ca-client enroll --tls.client.certfile cert3.pem

The same approach applies to fabric-ca-server, except instead of using FABIRC_CA_CLIENT as the prefix to environment variables, FABRIC_CA_SERVER is used.

Fabric CA Server¶

This section describes the Fabric CA server.

You may initialize the Fabric CA server before starting it if you prefer. This provides an opportunity for you to generate a default configuration file but to review and customize its settings before starting it.

For the remainder of this server section, we assume that you have set the FABRIC_CA_HOME environment variable to $HOME/fabric-ca/server.

The instructions below assume that the server configuration file exists in the server’s home directory.

# fabric-ca-server init -b admin:adminpw

cn: localhost
key:
    algo: ecdsa
    size: 256
names:
  - C: US
    ST: "North Carolina"
    L:
    O: Hyperledger
    OU: Fabric

key:
   algo: ecdsa
   size: 256

# fabric-ca-server start -b <admin>:<adminpw>

# fabric-ca-server start -c <path-to-config-file> -b <admin>:<adminpw>

db:
  type: postgres
  datasource: host=localhost port=5432 user=Username password=Password dbname=fabric-ca-server sslmode=verify-full

db:
  ...
  tls:
      enabled: false
      certfiles: db-server-cert.pem
      client:
            certfile: db-client-cert.pem
            keyfile: db-client-key.pem

db:
  type: mysql
  datasource: root:rootpw@tcp(localhost:3306)/fabric-ca?parseTime=true&tls=custom

ldap:
   # Enables or disables the LDAP client (default: false)
   enabled: false
   # The URL of the LDAP server
   url: scheme://<adminDN>:<adminPassword>@<host>:<port>/<base>
   userfilter: filter

ldap:
   enabled: true
   url: ldap://cn=admin,dc=example,dc=org:admin@localhost:10389/dc=example,dc=org
   userfilter: (uid=%s)

global
      maxconn 4096
      daemon

defaults
      mode http
      maxconn 2000
      timeout connect 5000
      timeout client 50000
      timeout server 50000

listen http-in
      bind *:7054
      balance roundrobin
      server server1 hostname1:port
      server server2 hostname2:port
      server server3 hostname3:port

Fabric CA Client¶

This section describes how to use the fabric-ca-client command.

The default fabric-ca-client’s home directory is $HOME/.fabric-ca-client, but this can be changed by setting the FABRIC_CA_HOME or FABRIC_CA_CLIENT_HOME environment variable.

The instructions below assume that the client configuration file exists in the client’s home directory.

csr:
  key:
    algo: ecdsa
    size: 256
  names:
    - C: US
      ST: North Carolina
      L: Raleigh
      O: Hyperledger Fabric
      OU: Fabric CA
  hosts:
   - hostname
  ca:
    pathlen:
    pathlenzero:
    expiry:

# export FABRIC_CA_CLIENT_HOME=$HOME/fabric-ca/clients/admin
# fabric-ca-client enroll -u http://admin:adminpw@localhost:7054

id:
  name: MyPeer1
  type: peer
  affiliation: org1.department1
  attributes:
    - name: SomeAttrName
      value: SomeAttrValue
    - name: foo
      value: bar

# export FABRIC_CA_CLIENT_HOME=$HOME/fabric-ca/clients/admin
# fabric-ca-client register

# export FABRIC_CA_CLIENT_HOME=$HOME/fabric-ca/clients/admin
# fabric-ca-client register --id.name MyPeer2 --id.secret mypassword

# export FABRIC_CA_CERT_FILE=$MSP_DIR/signcerts/peer.pem
# export FABRIC_CA_KEY_FILE=$MSP_DIR/keystore/key.pem
# fabric-ca client enroll -u http://peer1:<password>@localhost:7054

# export FABRIC_CA_CLIENT_HOME=$HOME/fabric-ca/clients/admin
# cd $FABRIC_CA_HOME
# fabric-ca-client reenroll

fabric-ca-client revoke -a xxx -s yyy -r <reason>

fabric-ca-client revoke -e <enrollment_id> -r <reason>

tls:
  # Enable TLS (default: false)
  enabled: true

  # TLS for the client's listenting port (default: false)
  certfiles: root.pem   # Comma Separated (e.g. root.pem,root2.pem)
  client:
    certfile: tls_client-cert.pem
    keyfile: tls_client-key.pem

Appendix¶

Chaincode interfaces¶

A chaincode implements the Chaincode Interface that supports two methods:

• Init
• Invoke

Dependencies¶

The import statement lists a few dependencies for the chaincode to compile successfully.

• fmt – contains Println for debugging/logging.
• errors – standard go error format.
• shim – contains the definitions for the chaincode interface and the chaincode stub, which are required to interact with the ledger.

Chaincode APIs¶

When the Init or Invoke function of a chaincode is called, the fabric passes the stub shim.ChaincodeStubInterface parameter and the chaincode returns a pb.Response. This stub can be used to call APIs to access to the ledger services, transaction context, or to invoke other chaincodes.

The current APIs are defined in the shim package, and can be generated with the following command:

godoc github.com/hyperledger/fabric/core/chaincode/shim

However, it also includes functions from chaincode.pb.go (protobuffer functions) that are not intended as public APIs. The best practice is to look at the function definitions in chaincode.go and and the examples directory.

Response¶

The chaincode response comes in the form of a protobuffer.

message Response {

    // A status code that should follow the HTTP status codes.
    int32 status = 1;

    // A message associated with the response code.
    string message = 2;

    // A payload that can be used to include metadata with this response.
    bytes payload = 3;

}

The chaincode will also return events. Message events and chaincode events.

messageEvent {

    oneof Event {

    //Register consumer sent event
    Register register = 1;

    //producer events common.
    Block block = 2;
    ChaincodeEvent chaincodeEvent = 3;
    Rejection rejection = 4;

    //Unregister consumer sent events
    Unregister unregister = 5;

    }

}

messageChaincodeEvent {

    string chaincodeID = 1;
    string txID = 2;
    string eventName = 3;
    bytes payload = 4;

}

Once developed and deployed, there are two ways to interact with the chaincode - through an SDK or the CLI. The steps for CLI are described below. For SDK interaction, refer to the balance transfer samples. Note: This SDK interaction is covered in the Getting Started section.

Command Line Interfaces¶

To view the currently available CLI commands, execute the following:

# this assumes that you have correctly set the GOPATH variable and cloned the Fabric codebase into that path
cd /opt/gopath/src/github.com/hyperledger/fabric
build /bin/peer

You will see output similar to the example below. (NOTE: rootcommand below is hardcoded in main.go. Currently, the build will create a peer executable file).

Usage:
      peer [flags]
      peer [command]

    Available Commands:
      version     Print fabric peer version.
      node        node specific commands.
      channel     channel specific commands.
      chaincode   chaincode specific commands.
      logging     logging specific commands


    Flags:
      --logging-level string: Default logging level and overrides, see core.yaml for full syntax
      --test.coverprofile string: Done (default “coverage.cov)
      -v, --version: Display current version of fabric peer server
    Use "peer [command] --help" for more information about a command.

The peer command supports several subcommands and flags, as shown above. To facilitate its use in scripted applications, the peer command always produces a non-zero return code in the event of command failure. Upon success, many of the subcommands produce a result on stdout as shown in the table below:

--peer-defaultchain=true

Chaincode Swimlanes¶

Deploy a chaincode¶

[WIP] - the CLI commands need to be refactored based on the new deployment model. Channel Create and Channel Join will remain the same.

Endorsement policy design¶

Endorsement policies have two main components: - a principal - a threshold gate

A principal P identifies the entity whose signature is expected.

A threshold gate T takes two inputs: an integer t (the threshold) and a list of n principals or gates; this gate essentially captures the expectation that out of those n principals or gates, t are requested to be satisfied.

For example: - T(2, 'A', 'B', 'C') requests a signature from any 2 principals out of ‘A’, ‘B’ or ‘C’; - T(1, 'A', T(2, 'B', 'C')) requests either one signature from principal A or 1 signature from B and C each.

Endorsement policy syntax in the CLI¶

In the CLI, a simple language is used to express policies in terms of boolean expressions over principals.

A principal is described in terms of the MSP that is tasked to validate the identity of the signer and of the role that the signer has within that MSP. Currently, two roles are supported: member and admin. Principals are described as MSP.ROLE, where MSP is the MSP ID that is required, and ROLE is either one of the two strings member and admin. Examples of valid principals are 'Org0.admin' (any administrator of the Org0 MSP) or 'Org1.member' (any member of the Org1 MSP).

The syntax of the language is:

EXPR(E[, E...])

where EXPR is either AND or OR, representing the two boolean expressions and E is either a principal (with the syntax described above) or another nested call to EXPR.

For example: - AND('Org1.member', 'Org2.member', 'Org3.member') requests 1 signature from each of the three principals - OR('Org1.member', 'Org2.member') requests 1 signature from either one of the two principals - OR('Org1.member', AND('Org2.member', 'Org3.member')) requests either one signature from a member of the Org1 MSP or 1 signature from a member of the Org2 MSP and 1 signature from a member of the Org3 MSP.

Specifying endorsement policies for a chaincode¶

Using this language, a chaincode deployer can request that the endorsements for a chaincode be validated against the specified policy. NOTE - the default policy requires one signature from a member of the DEFAULT MSP). This is used if a policy is not specified in the CLI.

The policy can be specified at deploy time using the -P switch, followed by the policy.

For example:

peer chaincode deploy -C testchainid -n mycc -p github.com/hyperledger/fabric/examples/chaincode/go/chaincode_example02 -c '{"Args":["init","a","100","b","200"]}' -P "AND('Org1.member', 'Org2.member')"

This command deploys chaincode mycc on chain testchainid with the policy AND('Org1.member', 'Org2.member').

Future enhancements¶

In this section we list future enhancements for endorsement policies: - alongside the existing way of identifying principals by their relationship with an MSP, we plan to identify principals in terms of the Organization Unit (OU) expected in their certificates; this is useful to express policies where we request signatures from any identity displaying a valid certificate with an OU matching the one requested in the definition of the principal. - instead of the syntax AND(., .) we plan to move to a more intuitive syntax . AND . - we plan to expose generalized threshold gates in the language as well alongside AND (which is the special n-out-of-n gate) and OR (which is the special 1-out-of-n gate)

Chain¶

The chain is a transaction log, structured as hash-linked blocks, where each block contains a sequence of N transactions. The block header includes a hash of the block’s transactions, as well as a hash of the prior block’s header. In this way, all transactions on the ledger are sequenced and cryptographically linked together. In other words, it is not possible to tamper with the ledger data, without breaking the hash links. The hash of the latest block represents every transaction that has come before, making it possible to ensure that all peers are in a consistent and trusted state.

The chain is stored on the peer file system (either local or attached storage), efficiently supporting the append-only nature of the blockchain workload.

State Database¶

The ledger’s current state data represents the latest values for all keys ever included in the chain transaction log. Since current state represents all latest key values known to the channel, it is sometimes referred to as World State.

Chaincode invocations execute transactions against the current state data. To make these chaincode interactions extremely efficient, the latest values of all keys are stored in a state database. The state database is simply an indexed view into the chain’s transaction log, it can therefore be regenerated from the chain at any time. The state database will automatically get recovered (or generated if needed) upon peer startup, before transactions are accepted.

Transaction Flow¶

At a high level, the transaction flow consists of a transaction proposal sent by an application client to specific endorsing peers. The endorsing peers verify the client signature, and execute a chaincode function to simulate the transaction. The output is the chaincode results, a set of key/value versions that were read in the chaincode (read set), and the set of keys/values that were written in chaincode (write set). The proposal response gets sent back to the client along with an endorsement signature.

The client assembles the endorsements into a transaction payload and broadcasts it to an ordering service. The ordering service delivers ordered transactions as blocks to all peers on a channel.

Before committal, peers will validate the transactions. First, they will check the endorsement policy to ensure that the correct allotment of the specified peers have signed the results, and they will authenticate the signatures against the transaction payload.

Secondly, peers will perform a versioning check against the transaction read set, to ensure data integrity and protect against threats such as double-spending. The fabric has concurrency control whereby transactions execute in parallel (by endorsers) to increase throughput, and upon commit (by all peers) each transaction is verified to ensure that no other transaction has modified data it has read. In other words, it ensures that the data that was read during chaincode execution has not changed since execution (endorsement) time, and therefore the execution results are still valid and can be committed to the ledger state database. If the data that was read has been changed by another transaction, then the transaction in the block is marked as invalid and is not applied to the ledger state database. The client application is alerted, and can handle the error or retry as appropriate.

See the Transaction Flow and Read-Write set semantics topics for a deeper dive on transaction structure, concurrency control, and the state DB.

State Database options¶

State database options include LevelDB and CouchDB (beta). LevelDB is the default key/value state database embedded in the peer process. CouchDB is an optional alternative external state database. Like the LevelDB key/value store, CouchDB can store any binary data that is modeled in chaincode (CouchDB attachment functionality is used internally for non-JSON binary data). But as a JSON document store, CouchDB additionally enables rich query against the chaincode data, when chaincode values (e.g. assets) are modeled as JSON data.

Both LevelDB and CouchDB support core chaincode operations such as getting and setting a key (asset), and querying based on keys. Keys can be queried by range, and composite keys can be modeled to enable equivalence queries against multiple parameters. For example a composite key of (owner,asset_id) can be used to query all assets owned by a certain entity. These key-based queries can be used for read-only queries against the ledger, as well as in transactions that update the ledger.

If you model assets as JSON and use CouchDB, you can also perform complex rich queries against the chaincode data values, using the CouchDB JSON query language within chaincode. These types of queries are excellent for understanding what is on the ledger. Proposal responses for these types of queries are typically useful to the client application, but are not typically submitted as transactions to the ordering service. In fact the fabric does not guarantee the result set is stable between chaincode execution and commit time for rich queries, and therefore rich queries are not appropriate for use in update transactions, unless your application can guarantee the result set is stable between chaincode execution time and commit time, or can handle potential changes in subsequent transactions. For example, if you perform a rich query for all assets owned by Alice and transfer them to Bob, a new asset may be assigned to Alice by another transaction between chaincode execution time and commit time, and you would miss this ‘phantom’ item.

CouchDB runs as a separate database process alongside the peer, therefore there are additional considerations in terms of setup, management, and operations. You may consider starting with the default embedded LevelDB, and move to CouchDB if you require the additional complex rich queries. It is a good practice to model chaincode asset data as JSON, so that you have the option to perform complex rich queries if needed in the future.

To enable CouchDB as the state database, configure the /fabric/peer/core.yaml stateDatabase section.

Transaction simulation and read-write set¶

During simulation of a transaction at an endorser, a read-write set is prepared for the transaction. The read set contains a list of unique keys and their committed versions that the transaction reads during simulation. The write set contains a list of unique keys (though there can be overlap with the keys present in the read set) and their new values that the transaction writes. A delete marker is set (in the place of new value) for the key if the update performed by the transaction is to delete the key.

Further, if the transaction writes a value multiple times for a key, only the last written value is retained. Also, if a transaction reads a value for a key, the value in the committed state is returned even if the transaction has updated the value for the key before issuing the read. In another words, Read-your-writes semantics are not supported.

As noted earlier, the versions of the keys are recorded only in the read set; the write set just contains the list of unique keys and their latest values set by the transaction.

There could be various schemes for implementing versions. The minimal requirement for a versioning scheme is to produce non-repeating identifiers for a given key. For instance, using monotonically increasing numbers for versions can be one such scheme. In the current implementation, we use a blockchain height based versioning scheme in which the height of the committing transaction is used as the latest version for all the keys modified by the transaction. In this scheme, the height of a transaction is represented by a tuple (txNumber is the height of the transaction within the block). This scheme has many advantages over the incremental number scheme - primarily, it enables other components such as statedb, transaction simulation and validation for making efficient design choices.

Following is an illustration of an example read-write set prepared by simulation of a hypothetical transaction. For the sake of simplicity, in the illustrations, we use the incremental numbers for representing the versions.

<TxReadWriteSet>
  <NsReadWriteSet name="chaincode1">
    <read-set>
      <read key="K1", version="1">
      <read key="K2", version="1">
    </read-set>
    <write-set>
      <write key="K1", value="V1"
      <write key="K3", value="V2"
      <write key="K4", isDelete="true"
    </write-set>
  </NsReadWriteSet>
<TxReadWriteSet>

Additionally, if the transaction performs a range query during simulation, the range query as well as its results will be added to the read-write set as query-info.

Transaction validation and updating world state using read-write set¶

A committer uses the read set portion of the read-write set for checking the validity of a transaction and the write set portion of the read-write set for updating the versions and the values of the affected keys.

In the validation phase, a transaction is considered valid if the version of each key present in the read set of the transaction matches the version for the same key in the world state - assuming all the preceding valid transactions (including the preceding transactions in the same block) are committed (committed-state). An additional validation is performed if the read-write set also contains one or more query-info.

This additional validation should ensure that no key has been inserted/deleted/updated in the super range (i.e., union of the ranges) of the results captured in the query-info(s). In other words, if we re-execute any of the range queries (that the transaction performed during simulation) during validation on the committed-state, it should yield the same results that were observed by the transaction at the time of simulation. This check ensures that if a transaction observes phantom items during commit, the transaction should be marked as invalid. Note that the this phantom protection is limited to range queries (i.e., GetStateByRange function in the chaincode) and not yet implemented for other queries (i.e., GetQueryResult function in the chaincode). Other queries are at risk of phantoms, and should therefore only be used in read-only transactions that are not submitted to ordering, unless the application can guarantee the stability of the result set between simulation and validation/commit time.

If a transaction passes the validity check, the committer uses the write set for updating the world state. In the update phase, for each key present in the write set, the value in the world state for the same key is set to the value as specified in the write set. Further, the version of the key in the world state is changed to reflect the latest version.

Example simulation and validation¶

This section helps with understanding the semantics through an example scenario. For the purpose of this example, the presence of a key, k, in the world state is represented by a tuple (k,ver,val) where ver is the latest version of the key k having val as its value.

Now, consider a set of five transactions T1, T2, T3, T4, and T5, all simulated on the same snapshot of the world state. The following snippet shows the snapshot of the world state against which the transactions are simulated and the sequence of read and write activities performed by each of these transactions.

World state: (k1,1,v1), (k2,1,v2), (k3,1,v3), (k4,1,v4), (k5,1,v5)
T1 -> Write(k1, v1'), Write(k2, v2')
T2 -> Read(k1), Write(k3, v3')
T3 -> Write(k2, v2'')
T4 -> Write(k2, v2'''), read(k2)
T5 -> Write(k6, v6'), read(k5)

Now, assume that these transactions are ordered in the sequence of T1,..,T5 (could be contained in a single block or different blocks)

1. T1 passes validation because it does not perform any read. Further, the tuple of keys k1 and k2 in the world state are updated to (k1,2,v1'), (k2,2,v2')
2. T2 fails validation because it reads a key, k1, which was modified by a preceding transaction - T1
3. T3 passes the validation because it does not perform a read. Further the tuple of the key, k2, in the world state is updated to (k2,3,v2'')
4. T4 fails the validation because it reads a key, k2, which was modified by a preceding transaction T1
5. T5 passes validation because it reads a key, k5, which was not modified by any of the preceding transactions

Note: Transactions with multiple read-write sets are not yet supported.

Gossip protocol¶

Peers leverage gossip to broadcast ledger and channel data in a scalable fashion. Gossip messaging is continuous, and each peer on a channel is constantly receiving current and consistent ledger data, from multiple peers. Each gossiped message is signed, thereby allowing Byzantine participants sending faked messages to be easily identified and the distribution of the message(s) to unwanted targets to be prevented. Peers affected by delays, network partitions or other causations resulting in missed blocks, will eventually be synced up to the current ledger state by contacting peers in possession of these missing blocks.

The gossip-based data dissemination protocol performs three primary functions on a Fabric network:

1. Manages peer discovery and channel membership, by continually identifying available member peers, and eventually detecting peers that have gone offline.
2. Disseminates ledger data across all peers on a channel. Any peer with data that is out of sync with the rest of the channel identifies the missing blocks and syncs itself by copying the correct data.
3. Bring newly connected peers up to speed by allowing peer-to-peer state transfer update of ledger data.

Gossip-based broadcasting operates by peers receiving messages from other peers on the channel, and then forwarding these messages to a number of randomly-selected peers on the channel, where this number is a configurable constant. Peers can also exercise a pull mechanism, rather than waiting for delivery of a message. This cycle repeats, with the result of channel membership, ledger and state information continually being kept current and in sync. For dissemination of new blocks, the leader peer on the channel pulls the data from the ordering service and initiates gossip dissemination to peers.

Gossip messaging¶

Online peers indicate their availability by continually broadcasting “alive” messages, with each containing the public key infrastructure (PKI) ID and the signature of the sender over the message. Peers maintain channel membership by collecting these alive messages; if no peer receives an alive message from a specific peer, this “dead” peer is eventually purged from channel membership. Because “alive” messages are cryptographically signed, malicious peers can never impersonate other peers, as they lack a signing key authorized by a root certificate authority (CA).

In addition to the automatic forwarding of received messages, a state reconciliation process synchronizes world state across peers on each channel. Each peer continually pulls blocks from other peers on the channel, in order to repair its own state if discrepancies are identified. Because fixed connectivity is not required to maintain gossip-based data dissemination, the process reliably provides data consistency and integrity to the shared ledger, including tolerance for node crashes.

Because channels are segregated, peers on one channel cannot message or share information on any other channel. Though any peer can belong to multiple channels, partitioned messaging prevents blocks from being disseminated to peers that are not in the channel by applying message routing policies based on peers’ channel subscriptions.

Endorsement¶

Endorsement architecture:

17. How many peers in the network need to endorse a transaction?

A. The number of peers required to endorse a transaction is driven by the endorsement policy that is specified at chaincode deployment time.

17. Does an application client need to connect to all peers?

A. Clients only need to connect to as many peers as are required by the endorsement policy for the chaincode.

Security & Access Control¶

Data Privacy and Access Control:

17. How do I ensure data privacy?

A. There are various aspects to data privacy. First, you can segregate your network into channels, where each channel represents a subset of participants that are authorized to see the data for the chaincodes that are deployed to that channel. Second, within a channel you can restrict the input data to chaincode to the set of endorsers only, by using visibility settings. The visibility setting will determine whether input and output chaincode data is included in the submitted transaction, versus just output data. Third, you can hash or encrypt the data before calling chaincode. If you hash the data then you will need a way to share the source data outside of fabric. If you encrypt the data then you will need a way to share the decryption keys outside of fabric. Fourth, you can restrict data access to certain roles in your organization, by building access control into the chaincode logic. Fifth, ledger data at rest can be encrypted via file system encryption on the peer, and data in transit is encrypted via TLS.

17. Do the orderers see the transaction data?

A. No, the orderers only order transactions, they do not open the transactions. If you do not want the data to go through the orderers at all, and you are only concerned about the input data, then you can use visibility settings. The visibility setting will determine whether input and output chaincode data is included in the submitted transaction, versus just output data. Therefore the input data can be private to the endorsers only. If you do not want the orderers to see chaincode output, then you can hash or encrypt the data before calling chaincode. If you hash the data then you will need a way to share the source data outside of fabric. If you encrypt the data then you will need a way to share the decryption keys outside of fabric.

Application-side Programming Model¶

Transaction execution result:

17. How do application clients know the outcome of a transaction?

A. The transaction simulation results are returned to the client by the endorser in the proposal response. If there are multiple endorsers, the client can check that the responses are all the same, and submit the results and endorsements for ordering and commitment. Ultimately the committing peers will validate or invalidate the transaction, and the client becomes aware of the outcome via an event, that the SDK makes available to the application client.

Ledger queries:

17. How do I query the ledger data?

Within chaincode you can query based on keys. Keys can be queried by range, and composite keys can be modeled to enable equivalence queries against multiple parameters. For example a composite key of (owner,asset_id) can be used to query all assets owned by a certain entity. These key-based queries can be used for read-only queries against the ledger, as well as in transactions that update the ledger.

If you model asset data as JSON in chaincode and use CouchDB as the state database, you can also perform complex rich queries against the chaincode data values, using the CouchDB JSON query language within chaincode. The application client can perform read-only queries, but these responses are not typically submitted as part of transactions to the ordering service.

17. How do I query the historical data to understand data provenance?

A. The chaincode API GetHistoryForKey() will return history of values for a key.

Q. How to guarantee the query result is correct, especially when the peer being queried may be recovering and catching up on block processing?

A. The client can query multiple peers, compare their block heights, compare their query results, and favor the peers at the higher block heights.

Anchor Peer¶

A peer node on a channel that all other peers can discover and communicate with. Each Member on a channel has an anchor peer (or multiple anchor peers to prevent single point of failure), allowing for peers belonging to different Members to discover all existing peers on a channel.

Block¶

An ordered set of transactions that is cryptographically linked to the preceding block(s) on a channel.

Chain¶

The ledger’s chain is a transaction log structured as hash-linked blocks of transactions. Peers receive blocks of transactions from the ordering service, mark the block’s transactions as valid or invalid based on endorsement policies and concurrency violations, and append the block to the hash chain on the peer’s file system.

Chaincode¶

Chaincode is software, running on a ledger, to encode assets and the transaction instructions (business logic) for modifying the assets.

Channel¶

A channel is a private blockchain overlay on a Fabric network, allowing for data isolation and confidentiality. A channel-specific ledger is shared across the peers in the channel, and transacting parties must be properly authenticated to a channel in order to interact with it. Channels are defined by a Configuration-Block.

Commitment¶

Each Peer on a channel validates ordered blocks of transactions and then commits (writes/appends) the blocks to its replica of the channel Ledger. Peers also mark each transaction in each block as valid or invalid.

Concurrency Control Version Check¶

Concurrency Control Version Check is a method of keeping state in sync across peers on a channel. Peers execute transactions in parallel, and before commitment to the ledger, peers check that the data read at execution time has not changed. If the data read for the transaction has changed between execution time and commitment time, then a Concurrency Control Version Check violation has occurred, and the transaction is marked as invalid on the ledger and values are not updated in the state database.

Configuration Block¶

Contains the configuration data defining members and policies for a system chain (ordering service) or channel. Any configuration modifications to a channel or overall network (e.g. a member leaving or joining) will result in a new configuration block being appended to the appropriate chain. This block will contain the contents of the genesis block, plus the delta.

Consensus¶

A broader term overarching the entire transactional flow, which serves to generate an agreement on the order and to confirm the correctness of the set of transactions constituting a block.

Current State¶

The current state of the ledger represents the latest values for all keys ever included in its chain transaction log. Peers commit the latest values to ledger current state for each valid transaction included in a processed block. Since current state represents all latest key values known to the channel, it is sometimes referred to as World State. Chaincode executes transaction proposals against current state data.

Dynamic Membership¶

Fabric supports the addition/removal of members, peers, and ordering service nodes, without compromising the operationality of the overall network. Dynamic membership is critical when business relationships adjust and entities need to be added/removed for various reasons.

Endorsement¶

Refers to the process where specific peer nodes execute a transaction and return a YES/NO response to the client application that generated the transaction proposal. Chaincode applications have corresponding endorsement policies, in which the endorsing peers are specified.

Endorsement policy¶

Defines the peer nodes on a channel that must execute transactions attached to a specific chaincode application, and the required combination of responses (endorsements). A policy could require that a transaction be endorsed by a minimum number of endorsing peers, a minimum percentage of endorsing peers, or by all endorsing peers that are assigned to a specific chaincode application. Policies can be curated based on the application and the desired level of resilience against misbehavior (deliberate or not) by the endorsing peers. A distinct endorsement policy for deploy transactions, which install new chaincode, is also required.

Genesis Block¶

The configuration block that initializes a blockchain network or channel, and also serves as the first block on a chain.

Gossip Protocol¶

The gossip data dissemination protocol performs three functions: 1) manages peer discovery and channel membership; 2) disseminates ledger data across all peers on the channel; 3) syncs ledger state across all peers on the channel. Refer to the Gossip topic for more details.

Initialize¶

A method to initialize a chaincode application.

Install¶

The process of placing a chaincode on a peer’s file system.

Instantiate¶

The process of starting a chaincode container.

Invoke¶

Used to call chaincode functions. Invocations are captured as transaction proposals, which then pass through a modular flow of endorsement, ordering, validation, committal. The structure of invoke is a function and an array of arguments.

Leading Peer¶

Each Member can own multiple peers on each channel that it subscribes to. One of these peers is serves as the leading peer for the channel, in order to communicate with the network ordering service on behalf of the member. The ordering service “delivers” blocks to the leading peer(s) on a channel, who then distribute them to other peers within the same member cluster.

Ledger¶

A ledger is a channel’s chain and current state data which is maintained by each peer on the channel.

Member¶

A legally separate entity that owns a unique root certificate for the network. Network components such as peer nodes and application clients will be linked to a member.

Membership Service Provider¶

The Membership Service Provider (MSP) refers to an abstract component of the system that provides credentials to clients, and peers for them to participate in a Hyperledger Fabric network. Clients use these credentials to authenticate their transactions, and peers use these credentials to authenticate transaction processing results (endorsements). While strongly connected to the transaction processing components of the systems, this interface aims to have membership services components defined, in such a way that alternate implementations of this can be smoothly plugged in without modifying the core of transaction processing components of the system.

Membership Services¶

Membership Services authenticates, authorizes, and manages identities on a permissioned blockchain network. The membership services code that runs in peers and orderers both authenticates and authorizes blockchain operations. It is a PKI-based implementation of the Membership Services Provider (MSP) abstraction.

The fabric-ca component is an implementation of membership services to manage identities. In particular, it handles the issuance and revocation of enrollment certificates and transaction certificates.

An enrollment certificate is a long-term identity credential; a transaction certificate is a short-term identity credential which is both anonymous and un-linkable.

Ordering Service¶

A defined collective of nodes that orders transactions into a block. The ordering service exists independent of the peer processes and orders transactions on a first-come-first-serve basis for all channel’s on the network. The ordering service is designed to support pluggable implementations beyond the out-of-the-box SOLO and Kafka varieties. The ordering service is a common binding for the overall network; it contains the cryptographic identity material tied to each Member.

Peer¶

A network entity that maintains a ledger and runs chaincode containers in order to perform read/write operations to the ledger. Peers are owned and maintained by members.

Policy¶

There are policies for endorsement, validation, block committal, chaincode management and network/channel management.

Proposal¶

A request for endorsement that is aimed at specific peers on a channel. Each proposal is either an instantiate or an invoke (read/write) request.

Query¶

A query requests the value of a key(s) against the current state.

Software Development Kit (SDK)¶

The Hyperledger Fabric client SDK provides a structured environment of libraries for developers to write and test chaincode applications. The SDK is fully configurable and extensible through a standard interface. Components, including cryptographic algorithms for signatures, logging frameworks and state stores, are easily swapped in and out of the SDK. The SDK API uses protocol buffers over gRPC for transaction processing, membership services, node traversal and event handling applications to communicate across the fabric. The SDK comes in multiple flavors - Node.js, Java. and Python.

State Database¶

Current state data is stored in a state database for efficient reads and queries from chaincode. These databases include levelDB and couchDB.

System Chain¶

Contains a configuration block defining the network at a system level. The system chain lives within the ordering service, and similar to a channel, has an initial configuration containing information such as: MSP information, policies, and configuration details. Any change to the overall network (e.g. a new org joining or a new ordering node being added) will result in a new configuration block being added to the system chain.

The system chain can be thought of as the common binding for a channel or group of channels. For instance, a collection of financial institutions may form a consortium (represented through the system chain), and then proceed to create channels relative to their aligned and varying business agendas.

Transaction¶

An invoke or instantiate operation. Invokes are requests to read/write data from the ledger. Instantiate is a request to start a chaincode container on a peer.

Getting a Linux Foundation account¶

In order to participate in the development of the Hyperledger Fabric project, you will need a Linux Foundation account. You will need to use your LF ID to access to all the Hyperledger community development tools, including Gerrit, Jira and the Wiki (for editing, only).

Getting help¶

If you are looking for something to work on, or need some expert assistance in debugging a problem or working out a fix to an issue, our community is always eager to help. We hang out on Chat, IRC (#hyperledger on freenode.net) and the mailing lists. Most of us don’t bite :grin: and will be glad to help. The only silly question is the one you don’t ask. Questions are in fact a great way to help improve the project as they highlight where our documentation could be clearer.

Requirements and Use Cases¶

We have a Requirements WG that is documenting use cases and from those use cases deriving requirements. If you are interested in contributing to this effort, please feel free to join the discussion in chat.

Reporting bugs¶

If you are a user and you find a bug, please submit an issue using JIRA. Please try to provide sufficient information for someone else to reproduce the issue. One of the project’s maintainers should respond to your issue within 24 hours. If not, please bump the issue with a comment and request that it be reviewed. You can also post to the #fabric-maintainers channel in chat.

Fixing issues and working stories¶

Review the issues list and find something that interests you. You could also check the “help-wanted” list. It is wise to start with something relatively straight forward and achievable, and that no one is already assigned. If no one is assigned, then assign the issue to yourself. Please be considerate and rescind the assignment if you cannot finish in a reasonable time, or add a comment saying that you are still actively working the issue if you need a little more time.

Working with a local clone and Gerrit¶

We are using Gerrit to manage code contributions. If you are unfamiliar, please review this document before proceeding.

After you have familiarized yourself with Gerrit, and maybe played around with the lf-sandbox project, you should be ready to set up your local development environment.

Next, try building the project in your local development environment to ensure that everything is set up correctly.

Logging control describes how to tweak the logging levels of various components within the Fabric. Finally, every source file needs to include a license header: modified to include a copyright statement for the principle author(s).

What makes a good change request?¶

• One change at a time. Not five, not three, not ten. One and only one. Why? Because it limits the blast area of the change. If we have a regression, it is much easier to identify the culprit commit than if we have some composite change that impacts more of the code.

• Include a link to the JIRA story for the change. Why? Because a) we want to track our velocity to better judge what we think we can deliver and when and b) because we can justify the change more effectively. In many cases, there should be some discussion around a proposed change and we want to link back to that from the change itself.

• Include unit and integration tests (or changes to existing tests) with every change. This does not mean just happy path testing, either. It also means negative testing of any defensive code that it correctly catches input errors. When you write code, you are responsible to test it and provide the tests that demonstrate that your change does what it claims. Why? Because without this we have no clue whether our current code base actually works.

• Unit tests should have NO external dependencies. You should be able to run unit tests in place with go test or equivalent for the language. Any test that requires some external dependency (e.g. needs to be scripted to run another component) needs appropriate mocking. Anything else is not unit testing, it is integration testing by definition. Why? Because many open source developers do Test Driven Development. They place a watch on the directory that invokes the tests automagically as the code is changed. This is far more efficient than having to run a whole build between code changes.

• Minimize the lines of code per CR. Why? Maintainers have day jobs, too. If you send a 1,000 or 2,000 LOC change, how long do you think it takes to review all of that code? Keep your changes to < 200-300 LOC, if possible. If you have a larger change, decompose it into multiple independent changess. If you are adding a bunch of new functions to fulfill the requirements of a new capability, add them separately with their tests, and then write the code that uses them to deliver the capability. Of course, there are always exceptions. If you add a small change and then add 300 LOC of tests, you will be forgiven;-) If you need to make a change that has broad impact or a bunch of generated code (protobufs, etc.). Again, there can be exceptions.

  Note: large change requests, e.g. those with more than 300 LOC are more likely than not going to receive a -2, and you’ll be asked to refactor the change to conform with this guidance.

• Do not stack change requests (e.g. submit a CR from the same local branch as your previous CR) unless they are related. This will minimize merge conflicts and allow changes to be merged more quickly. If you stack requests your subsequent requests may be held up because of review comments in the preceding requests.

• Write a meaningful commit message. Include a meaningful 50 (or less) character title, followed by a blank line, followed by a more comprehensive description of the change. Each change MUST include the JIRA identifier corresponding to the change (e.g. [FAB-1234]). This can be in the title but should also be in the body of the commit message.

  Note that Gerrit will automatically create a hyperlink to the JIRA item.

e.g.

[FAB-1234] fix foobar() panic

Fix [FAB-1234] added a check to ensure that when foobar(foo string) is called,
that there is a non-empty string argument.

Finally, be responsive. Don’t let a change request fester with review comments such that it gets to a point that it requires a rebase. It only further delays getting it merged and adds more work for you - to remediate the merge conflicts.

Coding guidelines¶

Be sure to check out the language-specific style guides before making any changes. This will ensure a smoother review.

Communication¶

We use RocketChat for communication and Google Hangouts™ for screen sharing between developers. Our development planning and prioritization is done in JIRA, and we take longer running discussions/decisions to the mailing list.

Maintainers¶

The project’s maintainers are responsible for reviewing and merging all patches submitted for review and they guide the over-all technical direction of the project within the guidelines established by the Hyperledger Project’s Technical Steering Committee (TSC).

Legal stuff¶

Note: Each source file must include a license header for the Apache Software License 2.0. A template of that header can be found here.

We have tried to make it as easy as possible to make contributions. This applies to how we handle the legal aspects of contribution. We use the same approach—the Developer’s Certificate of Origin 1.1 (DCO)—that the Linux® Kernel community uses to manage code contributions.

We simply ask that when submitting a patch for review, the developer must include a sign-off statement in the commit message.

Here is an example Signed-off-by line, which indicates that the submitter accepts the DCO:

Signed-off-by: John Doe <john.doe@hisdomain.com>

You can include this automatically when you commit a change to your local git repository using git commit -s.

Overview¶

Prior to the v1.0.0 release, the development environment utilized Vagrant running an Ubuntu image, which in turn launched Docker containers as a means of ensuring a consistent experience for developers who might be working with varying platforms, such as macOS, Windows, Linux, or whatever. Advances in Docker have enabled native support on the most popular development platforms: macOS and Windows. Hence, we have reworked our build to take full advantage of these advances. While we still maintain a Vagrant based approach that can be used for older versions of macOS and Windows that Docker does not support, we strongly encourage that the non-Vagrant development setup be used.

Note that while the Vagrant-based development setup could not be used in a cloud context, the Docker-based build does support cloud platforms such as AWS, Azure, Google and IBM to name a few. Please follow the instructions for Ubuntu builds, below.

Prerequisites¶

• Git client
• Go - 1.7 or later (for releases before v1.0, 1.6 or later)
• For macOS, Xcode must be installed
• Docker - 1.12 or later
• Docker Compose - 1.8.1 or later
• Pip
• (macOS) you may need to install gnutar, as macOS comes with bsdtar as the default, but the build uses some gnutar flags. You can use Homebrew to install it as follows:

brew install gnu-tar --with-default-names

• (only if using Vagrant) - Vagrant - 1.7.4 or later
• (only if using Vagrant) - VirtualBox - 5.0 or later
• BIOS Enabled Virtualization - Varies based on hardware
• Note: The BIOS Enabled Virtualization may be within the CPU or Security settings of the BIOS

pip, behave and docker-compose¶

pip install --upgrade pip
pip install behave nose docker-compose
pip install -I flask==0.10.1 python-dateutil==2.2 pytz==2014.3 pyyaml==3.10 couchdb==1.0 flask-cors==2.0.1 requests==2.4.3 pyOpenSSL==16.2.0 pysha3==1.0b1 grpcio==1.0.4

#PIP packages required for some behave tests
pip install urllib3 ndg-httpsclient pyasn1 ecdsa python-slugify grpcio-tools jinja2 b3j0f.aop six

Steps¶

git config --get core.autocrlf

git config --global core.autocrlf false

cd $GOPATH/src
mkdir -p github.com/hyperledger
cd github.com/hyperledger

git clone ssh://LFID@gerrit.hyperledger.org:29418/fabric && scp -p -P 29418 LFID@gerrit.hyperledger.org:hooks/commit-msg fabric/.git/hooks/

cd $GOPATH/src/github.com/hyperledger/fabric/devenv
vagrant up

vagrant ssh

Building the fabric¶

Once you have all the dependencies installed, and have cloned the repository, you can proceed to build and test the fabric.

Notes¶

NOTE: Any time you change any of the files in your local fabric directory (under $GOPATH/src/github.com/hyperledger/fabric), the update will be instantly available within the VM fabric directory.

NOTE: If you intend to run the development environment behind an HTTP Proxy, you need to configure the guest so that the provisioning process may complete. You can achieve this via the vagrant-proxyconf plugin. Install with vagrant plugin install vagrant-proxyconf and then set the VAGRANT_HTTP_PROXY and VAGRANT_HTTPS_PROXY environment variables before you execute vagrant up. More details are available here: https://github.com/tmatilai/vagrant-proxyconf/

NOTE: The first time you run this command it may take quite a while to complete (it could take 30 minutes or more depending on your environment) and at times it may look like it’s not doing anything. As long you don’t get any error messages just leave it alone, it’s all good, it’s just cranking.

NOTE to Windows 10 Users: There is a known problem with vagrant on Windows 10 (see mitchellh/vagrant#6754). If the vagrant up command fails it may be because you do not have the Microsoft Visual C++ Redistributable package installed. You can download the missing package at the following address: http://www.microsoft.com/en-us/download/details.aspx?id=8328

Running the unit tests¶

Use the following sequence to run all unit tests

cd $GOPATH/src/github.com/hyperledger/fabric
make unit-test

To run a specific test use the -run RE flag where RE is a regular expression that matches the test case name. To run tests with verbose output use the -v flag. For example, to run the TestGetFoo test case, change to the directory containing the foo_test.go and call/excecute

go test -v -run=TestGetFoo

Running Node.js Unit Tests¶

You must also run the Node.js unit tests to insure that the Node.js client SDK is not broken by your changes. To run the Node.js unit tests, follow the instructions here.

Running Behave BDD Tests¶

Note: currently, the behave tests must be run from within in the Vagrant environment. See the devenv setup instructions if you have not already set up your Vagrant environment.

Behave tests will setup networks of peers with different security and consensus configurations and verify that transactions run properly. To run these tests

cd $GOPATH/src/github.com/hyperledger/fabric
make behave

Some of the Behave tests run inside Docker containers. If a test fails and you want to have the logs from the Docker containers, run the tests with this option:

cd $GOPATH/src/github.com/hyperledger/fabric/bddtests
behave -D logs=Y

Building on Z¶

To make building on Z easier and faster, this script is provided (which is similar to the setup file provided for vagrant). This script has been tested only on RHEL 7.2 and has some assumptions one might want to re-visit (firewall settings, development as root user, etc.). It is however sufficient for development in a personally-assigned VM instance.

To get started, from a freshly installed OS:

sudo su
yum install git
mkdir -p $HOME/git/src/github.com/hyperledger
cd $HOME/git/src/github.com/hyperledger
git clone http://gerrit.hyperledger.org/r/fabric
source fabric/devenv/setupRHELonZ.sh

From this point, you can proceed as described above for the Vagrant development environment.

cd $GOPATH/src/github.com/hyperledger/fabric
make peer unit-test behave

Building on Power Platform¶

Development and build on Power (ppc64le) systems is done outside of vagrant as outlined here. For ease of setting up the dev environment on Ubuntu, invoke this script as root. This script has been validated on Ubuntu 16.04 and assumes certain things (like, development system has OS repositories in place, firewall setting etc) and in general can be improvised further.

To get started on Power server installed with Ubuntu, first ensure you have properly setup your Host’s GOPATH environment variable. Then, execute the following commands to build the fabric code:

mkdir -p $GOPATH/src/github.com/hyperledger
cd $GOPATH/src/github.com/hyperledger
git clone http://gerrit.hyperledger.org/r/fabric
sudo ./fabric/devenv/setupUbuntuOnPPC64le.sh
cd $GOPATH/src/github.com/hyperledger/fabric
make dist-clean all

Creating a Linux Foundation ID¶

1. Go to the Linux Foundation ID website.
2. Select the option I need to create a Linux Foundation ID.
3. Fill out the form that appears:
4. Open your email account and look for a message with the subject line: “Validate your Linux Foundation ID email”.
5. Open the received URL to validate your email address.
6. Verify the browser displays the message You have successfully    validated your e-mail address.
7. Access Gerrit by selecting Sign In:
8. Use your Linux Foundation ID to Sign In:

Configuring Gerrit to Use SSH¶

Gerrit uses SSH to interact with your Git client. A SSH private key needs to be generated on the development machine with a matching public key on the Gerrit server.

If you already have a SSH key-pair, skip this section.

As an example, we provide the steps to generate the SSH key-pair on a Linux environment. Follow the equivalent steps on your OS.

1. Create a key-pair, enter:

ssh-keygen -t rsa -C "John Doe john.doe@example.com"

Note: This will ask you for a password to protect the private key as it generates a unique key. Please keep this password private, and DO NOT enter a blank password.

The generated key-pair is found in: ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub.

1. Add the private key in the id_rsa file in your key ring, e.g.:

ssh-add ~/.ssh/id_rsa

Once the key-pair has been generated, the public key must be added to Gerrit.

Follow these steps to add your public key id_rsa.pub to the Gerrit account:

1. Go to Gerrit.
2. Click on your account name in the upper right corner.
3. From the pop-up menu, select Settings.
4. On the left side menu, click on SSH Public Keys.
5. Paste the contents of your public key ~/.ssh/id_rsa.pub and click Add key.

Note: The id_rsa.pub file can be opened with any text editor. Ensure that all the contents of the file are selected, copied and pasted into the Add SSH key window in Gerrit.

Note: The ssh key generation instructions operate on the assumtion that you are using the default naming. It is possible to generate multiple ssh Keys and to name the resulting files differently. See the ssh-keygen documentation for details on how to do that. Once you have generated non-default keys, you need to configure ssh to use the correct key for Gerrit. In that case, you need to create a ~/.ssh/config file modeled after the one below.

host gerrit.hyperledger.org
 HostName gerrit.hyperledger.org
 IdentityFile ~/.ssh/id_rsa_hyperledger_gerrit
 User <LFID>

where is your Linux Foundation ID and the value of IdentityFile is the name of the public key file you generated.

Warning: Potential Security Risk! Do not copy your private key ~/.ssh/id_rsa Use only the public ~/.ssh/id_rsa.pub.

Checking Out the Source Code¶

1. Ensure that SSH has been set up properly. See Configuring Gerrit to Use SSH for details.
2. Clone the repository with your Linux Foundation ID ():

git clone ssh://<LFID>@gerrit.hyperledger.org:29418/fabric fabric

You have successfully checked out a copy of the source code to your local machine.

Git-review¶

There’s a very useful tool for working with Gerrit called git-review. This command-line tool can automate most of the ensuing sections for you. Of course, reading the information below is also highly recommended so that you understand what’s going on behind the scenes.

Sandbox project¶

We have created a sandbox project to allow developers to familiarize themselves with Gerrit and our workflows. Please do feel free to use this project to experiment with the commands and tools, below.

Getting deeper into Gerrit¶

A comprehensive walk-through of Gerrit is beyond the scope of this document. There are plenty of resources available on the Internet. A good summary can be found here. We have also provided a set of Best Practices that you may find helpful.

Working with a local clone of the repository¶

To work on something, whether a new feature or a bugfix:

1. Open the Gerrit Projects page

2. Select the project you wish to work on.

3. Open a terminal window and clone the project locally using the Clone with git hook URL. Be sure that ssh is also selected, as this will make authentication much simpler:

   git clone ssh://LFID@gerrit.hyperledger.org:29418/fabric && scp -p -P 29418 LFID@gerrit.hyperledger.org:hooks/commit-msg fabric/.git/hooks/
   

Note: if you are cloning the fabric project repository, you will want to clone it to the $GOPATH/src/github.com/hyperledger directory so that it will build, and so that you can use it with the Vagrant development environment.

4. Create a descriptively-named branch off of your cloned repository

cd fabric
git checkout -b issue-nnnn

5. Commit your code. For an in-depth discussion of creating an effective commit, please read this document.

git commit -s -a

Then input precise and readable commit msg and submit.

6. Any code changes that affect documentation should be accompanied by corresponding changes (or additions) to the documentation and tests. This will ensure that if the merged PR is reversed, all traces of the change will be reversed as well.

Submitting a Change¶

Currently, Gerrit is the only method to submit a change for review. Please review the `guidelines <changes.md>`__ for making and submitting a change.

[remote "gerrit"]
    url = ssh://<USERNAME>@gerrit.hyperledger.org:29418/fabric.git
    fetch = +refs/heads/*:refs/remotes/gerrit/*

$ cd <your code dir>
$ git review

cd <your clone dir>
git push origin HEAD:refs/for/master

Counting objects: 3, done.
Writing objects: 100% (3/3), 306 bytes | 0 bytes/s, done.
Total 3 (delta 0), reused 0 (delta 0)
remote: Processing changes: new: 1, refs: 1, done
remote:
remote: New Changes:
remote:   https://gerrit.hyperledger.org/r/6 Test commit
remote:
To ssh://LFID@gerrit.hyperledger.org:29418/fabric
* [new branch]      HEAD -> refs/for/master

Adding reviewers¶

Optionally, you can add reviewers to your change.

To specify a list of reviewers via the command line, add %r=reviewer@project.org to your push command. For example:

git push origin HEAD:refs/for/master%r=rev1@email.com,r=rev2@notemail.com

Alternatively, you can auto-configure GIT to add a set of reviewers if your commits will have the same reviewers all at the time.

To add a list of default reviewers, open the :file:.git/config file in the project directory and add the following line in the [ branch “master” ] section:

[branch "master"] #.... push =
HEAD:refs/for/master%r=rev1@email.com,r=rev2@notemail.com`

Make sure to use actual email addresses instead of the @email.com and @notemail.com addressses. Don’t forget to replace origin with your git remote name.