Mastering Bitcoin 2nd .pdf – 2n d Ed iti on Mastering Bitcoin PROGRAMMING THE OPEN BLOCKCHAIN Andreas M. | Course Hero

Unformatted text preview: 2n
d
Ed
iti
on Mastering Bitcoin
PROGRAMMING THE OPEN BLOCKCHAIN Andreas M. Antonopoulos
SECOND EDITION Mastering Bitcoin Programming the Open Blockchain Andreas M. Antonopoulos Beijing Boston Farnham Sebastopol Tokyo Mastering Bitcoin
by Andreas M. Antonopoulos
Copyright © 2017 Andreas M. Antonopoulos, LLC. All rights reserved.
Printed in the United States of America.
Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.
O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are
also available for most titles ( ). For more information, contact our corporate/insti‐
tutional sales department: 800-998-9938 or [email protected] Editor: Tim McGovern
Production Editor: Nicholas Adams
Copyeditor: Kim Cofer
Proofreader: Christina Edwards Indexer: Judy McConville
Interior Designer: David Futato
Cover Designer: Randy Comer
Illustrator: Rebecca Demarest Second Edition June 2017: Revision History for the Second Edition
2017-06-01:
2017-07-21: First Release
Second Release See for release details.
The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Mastering Bitcoin, the cover image, and
related trade dress are trademarks of O’Reilly Media, Inc.
While the publisher and the author have used good faith efforts to ensure that the information and
instructions contained in this work are accurate, the publisher and the author disclaim all responsibility
for errors or omissions, including without limitation responsibility for damages resulting from the use of
or reliance on this work. Use of the information and instructions contained in this work is at your own
risk. If any code samples or other technology this work contains or describes is subject to open source
licenses or the intellectual property rights of others, it is your responsibility to ensure that your use
thereof complies with such licenses and/or rights. 978-1-491-95438-6
[LSI] Dedicated to my mum, Theresa (1946–2017)
She taught me to love books and question authority
Thank you, mum Table of Contents Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
Quick Glossary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii
1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
What Is Bitcoin?
History of Bitcoin
Bitcoin Uses, Users, and Their Stories
Getting Started
Choosing a Bitcoin Wallet
Quick Start
Getting Your First Bitcoin
Finding the Current Price of Bitcoin
Sending and Receiving Bitcoin 1
4
5
6
6
8
10
11
12 2. How Bitcoin Works. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Transactions, Blocks, Mining, and the Blockchain
Bitcoin Overview
Buying a Cup of Coffee
Bitcoin Transactions
Transaction Inputs and Outputs
Transaction Chains
Making Change
Common Transaction Forms
Constructing a Transaction
Getting the Right Inputs
Creating the Outputs
Adding the Transaction to the Ledger
Bitcoin Mining
Mining Transactions in Blocks 15
15
16
18
18
19
20
21
22
22
24
25
26
27 v Spending the Transaction 29 3. Bitcoin Core: The Reference Implementation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Bitcoin Development Environment
Compiling Bitcoin Core from the Source Code
Selecting a Bitcoin Core Release
Configuring the Bitcoin Core Build
Building the Bitcoin Core Executables
Running a Bitcoin Core Node
Running Bitcoin Core for the First Time
Configuring the Bitcoin Core Node
Bitcoin Core Application Programming Interface (API)
Getting Information on the Bitcoin Core Client Status
Exploring and Decoding Transactions
Exploring Blocks
Using Bitcoin Core’s Programmatic Interface
Alternative Clients, Libraries, and Toolkits
C/C++
JavaScript
Java
Python
Ruby
Go
Rust
C#
Objective-C 32
32
33
34
37
38
39
39
43
44
45
47
48
51
52
52
52
52
53
53
53
53
53 4. Keys, Addresses. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Introduction
Public Key Cryptography and Cryptocurrency
Private and Public Keys
Private Keys
Public Keys
Elliptic Curve Cryptography Explained
Generating a Public Key
Bitcoin Addresses
Base58 and Base58Check Encoding
Key Formats
Implementing Keys and Addresses in Python
Advanced Keys and Addresses
Encrypted Private Keys (BIP-38)
Pay-to-Script Hash (P2SH) and Multisig Addresses vi | Table of Contents 55
56
57
58
60
60
63
64
66
70
76
80
80
81 Vanity Addresses
Paper Wallets 82
88 5. Wallets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
Wallet Technology Overview
Nondeterministic (Random) Wallets
Deterministic (Seeded) Wallets
HD Wallets (BIP-32/BIP-44)
Seeds and Mnemonic Codes (BIP-39)
Wallet Best Practices
Using a Bitcoin Wallet
Wallet Technology Details
Mnemonic Code Words (BIP-39)
Creating an HD Wallet from the Seed
Using an Extended Public Key on a Web Store 93
94
95
96
97
97
98
99
99
106
110 6. Transactions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
Introduction
Transactions in Detail
Transactions—Behind the Scenes
Transaction Outputs and Inputs
Transaction Outputs
Transaction Inputs
Transaction Fees
Adding Fees to Transactions
Transaction Scripts and Script Language
Turing Incompleteness
Stateless Verification
Script Construction (Lock + Unlock)
Pay-to-Public-Key-Hash (P2PKH)
Digital Signatures (ECDSA)
How Digital Signatures Work
Verifying the Signature
Signature Hash Types (SIGHASH)
ECDSA Math
The Importance of Randomness in Signatures
Bitcoin Addresses, Balances, and Other Abstractions 117
117
118
119
121
123
126
129
131
131
132
132
136
138
139
141
141
143
145
145 7. Advanced Transactions and Scripting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
Introduction
Multisignature
Pay-to-Script-Hash (P2SH) 149
149
151 Table of Contents | vii P2SH Addresses
Benefits of P2SH
Redeem Script and Validation
Data Recording Output (RETURN)
Timelocks
Transaction Locktime (nLocktime)
Check Lock Time Verify (CLTV)
Relative Timelocks
Relative Timelocks with nSequence
Relative Timelocks with CSV
Median-Time-Past
Timelock Defense Against Fee Sniping
Scripts with Flow Control (Conditional Clauses)
Conditional Clauses with VERIFY Opcodes
Using Flow Control in Scripts
Complex Script Example 153
154
154
155
157
157
158
160
160
162
162
163
164
165
166
167 8. The Bitcoin Network. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
Peer-to-Peer Network Architecture
Node Types and Roles
The Extended Bitcoin Network
Bitcoin Relay Networks
Network Discovery
Full Nodes
Exchanging “Inventory”
Simplified Payment Verification (SPV) Nodes
Bloom Filters
How Bloom Filters Work
How SPV Nodes Use Bloom Filters
SPV Nodes and Privacy
Encrypted and Authenticated Connections
Tor Transport
Peer-to-Peer Authentication and Encryption
Transaction Pools 171
172
173
176
176
180
181
183
185
186
189
190
191
191
191
192 9. The Blockchain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
Introduction
Structure of a Block
Block Header
Block Identifiers: Block Header Hash and Block Height
The Genesis Block
Linking Blocks in the Blockchain viii | Table of Contents 195
196
197
197
198
200 Merkle Trees
Merkle Trees and Simplified Payment Verification (SPV)
Bitcoin’s Test Blockchains
Testnet—Bitcoin’s Testing Playground
Segnet—The Segregated Witness Testnet
Regtest—The Local Blockchain
Using Test Blockchains for Development 201
207
207
208
210
210
211 10. Mining and Consensus. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
Introduction
Bitcoin Economics and Currency Creation
Decentralized Consensus
Independent Verification of Transactions
Mining Nodes
Aggregating Transactions into Blocks
The Coinbase Transaction
Coinbase Reward and Fees
Structure of the Coinbase Transaction
Coinbase Data
Constructing the Block Header
Mining the Block
Proof-of-Work Algorithm
Target Representation
Retargeting to Adjust Difficulty
Successfully Mining the Block
Validating a New Block
Assembling and Selecting Chains of Blocks
Blockchain Forks
Mining and the Hashing Race
The Extra Nonce Solution
Mining Pools
Consensus Attacks
Changing the Consensus Rules
Hard Forks
Hard Forks: Software, Network, Mining, and Chain
Diverging Miners and Difficulty
Contentious Hard Forks
Soft Forks
Criticisms of Soft Forks
Soft Fork Signaling with Block Version
BIP-34 Signaling and Activation
BIP-9 Signaling and Activation 213
215
217
218
219
220
221
223
224
225
227
228
228
235
235
237
238
239
240
247
249
250
253
256
256
258
259
260
261
262
262
263
264 Table of Contents | ix Consensus Software Development 266 11. Bitcoin Security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
Security Principles
Developing Bitcoin Systems Securely
The Root of Trust
User Security Best Practices
Physical Bitcoin Storage
Hardware Wallets
Balancing Risk
Diversifying Risk
Multisig and Governance
Survivability
Conclusion 269
270
271
272
273
273
273
274
274
274
274 12. Blockchain Applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
Introduction
Building Blocks (Primitives)
Applications from Building Blocks
Colored Coins
Using Colored Coins
Issuing Colored Coins
Colored Coins Transactions
Counterparty
Payment Channels and State Channels
State Channels—Basic Concepts and Terminology
Simple Payment Channel Example
Making Trustless Channels
Asymmetric Revocable Commitments
Hash Time Lock Contracts (HTLC)
Routed Payment Channels (Lightning Network)
Basic Lightning Network Example
Lightning Network Transport and Routing
Lightning Network Benefits
Conclusion 275
276
278
278
279
280
280
283
284
285
286
289
292
296
297
298
301
303
304 A. The Bitcoin Whitepaper by Satoshi Nakamoto. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305 x | Table of Contents B. Transaction Script Language Operators, Constants, and Symbols. . . . . . . . . . . . . . . . . 317
C. Bitcoin Improvement Proposals. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323
D. Segregated Witness. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
E. Bitcore. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343
F. pycoin, ku, and tx. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347
G. Bitcoin Explorer (bx) Commands. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357
Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361 Table of Contents | xi Preface Writing the Bitcoin Book
I first stumbled upon bitcoin in mid-2011. My immediate reaction was more or less
“Pfft! Nerd money!” and I ignored it for another six months, failing to grasp its
importance. This is a reaction that I have seen repeated among many of the smartest
people I know, which gives me some consolation. The second time I came across bit‐
coin, in a mailing list discussion, I decided to read the whitepaper written by Satoshi
Nakamoto to study the authoritative source and see what it was all about. I still
remember the moment I finished reading those nine pages, when I realized that bit‐
coin was not simply a digital currency, but a network of trust that could also provide
the basis for so much more than just currencies. The realization that “this isn’t money,
it’s a decentralized trust network,” started me on a four-month journey to devour
every scrap of information about bitcoin I could find. I became obsessed and enthral‐
led, spending 12 or more hours each day glued to a screen, reading, writing, coding,
and learning as much as I could. I emerged from this state of fugue, more than 20
pounds lighter from lack of consistent meals, determined to dedicate myself to work‐
ing on bitcoin.
Two years later, after creating a number of small startups to explore various bitcoinrelated services and products, I decided that it was time to write my first book. Bit‐
coin was the topic that had driven me into a frenzy of creativity and consumed my
thoughts; it was the most exciting technology I had encountered since the internet. It
was now time to share my passion about this amazing technology with a broader
audience. Intended Audience
This book is mostly intended for coders. If you can use a programming language, this
book will teach you how cryptographic currencies work, how to use them, and how
to develop software that works with them. The first few chapters are also suitable as
xiii an in-depth introduction to bitcoin for noncoders—those trying to understand the
inner workings of bitcoin and cryptocurrencies. Why Are There Bugs on the Cover?
The leafcutter ant is a species that exhibits highly complex behavior in a colony
super-organism, but each individual ant operates on a set of simple rules driven by
social interaction and the exchange of chemical scents (pheromones). Per Wikipedia:
“Next to humans, leafcutter ants form the largest and most complex animal societies
on Earth.” Leafcutter ants don’t actually eat leaves, but rather use them to farm a fun‐
gus, which is the central food source for the colony. Get that? These ants are farming!
Although ants form a caste-based society and have a queen for producing offspring,
there is no central authority or leader in an ant colony. The highly intelligent and
sophisticated behavior exhibited by a multimillion-member colony is an emergent
property from the interaction of the individuals in a social network.
Nature demonstrates that decentralized systems can be resilient and can produce
emergent complexity and incredible sophistication without the need for a central
authority, hierarchy, or complex parts.
Bitcoin is a highly sophisticated decentralized trust network that can support myriad
financial processes. Yet, each node in the bitcoin network follows a few simple mathe‐
matical rules. The interaction between many nodes is what leads to the emergence of
the sophisticated behavior, not any inherent complexity or trust in any single node.
Like an ant colony, the bitcoin network is a resilient network of simple nodes follow‐
ing simple rules that together can do amazing things without any central coordina‐
tion. Conventions Used in This Book
The following typographical conventions are used in this book:
Italic
Indicates new terms, URLs, email addresses, filenames, and file extensions.
Constant width Used for program listings, as well as within paragraphs to refer to program ele‐
ments such as variable or function names, databases, data types, environment
variables, statements, and keywords.
Constant width bold Shows commands or other text that should be typed literally by the user. xiv | Preface Constant width italic Shows text that should be replaced with user-supplied values or by values deter‐
mined by context.
This icon signifies a tip or suggestion. This icon signifies a general note. This icon indicates a warning or caution. Code Examples
The examples are illustrated in Python, C++, and using the command line of a Unixlike operating system such as Linux or macOS. All code snippets are available in the
Github repository ( ) in the code subdirec‐
tory of the main repo. Fork the book code, try the code examples, or submit correc‐
tions via GitHub.
All the code snippets can be replicated on most operating systems with a minimal
installation of compilers and interpreters for the corresponding languages. Where
necessary, we provide basic installation instructions and step-by-step examples of the
output of those instructions.
Some of the code snippets and code output have been reformatted for print. In all
such cases, the lines have been split by a backslash (\) character, followed by a newline
character. When transcribing the examples, remove those two characters and join the
lines again and you should see identical results as shown in the example.
All the code snippets use real values and calculations where possible, so that you can
build from example to example and see the same results in any code you write to cal‐
culate the same values. For example, the private keys and corresponding public keys
and addresses are all real. The sample transactions, blocks, and blockchain references
have all been introduced in the actual bitcoin blockchain and are part of the public
ledger, so you can review them on any bitcoin system.
Preface | xv Using Code Examples
This book is here to help you get your job done. In general, if example code is offered
with this book, you may use it in your programs and documentation. You do not
need to contact us for permission unless you’re reproducing a significant portion of
the code. For example, writing a program that uses several chunks of code from this
book does not require permission. Selling or distributing a CD-ROM of examples
from O’Reilly books does require permission. Answering a question by citing this
book and quoting example code does not require permission. Incorporating a signifi‐
cant amount of example code from this book into your product’s documentation does
require permission.
We appreciate, but do not require, attribution. An attribution usually includes the
title, author, publisher, and ISBN. For example: “Mastering Bitcoin by Andreas M.
Antonopoulos (O’Reilly). Copyright 2017 Andreas M. Antonopoulos,
978-1-491-95438-6.”
Some editions of this book are offered under an open source license, such as CC-BYNC, in which case the terms of that license apply.
If you feel your use of code examples falls outside fair use or the permission given
above, feel free to contact us at [email protected] Bitcoin Addresses and Transactions in This Book
The bitcoin addresses, transactions, keys, QR codes, and blockchain data used in this
book are, for the most part, real. That means you can browse the blockchain, look at
the transactions offered as examples, retrieve them with your own scripts or pro‐
grams, etc.
However, note that the private keys used to construct addresses are either printed in
this book, or have been “burned.” That means that if you send money to any of these
addresses, the money will either be lost forever, or in some cases everyone who can
read the book can take it using the private keys printed in here.
DO NOT SEND MONEY TO ANY OF THE ADDRESSES IN
THIS BOOK. Your money will be taken by another reader, or lost
forever. xvi | Preface O’Reilly Safari
Safari (formerly Safari Books Online) is a membership-based
training and reference platform for enterprise, government,
educators, and individuals.
Members have access to thousands of books, training videos, Learning Paths, interac‐
tive tutorials, and curated playlists from over 250 publishers, including O’Reilly
Media, Harvard Business Review, Prentice Hall Professional, Addison-Wesley Profes‐
sional, Microsoft Press, Sams, Que, Peachpit Press, Adobe, Focal Press, Cisco Press,
John Wiley & Sons, Syngress, Morgan Kaufmann, IBM Redbooks, Packt, Adobe
Press, FT Press, Apress, Manning, New Riders, McGraw-Hill, Jones & Bartlett, and
Course Technology, among others.
For more information, please visit . How to Contact Us
Please address comments and question…

View
Full Document