Merge pull request #44 from Samourai-Wallet/develop

merge develop into master for v1.1.0

.gitignore

 db-scripts/updates/
+db-scripts/1_db.sql
+db-scripts/2_update.sql
+docker/my-dojo/conf/docker-bitcoind.conf
+docker/my-dojo/conf/docker-mysql.conf
+docker/my-dojo/conf/docker-node.conf
 keys/index.js
 keys/sslcert/
 node_modules/
 private-tests/
 *.log
-package-lock.json

README.md

@@ -71,7 +71,10 @@ Authentication is enforced by an API key and Json Web Tokens.
   * Edit /keys/index.js and set "explorers.bitcoind" to "inactive".
 
 * Main drawbacks of using your local bitcoind for these imports:
-  * It doesn't return the full transactional history associated to the HD account but only transactions having an unspent output controlled by the HD account.
-  * It's slightly slower than using the option relying on the OXT API.
-  * In some specific cases, the importer might miss the most recent unspent outputs. Higher values of gap.external and gap.internal in /keys/index.js should help to mitigate this issue. Another workaround is to request the endpoint /support/xpub/.../rescan provided by the REST API with the optional gap parameter.
   * This option is considered as experimental. 
+  * It doesn't return the full transactional history associated to an HD account or to an address but only transactions having an unspent output controlled by the HD account or the address.
+  * It's slightly slower than using the option relying on the OXT API.
+  * It may fail to correctly import an existing wallet if this wallet had a large activity.
+  * If you use bitcoind and if the import seems to return an invalid balance, you can use the "XPUB rescan" function provided by the maintenance tool. This function allows you to force the minimum number of addresses to be derived and the start index for the derivation.
+  * As a rule of thumb, we recommend to use bitcoind as the source of imports and to setup your Dojo with a new clean wallet. It increases your privacy and it removes all potential issues with the import of a large wallet.
+  

RELEASES.md

+# Release Notes
+
+
+## Samourai Dojo v1.1.0 ##
+
+
+### Notable changes ###
+
+
+#### Upgrade mechanism ####
+
+An upgrade mechanism for MyDojo has been added.
+
+See this [doc](./doc/DOCKER_setup.md#upgrade) for more details.
+
+
+#### Optional support of an external bitcoin full node ####
+
+Optional support of an existing Bitcoin full node running outside of Docker has been added.
+
+This setup can be configured thanks to new options defined in ./docker/my-dojo/conf/docker-bitcoind.conf. When this option is activated, the install command skips the installation of bitcoind in Docker.
+
+Note: The Bitcoin full node installed by MyDojo is configured for taking care of your privacy at a network level. You may lose the benefits provided by the default setup if your external full node isn't properly configured. Use at your own risk.
+
+See this [doc](./doc/DOCKER_advanced_setups.md#external_bitcoind) for more details.
+
+
+#### Optional support of external apps ####
+
+New options defined in ./docker/my-dojo/conf/docker-bitcoind.conf allow to expose the RPC API and ZMQ notifications provided by the full node of MyDojo to applications runnnig outside of Docker.
+
+Note: Exposing the full node of MyDojo to external applications may damage your privacy. Use at your own risk.
+
+See this [doc](./doc/DOCKER_advanced_setups.md#exposed_rpc_zmq) for more details.
+
+
+#### Optional support of a static onion address for the full node ####
+
+A new option defined in ./docker/my-dojo/conf/docker-bitcoind.conf allows to keep a static onion address for your full node.
+
+By default, MyDojo generates a new onion address at each startup. We recommend to keep this default setup for better privacy.
+
+See this [doc](./doc/DOCKER_advanced_setups.md#static_onion) for more details.
+
+
+#### Clean-up of Docker images ####
+
+A new "clean" command has been added to Dojo shell script for deleting old Docker images of MyDojo.
+
+This command allows to free disk space on the Docker host.
+
+
+#### Documentation ####
+
+Added a new [doc](./doc/DOCKER_advanced_setups.md) for advanced setups.
+
+Added a new [doc](./doc/DOCKER_mac_setup.MD) for MacOS users.
+
+
+### Change log ###
+
+#### MyDojo ####
+
+- [#1](https://github.com/Samourai-Wallet/samourai-dojo/pull/1) my-dojo upgrade mechanism
+- [#7](https://github.com/Samourai-Wallet/samourai-dojo/pull/7) support of inbound connections through Tor
+- [#8](https://github.com/Samourai-Wallet/samourai-dojo/pull/8) add config option exposing the rpc api and zmq notifications to external apps
+- [#10](https://github.com/Samourai-Wallet/samourai-dojo/pull/10) add an option allowing to run dojo on top of an external bitcoind
+- [#11](https://github.com/Samourai-Wallet/samourai-dojo/pull/11) clean-up
+- [#12](https://github.com/Samourai-Wallet/samourai-dojo/pull/12) extend support of external apps
+- [#15](https://github.com/Samourai-Wallet/samourai-dojo/pull/15) fix issue introduced by #10
+- [#19](https://github.com/Samourai-Wallet/samourai-dojo/pull/19) fix bitcoind port in torrc
+- [#20](https://github.com/Samourai-Wallet/samourai-dojo/pull/20) increase nginx timeout
+- [#25](https://github.com/Samourai-Wallet/samourai-dojo/pull/25) force the tracker to derive next indices if a hole is detected
+- [#27](https://github.com/Samourai-Wallet/samourai-dojo/pull/27) rework external loop of Orchestrator
+- [#28](https://github.com/Samourai-Wallet/samourai-dojo/pull/28) rework RemoteImporter
+- [#32](https://github.com/Samourai-Wallet/samourai-dojo/pull/32) change the conditions switching the startup mode of the tracker
+- [#33](https://github.com/Samourai-Wallet/samourai-dojo/pull/33) check authentication with admin key 
+- [#37](https://github.com/Samourai-Wallet/samourai-dojo/pull/37) automatic redirect of onion address to maintenance tool
+- [#38](https://github.com/Samourai-Wallet/samourai-dojo/pull/38) dojo shutdown - replace sleep with static delay by docker wait
+
+
+#### Security ####
+
+- [#5](https://github.com/Samourai-Wallet/samourai-dojo/pull/5) mydojo - install nodejs
+- [#6](https://github.com/Samourai-Wallet/samourai-dojo/pull/6) remove deprecated "new Buffer" in favor of "Buffer.from"
+- [#41](https://github.com/Samourai-Wallet/samourai-dojo/pull/41) update nodejs packages
+
+
+#### Documentation ####
+
+- [#13](https://github.com/Samourai-Wallet/samourai-dojo/pull/13) included Mac instructions
+- [92097d8](https://github.com/Samourai-Wallet/samourai-dojo/commit/92097d8ec7f9488ce0318c452356994315f4be72) doc
+- [de4c9b5](https://github.com/Samourai-Wallet/samourai-dojo/commit/de4c9b5e5078b673c7b199503d48e7ceca328285) doc - minor updates 
+- [fead0bb](https://github.com/Samourai-Wallet/samourai-dojo/commit/fead0bb4b2b6174e637f5cb8c57edd9b55c3a1c7) doc - add link to MacOS install doc
+- [#42](https://github.com/Samourai-Wallet/samourai-dojo/pull/42) fix few typos, add backticks for config values
+- [#43](https://github.com/Samourai-Wallet/samourai-dojo/pull/43) add missing `d` in `docker-bitcoind.conf`
+
+
+#### Misc ####
+
+- [a382e42](https://github.com/Samourai-Wallet/samourai-dojo/commit/a382e42469b884d2eda9fa6f5a3c8ce93a7cd39a) add sql scripts and config files to gitignore 
+
+
+### Credits ###
+
+- 05nelsonm
+- clarkmoody
+- dergigi
+- hkjn
+- kenshin-samourai
+- LaurentMT
+- michel-foucault
+- pxsocs
+- Technifocal

accounts/api-helper.js

@@ -28,10 +28,10 @@ class ApiHelper {
    */
   parseEntities(str) {
     const ret = new WalletEntities()
-    
+
     if (typeof str !== 'string')
       return ret
-    
+
     for (let item of str.split('|')) {
       try {
 
@@ -47,7 +47,7 @@ class ApiHelper {
 
         } else if (addrHelper.isSupportedPubKey(item) && !ret.hasPubKey(item)) {
           // Derive pubkey as 3 addresses (P1PKH, P2WPKH/P2SH, BECH32)
-          const bufItem = new Buffer(item, 'hex')
+          const bufItem = Buffer.from(item, 'hex')
 
           const funcs = [
             addrHelper.p2pkhAddress,

db-scripts/1_db.sql → db-scripts/1_db.sql.tpl

@@ -25,10 +25,9 @@
 -- Table structure for table `addresses`
 --
 
-DROP TABLE IF EXISTS `addresses`;
 /*!40101 SET @saved_cs_client     = @@character_set_client */;
 /*!40101 SET character_set_client = utf8 */;
-CREATE TABLE `addresses` (
+CREATE TABLE IF NOT EXISTS `addresses` (
   `addrID` int(10) unsigned NOT NULL AUTO_INCREMENT,
   `addrAddress` varchar(74) DEFAULT NULL,
   PRIMARY KEY (`addrID`),
@@ -40,10 +39,9 @@ CREATE TABLE `addresses` (
 -- Table structure for table `banned_addresses`
 --
 
-DROP TABLE IF EXISTS `banned_addresses`;
 /*!40101 SET @saved_cs_client     = @@character_set_client */;
 /*!40101 SET character_set_client = utf8 */;
-CREATE TABLE `banned_addresses` (
+CREATE TABLE IF NOT EXISTS `banned_addresses` (
   `bannedAddressId` int(11) NOT NULL AUTO_INCREMENT,
   `addrAddress` varchar(35) NOT NULL,
   PRIMARY KEY (`bannedAddressId`),
@@ -55,10 +53,9 @@ CREATE TABLE `banned_addresses` (
 -- Table structure for table `blocks`
 --
 
-DROP TABLE IF EXISTS `blocks`;
 /*!40101 SET @saved_cs_client     = @@character_set_client */;
 /*!40101 SET character_set_client = utf8 */;
-CREATE TABLE `blocks` (
+CREATE TABLE IF NOT EXISTS `blocks` (
   `blockID` int(10) unsigned NOT NULL AUTO_INCREMENT,
   `blockHash` char(64) NOT NULL DEFAULT '',
   `blockParent` int(10) unsigned DEFAULT NULL,
@@ -76,10 +73,9 @@ CREATE TABLE `blocks` (
 -- Table structure for table `hd`
 --
 
-DROP TABLE IF EXISTS `hd`;
 /*!40101 SET @saved_cs_client     = @@character_set_client */;
 /*!40101 SET character_set_client = utf8 */;
-CREATE TABLE `hd` (
+CREATE TABLE IF NOT EXISTS `hd` (
   `hdID` int(10) unsigned NOT NULL AUTO_INCREMENT,
   `hdXpub` char(112) DEFAULT NULL,
   `hdCreated` int(10) unsigned NOT NULL DEFAULT '0',
@@ -94,10 +90,9 @@ CREATE TABLE `hd` (
 -- Table structure for table `hd_addresses`
 --
 
-DROP TABLE IF EXISTS `hd_addresses`;
 /*!40101 SET @saved_cs_client     = @@character_set_client */;
 /*!40101 SET character_set_client = utf8 */;
-CREATE TABLE `hd_addresses` (
+CREATE TABLE IF NOT EXISTS `hd_addresses` (
   `hdAddrID` int(10) unsigned NOT NULL AUTO_INCREMENT,
   `hdID` int(10) unsigned NOT NULL DEFAULT '0',
   `addrID` int(10) unsigned NOT NULL DEFAULT '0',
@@ -116,10 +111,9 @@ CREATE TABLE `hd_addresses` (
 -- Table structure for table `inputs`
 --
 
-DROP TABLE IF EXISTS `inputs`;
 /*!40101 SET @saved_cs_client     = @@character_set_client */;
 /*!40101 SET character_set_client = utf8 */;
-CREATE TABLE `inputs` (
+CREATE TABLE IF NOT EXISTS `inputs` (
   `inID` int(10) unsigned NOT NULL AUTO_INCREMENT,
   `outID` int(10) unsigned NOT NULL DEFAULT '0',
   `txnID` int(10) unsigned NOT NULL DEFAULT '0',
@@ -138,10 +132,9 @@ CREATE TABLE `inputs` (
 -- Table structure for table `outputs`
 --
 
-DROP TABLE IF EXISTS `outputs`;
 /*!40101 SET @saved_cs_client     = @@character_set_client */;
 /*!40101 SET character_set_client = utf8 */;
-CREATE TABLE `outputs` (
+CREATE TABLE IF NOT EXISTS `outputs` (
   `outID` int(10) unsigned NOT NULL AUTO_INCREMENT,
   `txnID` int(10) unsigned NOT NULL DEFAULT '0',
   `addrID` int(10) unsigned NOT NULL DEFAULT '0',
@@ -161,10 +154,9 @@ CREATE TABLE `outputs` (
 -- Table structure for table `transactions`
 --
 
-DROP TABLE IF EXISTS `transactions`;
 /*!40101 SET @saved_cs_client     = @@character_set_client */;
 /*!40101 SET character_set_client = utf8 */;
-CREATE TABLE `transactions` (
+CREATE TABLE IF NOT EXISTS `transactions` (
   `txnID` int(10) unsigned NOT NULL AUTO_INCREMENT,
   `txnTxid` char(64) DEFAULT NULL,
   `txnCreated` int(10) unsigned NOT NULL DEFAULT '0',
@@ -183,10 +175,9 @@ CREATE TABLE `transactions` (
 -- Table structure for table `scheduled_transactions`
 --
 
-DROP TABLE IF EXISTS `scheduled_transactions`;
 /*!40101 SET @saved_cs_client     = @@character_set_client */;
 /*!40101 SET character_set_client = utf8 */;
-CREATE TABLE `scheduled_transactions` (
+CREATE TABLE IF NOT EXISTS `scheduled_transactions` (
   `schID` int(10) unsigned NOT NULL AUTO_INCREMENT,
   `schTxid` char(64) NOT NULL DEFAULT '',
   `schCreated` int(10) unsigned NOT NULL DEFAULT '0',

db-scripts/2_update.sql.tpl

+# SQL scripts for incremental updates
+
+# Copyright © 2019 – Katana Cryptographic Ltd. All Rights Reserved.
+ 
+--
+-- UPDATES vX.Y.Z
+--

doc/DOCKER_advanced_setups.md

+# MyDojo - Advanced Setups
+
+
+The 3 configuration files of Dojo provide a few advanced options allowing to tune your setup.
+
+A word of caution, though, the default values of these options try to maximize your privacy at a network level. All the advanced setups described in this document may damage your privacy. Use at your own risk!
+
+
+<a name="external_bitcoind"/>
+
+## External Bitcoin full node ##
+
+By default, Dojo installs and runs a Bitcoin full node in Docker.
+
+The following procedure allows to bypass the installation of this full node by telling Dojo to rely on an external bitcoind running on your host machine.
+
+
+### Requirements ###
+
+The external full node mustn't be pruned.
+
+The external full node must be configured for the support of Dojo. Edit the bitcoin.conf file of your external full node and check that the following lines are properly initialized.
+
+```
+# Force bitcoind to accept JSON-RPC commands
+server=1
+
+# Force bitcoind to index all the transactions
+txindex=1
+
+# Check that bitcoind accepts connections from 127.0.0.1 (linux)
+# or from the IP address of the Docker Virtual Machine (MacOS, Windows)
+rpcallowip=... 
+
+# Check that a port is defined for the RPC API (or 8332 will be used as default value)
+rpcport=...
+
+# Check that the RPC API listens on an IP address accessible from the nodejs container
+rpcbind=...
+
+# Check that the RPC user is set
+rpcuser=...
+
+# Check that the RPC password is set
+rpcpassword=
+
+# Enable publish hash block on an IP address accessible from the nodejs container
+zmqpubhashblock=...
+
+# Enable publish raw transaction on an IP address accessible from the nodejs container
+zmqpubrawtx=...
+```
+
+
+### Procedure ###
+
+#### Configuration of Dojo ####
+
+```
+# Edit the bitcoin config template file
+nano ./conf/docker-bitcoind.conf.tpl
+
+#
+# Set the value of BITCOIND_INSTALL to "off"
+# Set the value of BITCOIND_IP with the IP address of you bitcoin full node
+# Set the value of BITCOIND_RPC_PORT with the port used by your bitcoin full node for the RPC API
+# Set the value of BITCOIND_ZMQ_RAWTXS with the port used by your bitcoin full node for ZMQ notifications of raw transactions
+#   (i.e. port defined for -zmqpubrawtx in the bitcoin.conf of your full node)
+# Set the value of BITCOIND_ZMQ_BLK_HASH with the port used by your bitcoin full node for ZMQ notifications of block hashes
+#   (i.e. port defined for -zmqpubhashblock in the bitcoin.conf of your full node)
+#
+# Save and exit nano
+#
+```
+
+### Fast import of block headers in Dojo (optional) ###
+
+When Dojo is installed for the first time, the Tracker imports the block headers in the database.
+
+Follow these steps if you want to speed up this operation by preloading an archive of the block headers.
+
+```
+# Download the archive [https://samouraiwallet.com/static/share/2_blocks.sql.gz](https://samouraiwallet.com/static/share/2_blocks.sql.gz) to the "<dojo_dir>/db-scripts/" directory. Don't modify the name of the archive.
+```
+
+
+#### Start the installation of Dojo ####
+
+```
+./dojo.sh install
+```
+
+
+<a name="exposed_rpc_zmq"/>
+
+## bitcoind RPC API ans ZMQ notifications exposed to external apps ##
+
+By default, access to the RPC API of your bitcoind is restricted to Docker containers hosted on the "dojonet" network.
+
+The following steps allow to expose the RPC API and ZMQ notifications to applications running on your local machine but outside of Docker.
+
+```
+# 
+# If your Docker runs on macos or windows,
+# retrieve the local IP address of the VM
+# hosting your Docker containers
+#
+
+# Stop your Dojo
+./dojo.sh stop
+
+# Edit the bitcoin config file
+nano ./conf/docker-bitcoind.conf
+
+#
+# Set the value of BITCOIND_RPC_EXTERNAL to "on"
+#
+# If your Docker runs on macos or windows,
+# set the value of BITCOIND_RPC_EXTERNAL_IP to the IP address of the VM
+#
+# Save and exit nano
+#
+
+# Start your Dojo
+./dojo.sh start
+```
+
+With this setting, external applications running on your local machine should be able to access the following ports:
+* 9500: bitcoind zmqpubhashtx notifications
+* 9501: bitcoind zmqpubrawtx notifications
+* 9502: bitcoind zmqpubhashblock notifications
+* 9503: bitcoind zmqpubrawblock notifications
+* 28256: bitcoind RPC API
+
+Note: this option has no effect if your setup relies on a external full node (i.e. if BITCOIND_INSTALL is set to "off").
+
+
+<a name="static_onion"/>
+
+## Static onion address for bitcoind hidden service ##
+
+By default, Dojo creates a new onion address for your bitcoind at each startup. 
+
+The following steps allow to keep a static onion address (not recommended).
+
+```
+# Stop your Dojo
+./dojo.sh stop
+
+# Edit the bitcoin config file
+nano ./conf/docker-bitcoind.conf
+
+#
+# Set the value of BITCOIND_EPHEMERAL_HS to "off"
+#
+
+# Start your Dojo
+./dojo.sh start
+```
+
+Note: this option has no effect if your setup relies on a external full node (i.e. if BITCOIND_INSTALL is set to "off").

doc/DOCKER_mac_setup.MD

+# MacOS Installation 
+
+This installation was tested on an iMac (mid 2011) with a 2.7GHz i5 processor with 8GB RAM and 1TB external Hard Drive. 
+
+## Getting Started
+
+####  Create a new user:
+
+1. Launch System Preferences by clicking the  **System Preferences**  icon in the  **Dock**, or selecting  **System Preferences**  from the Apple menu.
+2. Click on __Users & Groups__
+3. If settings are locked, click on the __Lock__ at the bottom of window and enter your password.
+4. Click on __+__ to add a new User
+5. Under __New Account__ select __Administrator__
+6. Fill the remaining fields with your choice of User Name and Password
+
+#### Move the __New User Folder__ into the __External HD__
+Note: _This is an important step, otherwise, it's probable that when you run the container, it will be installed in your main OS Hard Drive and will run out of space as it validates the Bitcoin blockchain._
+
+1. Open  **Finder** and navigate to your startup drive's **/Users** folder. For most people, this is **/Macintosh HD/Users**. In the **Users**  **folder**, you'll find your user's folder.
+2. On your external Hard Drive, create a folder named **Users**.
+3. Select your user folder and drag it to the external HD **/Users** folder you created. _Because you're using a different drive for the destination, the operating system will copy the data rather than move it. This ok for now but delete it later._
+4. Launch System Preferences again. 
+5. In the  **Users & Groups** click the lock icon in the bottom left corner, then provide an administrator name and password.
+6. From the list of user accounts, right-click on the account whose home folder you moved, and select  **Advanced Options**  from the pop-up menu.
+    _Do not make any changes to Advanced Options except for those noted here. Doing so can cause quite a few unforeseen problems that could lead to data loss or the need to reinstall the operating system._
+7.  In the  **Advanced Options**  sheet, click  **Choose**, located to the right of the  **Home directory**  field.    
+8.  Navigate to the location you moved your home folder to, select the new home folder, and click  **OK**.
+9.  Click  **OK**  to dismiss the  **Advanced Options**  sheet, and then close  **System Preferences**.
+10.  __Restart your Mac__ 
+
+#### Download and install Docker, Kitematic and TOR
+1. Make sure your system fills the [requirements]([https://docs.docker.com/docker-for-mac/install/](https://docs.docker.com/docker-for-mac/install/)) (particularly MacOS Sierra 10.12 or higher. If not, upgrade before proceeding).  
+2. [Download Docker]([https://docs.docker.com/docker-for-mac/install/](https://docs.docker.com/docker-for-mac/install/)) and follow the installation steps.
+3. _Optional_: Download [Kitematic]([https://kitematic.com/) and follow installation instructions. 
+(_This may be system specific but I've found that monitoring the logs with Kitematic was more stable than using the Terminal_).
+4. Install [Tor Browser](https://www.torproject.org/projects/torbrowser.html.en) on the host machine.
+
+## Adjust Docker Settings
+1. Click on the Docker icon (![whale menu](https://docs.docker.com/docker-for-mac/images/whale-x.png)) at the status bar and select __Preferences__.
+2.  Under Disk, click on __Reveal in Finder__ and double check that the disk image is saved under the external HD.
+3. __Adjust Disk__ Image size to 400GB+ and click Apply.
+4. Click __Advanced__ and increase the CPU count, Memory and Swap sizes. Adjusting these will speed up the blockchain validation process 
+(_At 4 CPUs, 8GB of RAM and a 4GiB Swap - the initial block download took 4.5 days at the time of writing_).
+
+## Install the DOJO
+Follow the instructions [here](https://github.com/Samourai-Wallet/samourai-dojo/blob/develop/doc/DOCKER_setup.md) starting at the step:
+__"Download the most recent release of Dojo from Github"__
+_Note: For tracking progress, open Kitematic and follow the bitcoind logs. You'll be able to see the Blockchain verification process under the _progress_ log variable (1.00 = fully validated). This process takes a long time. Just let it do its thing. In my system it took 3 days._
+
+__Some possible optimization tips:__
+. If you notice that progress has stopped. Click the whale icon and select Restart. Check Kitematic logs of bitcoind to confirm that progress has resumed. 
+. This may optimize speed: open __Activity Monitor__, check the PID (Process ID) of your docker process. Open Terminal and type:
+
+`sudo renice-20 -p [enter your PID]` 

doc/DOCKER_setup.md

@@ -2,11 +2,25 @@
 
 MyDojo is a set of Docker containers providing a full Samourai backend composed of:
 * a bitcoin full node accessible as an ephemeral Tor hidden service,
-* the backend database,
-* the backend modules with an API accessible as a static Tor hidden service,
+* a backend database,
+* a backend modules with an API accessible as a static Tor hidden service,
 * a maintenance tool accessible through a Tor web browser.
 
 
+## Table of Content ##
+- [Architecture](#architecture)
+- [Requirements](#requirements)
+- [First-time install procedure](#install)
+- [Upgrade procedure](#upgrade)
+- [Configuration files](#config_files)
+- [Dojo shell script](#shell_script)
+- [Dojo maintenance tool](#maintenance_tool)
+- [Pairing your wallet to your Dojo](#pairing)
+- [Network connections](#network)
+
+
+<a name="architecture"/>
+
 ## Architecture ##
 
 
@@ -21,7 +35,7 @@ MyDojo is a set of Docker containers providing a full Samourai backend composed
 
                                           ------------
                                                 |
-                  Host machine                  | (Tor - port 80)
+                  Host machine                  | (Tor hidden services)
                  ______________________________ | _____________________________
                 |                               |                              |
                 |                      -------------------                     |
@@ -42,6 +56,7 @@ MyDojo is a set of Docker containers providing a full Samourai backend composed
                 |______________________________________________________________|
 
 
+<a name="requirements"/>
 
 ## Requirements ##
 
@@ -54,41 +69,52 @@ MyDojo is a set of Docker containers providing a full Samourai backend composed
 * Tor Browser installed on the host machine (or on another machine if your host is a headless server)
 
 
-## Setup ##
+<a name="install"/>
 
-* Install [Docker and Docker Compose](https://docs.docker.com/compose/install/) on the host machine and check that your installation is working.
+## First-time Setup ##
 
-* Install [Tor Browser](https://www.torproject.org/projects/torbrowser.html.en) on the host machine.
+For MacOS, see this detailed [installation guide](./DOCKER_mac_setup.MD).
 
-* Download the most recent version of Dojo from [Github](https://github.com/Samourai-Wallet/samourai-dojo/archive/master.zip)
+This procedure allows to install a new Dojo from scratch.
 
-* Uncompress the archive on the host machine in a temporary directory of your choice (named /tmp_dir in this doc)
+* Install [Docker and Docker Compose](https://docs.docker.com/compose/install/) on the host machine and check that your installation is working.
 
-* Create a directory for Dojo (named /dojo_dir in this doc)
+* Install [Tor Browser](https://www.torproject.org/projects/torbrowser.html.en) on the host machine.
 
-* Copy the content of the "/tmp_dir/samourai-dojo-master" directory into the "/dojo_dir" directory
+* Download the most recent release of Dojo from [Github](https://github.com/Samourai-Wallet/samourai-dojo/archive/master.zip)
 
-* Customize the configuration of your Dojo
+* Uncompress the archive on the host machine in a temporary directory of your choice (named `<tmp_dir>` in this doc)
 
-  * Go to the "/dojo_dir/docker/my_dojo/conf" directory
+* Create a directory for Dojo (named `<dojo_dir>` in this doc)
 
-  * Edit docker-bitcoind.conf and provide a new value for the following parameters:
-      * BITCOIND_RPC_USER = login protecting the access to the RPC API of your full node,
-      * BITCOIND_RPC_PASSWORD = password protecting the access to the RPC API of your full node.
-      * If your machine has a lot of RAM, it's recommended that you increase the value of BITCOIND_DB_CACHE for a faster Initial Block Download.
+* Copy the content of the `<tmp_dir>/samourai-dojo-master` directory into the `<dojo_dir>` directory
 
-  * Edit docker-mysql.conf and provide a new value for the following parameters:
-      * MYSQL_ROOT_PASSWORD = password protecting the root account of MySQL,
-      * MYSQL_USER = login of the account used to access the database of your Dojo,
-      * MYSQL_PASSWORD = password of the account used to access the database of your Dojo.
+* Customize the configuration of your Dojo
 
-  * Edit docker-node.conf and provide a new value for the following parameters:
-      * NODE_API_KEY = API key which will be required from your Samourai Wallet / Sentinel for its interactions with the API of your Dojo,
-      * NODE_ADMIN_KEY = API key which will be required from the maintenance tool for accessing a set of advanced features provided by the API of your Dojo,
-      * NODE_JWT_SECRET = secret used by your Dojo for the initialization of a cryptographic key signing Json Web Tokens.
+  * Go to the `<dojo_dir>/docker/my-dojo/conf` directory
+
+  * Edit docker-bitcoind.conf.tpl and provide a new value for the following parameters:
+      * `BITCOIND_RPC_USER` = login protecting the access to the RPC API of your full node,
+      * `BITCOIND_RPC_PASSWORD` = password protecting the access to the RPC API of your full node.
+      * If your machine has a lot of RAM, it's recommended that you increase the value of `BITCOIND_DB_CACHE` for a faster Initial Block Download.
+      * This file also provides a few additional settings for advanced setups: 
+        * static onion address for your full node,
+        * bitcoind RPC API exposed to external apps,
+        * use of an external full node.
+        See this [doc](./DOCKER_advanced_setups.md) for more details.
+
+  * Edit docker-mysql.conf.tpl and provide a new value for the following parameters:
+      * `MYSQL_ROOT_PASSWORD` = password protecting the root account of MySQL,
+      * `MYSQL_USER` = login of the account used to access the database of your Dojo,
+      * `MYSQL_PASSWORD` = password of the account used to access the database of your Dojo.
+
+  * Edit docker-node.conf.tpl and provide a new value for the following parameters:
+      * `NODE_API_KEY` = API key which will be required from your Samourai Wallet / Sentinel for its interactions with the API of your Dojo,
+      * `NODE_ADMIN_KEY` = API key which will be required from the maintenance tool for accessing a set of advanced features provided by the API of your Dojo,
+      * `NODE_JWT_SECRET` = secret used by your Dojo for the initialization of a cryptographic key signing Json Web Tokens.
     These parameters will protect the access to your Dojo. Be sure to provide alphanumeric values with enough entropy.
 
-* Open the docker quickstart terminal or a terminal console and go to the "/dojo_dir/docker/my_dojo" directory. This directory contains a script named dojo.sh which will be your entrypoint for all operations related to the management of your Dojo.
+* Open the docker quickstart terminal or a terminal console and go to the `<dojo_dir>/docker/my-dojo` directory. This directory contains a script named dojo.sh which will be your entrypoint for all operations related to the management of your Dojo.
 
 
 * Launch the installation of your Dojo with
@@ -118,6 +144,59 @@ Exit the logs with CTRL+C when the syncing of the database has completed.
 * Restrict the access to your host machine as much as possible by configuring its firewall.
 
 
+<a name="upgrade"/>
+
+## Upgrade ##
+
+This procedure allows to upgrade your Dojo with a new version.
+
+* Stop your Dojo with
+
+```
+./dojo.sh stop
+```
+
+* Download the most recent release of Dojo from [Github](https://github.com/Samourai-Wallet/samourai-dojo/releases)
+
+* Uncompress the archive on the host machine in a temporary directory of your choice (named `<tmp_dir>` in this doc)
+
+* Copy the content of the `<tmp_dir>/samourai-dojo-1.x.y` directory into the `<dojo_dir>` directory (overwrite).
+
+* Launch the upgrade of your Dojo with
+