gitian building create vm debian vmware

Download and install latest VMWare (it is free)

Download VMware Workstation from here.

During creation of this guide, we used VMware Workstation 14.1.1 Player for Linux 64-bit (Download v14).

VMware Workstation 14.1.1 Player for Linux 64-bit (bundle | 110.47 MB).

File Name
VMware-Player-14.1.1-7528167.x86_64.bundle
Build Number
7528167
Release Date
2018-01-09
CHECKSUMS
MD5SUM: bef5c16ed5be5fb295b8c2316c104656
SHA1SUM: b643ae221f88d3e7652f3ed99f7a4e1ddf2655fd

Install vmware in linux

Install downloaded VMWare player package with this command:

# make bundle script executable
chmod +x VMware-Player-14.*.x86_64.bundle

# for installation you need to use sudo, as root access is required
sudo ./VMware-Player-14.*.x86_64.bundle

Follow instructions for installation and install default/your settings.

If you already have vmware installed then it will be either updated or nothing will be done:

Prepare your virtual machine and install OS

On first start after fresh installation or after the upgrade of vmware workstation, pop up for compile and load of modules appears. Click on "Install".

Create new VM (Debian)

1. • Launch VMWare workstation and create a new virtual machine, you can use shortcut by pressing "CTRL+N" or clicking on File => Create a New Virtual Machine

   

2. • Choose Debian ISO. We use for this guide version 8.10 which is the latest before openssl v1.1.x. Debian 9 and newer break currently and do not work.

     We will use Debian 8 netinstall iso for this guide.

   

3. • If you have choosen the ISO image, it mostly does automaticly recognise which guest OS should be set. In our cas guest OS is Linux and version is Debian 8.x 64-Bit

   

4. • Give your machine a name, I will just leave it as it was autocreated.

   

5. • Specify disk size. Choose 40GB as minimum. If you compile for all architectures I highly recommend taking 50GB or more. Optional: I do use here splitting files but it is up to you if you choose a single or splitted file.

   

6. • On the settings overview window, click Customze Hardware to customize memory size, ssh port forwarding and cpu amount as well some other settings if you require them.

   

7. • Set memory site to minimum 3072MB.

   

8. • Set cpu amount, you should choose at least two if available on your hardware. In this guide I will use 4.

   

9. • We will use for this guide NAT. Click on Advanced if you want to preset MAC address or to limit bandwith. You can click on Close when finished configuring your hardware.

   Prepare your virtual machine and install OS

10. • Click on finish if you are done with configuring hardware

    

    

Install Debian 8.x

1. • On first boot you will informed about your removable devices.

   

2. • Follow default Debian installation guide from here on.

   Setup Gitian on Debian

Prepare VMWare Debian 8.x

1. • On first OS boot after installation, login is as root with password: cevap. You will have to install sudo, add gitianuser to sudo and create your gpg/ssh keys.

   

2. • Update repositories and install sudo package as it is not installed by default and we require sudo.

   # update repository
   apt update
   
   # install sudo
   apt install -y sudo
   
   # add gitianuser to sudoers
   usermod -aG sudo gitianuser
   

   Let's check/test if gitianuser has sudo access, we will switch to gitianuser and run any command with sudo, I will use sudo apt update to check if it works.

   Switch to gitianuser from root

   su gitianuser
   

   Check if gitianuser has access rights by running any command

   sudo apt update
   

   as we switched to gitianuser we are currently in wrong dir, we will go to our home folder /home/gitianuser. As we switched to gitianuser, its variables are available and we can simply run cd ~/

   # cd to gitianuser's home folder, you could also use $HOME 
   cd ~/
   

3. • Create SSH key for gitianuser (you can follow this guide), I will use here ECDSA key instead of RSA.

   First create your SSH key for gitianuser. Create ssh key, I will use ECDSA key in this guide as well as our projects public address

   ssh-keygen -t ecdsa -b 521 -C "[email protected]"
   

   Add your key to ssh-agent

   eval "$(ssh-agent -s)"
   

   Add your SSH private key to the ssh-agent. If you created your key with a different name, or if you are adding an existing key that has a different name, replace id_rsa in the command with the name of your private key file.

   ssh-add ~/.ssh/id_ecdsa
   

4. • In many cases users receive error that max amount of authentifications has been reached. Please do not run this command if you run this virtual machine on a publicly accessable server.

   We will add MaxAuthTries=30 as workaround for those who receive it. If you do not receive this error connecting over ssh to your VM, skip this step.

   # you need to do it with root user, sudo will not work
   sudo su root
   # We require root permissions, thats why we use sudo here
   echo "MaxAuthTries=30" >> /etc/ssh/sshd_config
   # Show your IP to check if SSH works
   ifconfig
   

   In my case (normally default) this is the the ip which ifconfig shows me: 172.16.72.129, it looks like that:

   inet addr:172.16.72.129
   

5. • On your local pc (not VM) add following lines to your ~/.ssh/config file to make connecting easier as I do not want to remember the IP.

   We will add new host to our ssh config (make sure to use correct key, I use default ecdsa which is id_ecdsa, you can show your keys by running ls ~/.ssh):

   Explanation of ssh config file related settings:

   • host: hostname (this is the hostname which you will use on your local pc for the VM)
   • HostName: IP address (Here you set the IP of your VM, to find your IP, please run sudo ifconfig. If running ifconfig with gitianuser you require sudo)
   • IdentityFile (Path to your ssh private key, ~/.ssh is default location where your keys are stored, look up if you are unsure please.)
   • User: gitianuser (User on VM, we created this user during debian installation)

   cat <<EOF >> ~/.ssh/config
   
   host gitian
    HostName 	172.16.72.129
    IdentityFile ~/.ssh/id_ecdsa
    User gitianuser
   EOF
   

6. • Connect over SSH to your VM

   If you have configured your .ssh/config properly, you will be able to connect over SSH to your VM with following command

   ssh gitian
   

   On first connection ever, you will be asked if you want to connect to this server showing you fingerprins. Type yes and press enter:

   The authenticity of host '172.16.72.129 (172.16.72.129)' can't be established.
   ECDSA key fingerprint is SHA256:7jyABAvN9Xnfo67tLeHMuz3gILRJnF1y1qfdj0SH3x0.
   Are you sure you want to continue connecting (yes/no)? yes
   

   this is the output which I got when I created this guide:

   tor@tor ~ $ ssh gitian
   The authenticity of host '172.16.72.129 (172.16.72.129)' can't be established.
   ECDSA key fingerprint is SHA256:7jyABAvN9Xnfo67tLeHMuz3gILRJnF1y1qfdj0SH3x0.
   Are you sure you want to continue connecting (yes/no)? yes
   Warning: Permanently added '172.16.72.129' (ECDSA) to the list of known hosts.
   no such identity: /home/tor/.ssh/id_ecdsa: No such file or directory
   [email protected]'s password: 
   
   The programs included with the Debian GNU/Linux system are free software;
   the exact distribution terms for each program are described in the
   individual files in /usr/share/doc/*/copyright.
   
   Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
   permitted by applicable law.
   gitianuser@debian:~$ 
   

   Lets test if closing connection works, type following command: exit

   Output for this guide:

   gitianuser@debian:~$ exit
   logout
   Connection to 172.16.72.129 closed.
   tor@tor ~ $ 
   

7. • Open a new terminal tab and enter (check your .ssh folder for file name, id_rsa.pub and id_ecdsa.pub are defaults for RSA and ECDSA keys):

   Add your SSH key from your local machine to authorized keys file on VM.

   # Default filepath for default ECDSA keyfile
   scp ~/.ssh/id_ecdsa.pub gitian:.ssh/authorized_keys
   

   # Default filepath for default RSA keyfile
   scp ~/.ssh/id_rsa.pub gitian:.ssh/authorized_keys
   

   You will be asked for password, default password is cevap. This is the output after adding my RSA key to the authorized key file.

   tor@tor ~ $ scp ~/.ssh/id_rsa.pub gitian:.ssh/authorized_keys
   no such identity: /home/tor/.ssh/id_ecdsa: No such file or directory
   [email protected]'s password: 
   id_rsa.pub                                    100% 1421     1.4KB/s   00:00    
   

8. • Backup your public and private keys with cat ~/.ssh/id_ecdsa.pub and Add that key to your GitHub account.

   You can show your keys and copy and paste them to some editor and save it somewhere. In this guide I will copy that key to my machine and add it to the agent.

   # Copy Private SSH key to "id_ecdsa_gitian_vmware_debian_8.x"
   scp gitian:~/.ssh/id_ecdsa ~/.ssh/id_ecdsa_gitian_vmware_debian_8.x
   # Copy Public SSH key to "id_ecdsa_gitian_vmware_debian_8.x.pub"
   scp gitian:~/.ssh/id_ecdsa.pub ~/.ssh/id_ecdsa_gitian_vmware_debian_8.x.pub
   

   This is the ouput where we can see that files have been copied

   tor@tor ~ $ scp gitian:~/.ssh/id_ecdsa ~/.ssh/id_ecdsa_gitian_vmware_debian_8.x
   id_ecdsa                                      100%  365     0.4KB/s   00:00    
   tor@tor ~ $ scp gitian:~/.ssh/id_ecdsa.pub ~/.ssh/id_ecdsa_gitian_vmware_debian_8.x.pub
   id_ecdsa.pub                                  100%  272     0.3KB/s   00:00 
   

   Thats it, if you look up on your local machine, you will see that your keys are there.

   Now we will add this key

   Optional- add gitian key to your agent

   (_not required for gitian building, we have already added our key to the authorized file just to sort out keys which are used if you split VM's and want to use their keys to connect to those machines)

   Add your gitian key to ssh-agent

   eval "$(ssh-agent -s)"
   ssh-add ~/.ssh/id_ecdsa_gitian_vmware_debian_8.x
   

   Now we will add VM's key which we exported to its VM's gitianuser authorized keys file. VM has already the same keys, you could do this also directly after creating your SSH key.

   scp ~/.ssh/id_ecdsa_gitian_vmware_debian_8.x.pub gitian:.ssh/authorized_keys
   
   

9. • Create GPG key

   In this guide, I will include from beginn a workaround for known issue which many users face. If you do not receive GPG does not have enough entropy or similar message during GPG creation, ignore this step and proceed to the next. If you use high encryption and set your key to 4096bits, you might get some message similar to this GPG does not have enough entropy.

   One of workarounds would be package for generating entropy called haveged. Install haveged and rng-tools by running:

   sudo apt update
   sudo apt install -y haveged rng-tools
   

   Create entropy now. Please read more about this topic and security if you use /dev/random in combination with production server which is publicly life (this should be voided)

   cat /dev/random | rngtest -c 1000
   

   you should receive similar output to this:

   gitianuser@debian:~$ cat /dev/random | rngtest -c 1000
   rngtest 2-unofficial-mt.14
   Copyright (c) 2004 by Henrique de Moraes Holschuh
   This is free software; see the source for copying conditions.  There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
   
   rngtest: starting FIPS tests...
   rngtest: bits received from input: 20000032
   rngtest: FIPS 140-2 successes: 999
   rngtest: FIPS 140-2 failures: 1
   rngtest: FIPS 140-2(2001-10-10) Monobit: 1
   rngtest: FIPS 140-2(2001-10-10) Poker: 0
   rngtest: FIPS 140-2(2001-10-10) Runs: 0
   rngtest: FIPS 140-2(2001-10-10) Long run: 0
   rngtest: FIPS 140-2(2001-10-10) Continuous run: 0
   rngtest: input channel speed: (min=2.089; avg=12.515; max=19073.486)Mibits/s
   rngtest: FIPS tests speed: (min=76.600; avg=144.491; max=183.399)Mibits/s
   rngtest: Program run time: 1656303 microseconds
   

   Create now your GPG key. Run this command and follow instructions to create your key

   gpg --gen-key
   

   First you will be asked what kind of GPG key do you want to create, we will proceed for this guide with RSA key.

   Please select what kind of key you want:
      (1) RSA and RSA (default)
      (2) DSA and Elgamal
      (3) DSA (sign only)
      (4) RSA (sign only)
   Your selection? 1
   

   Set bit lenght for your new GPG key

   RSA keys may be between 1024 and 4096 bits long.
   What keysize do you want? (2048) 4096
   

   Specify how long this key should be valid. For this guide I will set key not to expire because this key is created for public is used by this release of our gitian VM. Please do specify always a validity length for your keys, you can extend them and you should do that because if somebody gains access to the key which never expires, then this person will always be able to use this key for signing/encryption/decryption.

   Please specify how long the key should be valid.
            0 = key does not expire
         <n>  = key expires in n days
         <n>w = key expires in n weeks
         <n>m = key expires in n months
         <n>y = key expires in n years
   Key is valid for? (0) 0
   

   Now you will be asked to fill out a real name, email address and a comment for your gpg key. When you finish, confirm it with pressing O (Okay)

   You need a user ID to identify your key; the software constructs the user ID
   from the Real Name, Comment and Email Address in this form:
       "Heinrich Heine (Der Dichter) <[email protected]>"
   
   Real name: CEVAP
   Email address: [email protected]
   Comment: Ionian Deterministic Building Process - Gitian Building
   You selected this USER-ID:
       "CEVAP (Ionian Deterministic Building Process - Gitian Building) <[email protected]>"
   
   Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? 
   

   As last step, you will be asked to enter your new passphrase. I strongly recommend to use some non online key management system, very usefull tool is keepass which I can only recommend for storing keys/passwords/infos/strings/2fa/... .

   We will not set a password for the key in this guide. Please do set always a password for your key.

   Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O
   You need a Passphrase to protect your secret key.
   
   Enter passphrase: 
   

   Your keys and short ID will be listed just after key creation, this is how it looks like, and ID which is to be used is in a line starting with sec. For this guide the short ID is: 2720E025

   gpg: /home/gitianuser/.gnupg/trustdb.gpg: trustdb created
   gpg: key 2720E025 marked as ultimately trusted
   public and secret key created and signed.
   
   gpg: checking the trustdb
   gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
   gpg: depth: 0  valid:   1  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 1u
   pub   4096R/2720E025 2018-03-20
         Key fingerprint = 200E F16E 95F6 82A7 3739  A257 2965 CA25 2720 E025
   uid                  CEVAP (Ionian Deterministic Building Process - Gitian Building) <[email protected]>
   sub   4096R/F0FEB8BB 2018-03-20
   

   List your keys and save the key ID of your key, in later step we will use this ID as SIGNER variable. We will use LONG ID and to get LONG id, please run:

   gpg --list-secret-keys --keyid-format LONG
   

   This is the output for the key which we created in/for this guide and it shows us now LONG key: 2965CA252720E025

   gitianuser@debian:~$ gpg --list-secret-keys --keyid-format LONG
   /home/gitianuser/.gnupg/secring.gpg
   -----------------------------------
   sec   4096R/2965CA252720E025 2018-03-20
   uid                          CEVAP (Ionian Deterministic Building Process - Gitian Building) <[email protected]>
   ssb   4096R/10E96958F0FEB8BB 2018-03-20
   

   If you want to show SHORT keys once again, just run

   gpg --list-secret-keys --keyid-format SHORT
   

   Output for this guide and our key is:

   gitianuser@debian:~$ gpg --list-secret-keys --keyid-format SHORT
   /home/gitianuser/.gnupg/secring.gpg
   -----------------------------------
   sec   4096R/2720E025 2018-03-20
   uid                  CEVAP (Ionian Deterministic Building Process - Gitian Building) <[email protected]>
   ssb   4096R/F0FEB8BB 2018-03-20
   

10. • Backup your GPG key

    Backup GPG key 2720E025.

    gpg --export --armor 2720E025 > public.asc                # backup public key
    gpg --export-secret-keys 2720E025 > secret.pgp            # backup private key encrypted
    gpg --export-secret-keys --armor 2720E025 > secret.asc    # backup private key as txt
    
    

    Transfer Keys. Open terminal on your local pc and run this to copy all 3 keys which we backed up.

    If you have setup VM to be able to connect over ssh to your local machine, you could copy all files directly from VM to your local pc wiht ~/.ssh/id_rsa public.asc secret.asc secret.pgp username@remote-server:~. To simplify it, we will use 3 commands and after each you will be asked for a password. Enter default password: cevap

    scp gitian:~/public.asc ~/public_2720E025.asc             # Copy Public GPG key to your home folder as "~/public_2720E025.asc"
    scp gitian:~/secret.asc ~/private_2720E025.asc            # Copy Private GPG key to your home folder as "~/private_2720E025.asc"
    scp gitian:~/secret.pgp ~/private_2720E025.pgp            # Copy Private GPG key to your home folder as "~/private_2720E025.pgp"
    

    As one line command: scp gitian:~/public.asc ~/public_2720E025.asc;scp gitian:~/secret.asc ~/private_2720E025.asc;scp gitian:~/secret.pgp ~/private_2720E025.pgp

11. • Send your public GPG key to keyservers

    Add new signatures to your key and push it to debian keyring server:

    gpg --keyserver keyring.debian.org --send-keys 0x2965CA252720E025
    gpg --keyserver keyserver.ubuntu.com --send-keys 0x2965CA252720E025
    gpg --keyserver pgp.mit.edu --send-keys 0x2965CA252720E025
    gpg --keyserver keyserver.ubuntu.com --send-keys 0x2965CA252720E025
    

    As output we see, example for keyring.debian.org: gpg: sending key 2720E025 to hkp server keyring.debian.org

    We will send this key as addition to all other key servers like MIT and ubuntu.

    New signatures will be included in our next keyring push (which happens approx. monthly)

12. • Add your public key to your Github account.

    Here is a guide how to import GPG key into your account.

Setup signer and version/branch

cd ./ion
export SIGNER="2965CA252720E025" # Replace with your Key
export VERSION=3.0.0 # github branch
git fetch
git checkout v${VERSION}
cd ..

# update signature
cd ./gitian.sigs
git pull
cd ..

# update gitian builder
cd ./gitian-builder
git pull
cd ..

# Fetch and create inputs: (first time, or when dependency versions change)
cd ./gitian-builder
mkdir -p inputs
wget -P inputs https://ioncore.xyz/cfields/osslsigncode-Backports-to-1.7.1.patch
wget -P inputs http://downloads.sourceforge.net/project/osslsigncode/osslsigncode/osslsigncode-1.7.1.tar.gz
wget -P inputs https://bitcoincore.org/depends-sources/sdks/MacOSX10.11.sdk.tar.gz
cd ..

# prebuild ion dependencies
cd ./gitian-builder
make -C ../ion/depends download SOURCES_PATH=`pwd`/cache/common
cd ..


# build offline
cd ./gitian-builder
./bin/gbuild --url ion=/home/gitianuser/vm-builder-0.12.4+bzr494/ion,signature=/home/gitianuser/vm-builder-0.12.4+bzr494/gitian.sigs --num-make 2 --memory 3000 --commit ion=v${VERSION} ../ion/contrib/gitian-descriptors/gitian-linux.yml
./bin/gsign --signer "$SIGNER" --release ${VERSION}-linux --destination ../gitian.sigs/ ../ion/contrib/gitian-descriptors/gitian-linux.yml
mv build/out/ion-*.tar.gz build/out/src/ion-*.tar.gz ../

./bin/gbuild --url ion=/home/gitianuser/vm-builder-0.12.4+bzr494/ion,signature=/home/gitianuser/vm-builder-0.12.4+bzr494/gitian.sigs --num-make 2 --memory 3000 --commit ion=v${VERSION} ../ion/contrib/gitian-descriptors/gitian-win.yml
./bin/gsign --signer "$SIGNER" --release ${VERSION}-win-unsigned --destination ../gitian.sigs/ ../ion/contrib/gitian-descriptors/gitian-win.yml
mv build/out/ion-*-win-unsigned.tar.gz inputs/ion-win-unsigned.tar.gz
mv build/out/ion-*.zip build/out/ion-*.exe ../

./bin/gbuild --url ion=/home/gitianuser/vm-builder-0.12.4+bzr494/ion,signature=/home/gitianuser/vm-builder-0.12.4+bzr494/gitian.sigs --num-make 2 --memory 3000 --commit ion=v${VERSION} ../ion/contrib/gitian-descriptors/gitian-osx.yml
./bin/gsign --signer "$SIGNER" --release ${VERSION}-osx-unsigned --destination ../gitian.sigs/ ../ion/contrib/gitian-descriptors/gitian-osx.yml
mv build/out/ion-*-osx-unsigned.tar.gz inputs/ion-osx-unsigned.tar.gz
mv build/out/ion-*.tar.gz build/out/ion-*.dmg ../
cd ..


# build online
cd ./gitian-builder
./bin/gbuild --num-make 2 --memory 3000 --commit ion=v${VERSION} ../ion/contrib/gitian-descriptors/gitian-linux.yml
./bin/gsign --signer "$SIGNER" --release ${VERSION}-linux --destination ../gitian.sigs/ ../ion/contrib/gitian-descriptors/gitian-linux.yml
mv build/out/ion-*.tar.gz build/out/src/ion-*.tar.gz ../

./bin/gbuild --num-make 2 --memory 3000 --commit ion=v${VERSION} ../ion/contrib/gitian-descriptors/gitian-win.yml
./bin/gsign --signer "$SIGNER" --release ${VERSION}-win-unsigned --destination ../gitian.sigs/ ../ion/contrib/gitian-descriptors/gitian-win.yml
mv build/out/ion-*-win-unsigned.tar.gz inputs/ion-win-unsigned.tar.gz
mv build/out/ion-*.zip build/out/ion-*.exe ../

./bin/gbuild --num-make 2 --memory 3000 --commit ion=v${VERSION} ../ion/contrib/gitian-descriptors/gitian-osx.yml
./bin/gsign --signer "$SIGNER" --release ${VERSION}-osx-unsigned --destination ../gitian.sigs/ ../ion/contrib/gitian-descriptors/gitian-osx.yml
mv build/out/ion-*-osx-unsigned.tar.gz inputs/ion-osx-unsigned.tar.gz
mv build/out/ion-*.tar.gz build/out/ion-*.dmg ../
cd ..