# The Ansible Edda Ansible playbooks for provisioning The Nine Worlds. ## Secrets vault - Encrypt with: ```ansible-vault encrypt vault.yml``` - Decrypt with: ```ansible-vault decrypt secrets.yml``` - Encrypt all `vault.yml` in a directory with: ```ansible-vault encrypt directory/**/vault.yml``` - Decrypt all `vault.yml` in a directory with: ```ansible-vault decrypt directory/**/vault.yml``` - Run a playbook with ```ansible-playbook --vault-id @prompt playbook.yml``` ## The Nine Worlds The main entrypoint for The Nine Worlds is [`main.yml`](main.yml). ### Keyring integration Keyring integration requires `python3-keyring` to be installed. To set the keyring password run: ``` sh ./vault-keyring-client.py --set [--vault-id ] ``` If `--vault-id` is not specified, the password will be stored under `ansible`. To use the password from the keyring invoke playbooks with: ``` sh ansible-playbook --vault-id @vault-keyring-client.py ... ``` ### Production and testing The inventory files are split into [`production`](production) and [`testing`](testing). To run the `main.yml` playbook on production hosts: ``` sh ansible-playbook main.yml -i production ``` To run the `main.yml` playbook on production hosts: ``` sh ansible-playbook main.yml -i testing ``` ### Testing virtual machines The scripts for starting, stopping, and reverting the testing virtual machines is located in `scripts/testing/vmgr.py`. ### Playbooks The Ansible Edda playbook is composed of smaller [`playbooks`](playbooks). To run a single playbook, invoke the relevant playbook directly from the playbook directory. For example, to run the [`system`](system) playbook, run: ``` sh ansible-playbook playbooks/system.yml ``` Alternatively you can use its tag as well: ``` sh ansible-playbook main.yml --tags "system" ``` ### Roles Playbooks are composed of roles defined in the `roles` directory, [`playbooks/roles`](playbooks/roles). To play only a specific role, e.g. `system/base` in the playbook `system`, run: ``` sh ansible-playbook playbooks/system.yml --tags "system:base" ``` Or from the main playbook: ``` sh ansible-playbook main.yml --tags "system:base" ``` ### Role sub-tasks Some roles are split into smaller groups of tasks. This can be checked by looking at the `tasks/main.yml` file of a role, e.g. [`playbooks/roles/system/base/tasks/main.yml`](playbooks/roles/system/base/tasks/main.yml). To play only a particular group within a role, e.g. `sshd` in `base` of `system`, run: ``` sh ansible-playbook playbooks/system.yml --tags "system:base:sshd" ``` Or from the main playbook: ``` sh ansible-playbook main.yml --tags "system:base:sshd" ``` ## Testing backups Before testing the backups, you may want to shut `yggdrasil` down for extra confidence that it is not being accessed/modified during this process. It is easy to access `yggdrasil` by accident if `/etc/hosts` is not modified in the test VM, something that is easy to forget. ### Baldur on Scaleway 1. Create `baldur` by running: ```sh python scripts/scaleway/baldur.py create --volume-size ``` Pick a volume size that's larger than what `yggdrasil` estimates for `rpool/var/lib/yggdrasil/data`. 2. When done destroy `baldur` by running: ```sh python scripts/scaleway/baldur.py delete ``` ### Baldur on Yggdrasil 1. Create a VM on `yggdrasil`. - Install the OS on a zvol on `rpool`. - Prepare a zvol on `hpool` of size that's larger than what `yggdrasil` estimates for `rpool/var/lib/yggdrasil/data` and mount at `/var/lib/baldur/data`. - Create non-root user `wojtek` with `sudo` privileges. 2. Configure SSH to use `yggdrasil` as a jump server. ### Test 1. Provision `baldur` by running ```sh ansible-playbook --vault-id @vault-keyring-client.py -i inventory/baldur_production playbooks/baldur.yml ``` 2. Restore all the backups by ssh'ing into `baldur` and running (as root): ```sh /usr/local/sbin/restic-batch --config-dir /etc/restic-batch.d restore ``` 3. Start all the pod services with: ```sh ansible-playbook --vault-id @vault-keyring-client.py -i inventory/baldur_production playbooks/services_start.yml ``` Give them some time to download all the images and start. 4. Once the CPU returns to idling check the state of all the pod services and their `veth` interfaces. If necessary restart the affected pod. Sometimes they fail to start (presumably due to issues related to limited CPU and RAM). 5. Boot into a test VM. Ideally, one installed onto a virtual disk since the live system might not have enough space. A VM is used to make sure that none of the services on the host workstation connect to `baldur` by accident. 6. Modify `/etc/hosts` in the VM to point at `baldur` for all relevant domains. 7. Test each service manually one by one. Use the Flagfox add-on to verify that you are indeed connecting to `baldur`. 8. Stop all the pod services with: ```sh ansible-playbook --vault-id @vault-keyring-client.py -i inventory/baldur_production playbooks/services_stop.yml ``` ## Music organisation The `playbooks/music.yml` playbook sets up tools and configuration for organising music. The process is manual though. The steps for adding a new CD. All steps below are to be executed as the `music` user. ### Note on tagging * For live albums add "YYYY-MM-DD at Venue, City, Country" in the "Subtitle" tag. * For remasters use original release tags and add "YYYY Remaster" in the "Subtitle" tag. ### Ripping a CD 1. Use a CD ripper and rip the CD to `/var/lib/yggdrasil/home/music/rip` using flac encoding. 2. Samba has been set up to give Windows access to the above directory. Therefore, CD rippers available only for Windows can also be used, e.g. dBpoweramp. ### Import new music 1. Run `beet import /var/lib/yggdrasil/home/music/rip`. This will move the music files to `/var/lib/yggdrasil/data/music/collection`. 2. Run `beet convert -a `, where `` is used to narrow down to new music only. This will convert the flac files into mp3 files for sharing via Nextcloud. 3. Run `nextcloud-upload /var/tmp/music/mp3/` for every artist to upload to Nextcloud. 4. Remove the `/var/tmp/music/mp3/` directory. #### Collections Every track has a `compilation` tag at track-level as well as at album-level (at least in Beets). To label the album as a compilation for sorting purposes, run `beet modify -a comp=True`. ### Archive music #### From rip 1. Run `beet --config .config/beets/archive.yaml import --move /var/lib/yggdrasil/home/music/rip`. This will move the music files to `/var/lib/yggdrasil/data/music/archive`. #### From collection 1. Run `beet --config .config/beets/archive.yaml import /var/lib/yggdrasil/data/music/collection//`. This will copy the music files to `/var/lib/yggdrasil/data/music/archive`. 2. Run `beet remove -d -a "album:"`. This will remove the music files from the collection.