ansible-edda/README.md

214 lines
7.5 KiB
Markdown
Raw Normal View History

2022-08-18 10:48:41 +02:00
# The Ansible Edda
2022-10-01 17:46:29 +02:00
Ansible playbooks for provisioning The Nine Worlds.
2022-08-18 10:48:41 +02:00
## Secrets vault
2022-12-18 21:14:04 +01:00
- Encrypt with: ```ansible-vault encrypt vault.yml```
2022-09-21 23:57:15 +02:00
- Decrypt with: ```ansible-vault decrypt secrets.yml```
2022-12-18 21:14:04 +01:00
- Encrypt all `vault.yml` in a directory with: ```ansible-vault encrypt directory/**/vault.yml```
- Decrypt all `vault.yml` in a directory with: ```ansible-vault decrypt directory/**/vault.yml```
2022-08-18 10:48:41 +02:00
- Run a playbook with ```ansible-playbook --vault-id @prompt playbook.yml```
2022-12-07 21:36:08 +01:00
## The Nine Worlds
The main entrypoint for The Nine Worlds is [`main.yml`](main.yml).
2022-12-28 14:21:33 +01:00
### Keyring integration
Keyring integration requires `python3-keyring` to be installed.
To set the keyring password run:
``` sh
./vault-keyring-client.py --set [--vault-id <vault-id>]
```
If `--vault-id` is not specified, the password will be stored under `ansible`.
To use the password from the keyring invoke playbooks with:
``` sh
ansible-playbook --vault-id @vault-keyring-client.py ...
```
### Production and testing
The inventory files are split into [`production`](production) and [`testing`](testing).
2022-12-07 21:36:08 +01:00
To run the `main.yml` playbook on production hosts:
``` sh
ansible-playbook main.yml -i production
```
2022-12-07 21:36:08 +01:00
To run the `main.yml` playbook on production hosts:
``` sh
ansible-playbook main.yml -i testing
```
### Testing virtual machines
2023-02-13 21:59:24 +01:00
The scripts for starting, stopping, and reverting the testing virtual machines is located in
`scripts/testing/vmgr.py`.
2022-12-07 21:36:08 +01:00
### Playbooks
2022-12-18 21:14:04 +01:00
The Ansible Edda playbook is composed of smaller [`playbooks`](playbooks). To run a single playbook,
invoke the relevant playbook directly from the playbook directory. For example, to run the
[`system`](system) playbook, run:
2022-12-07 21:36:08 +01:00
``` sh
2022-12-18 21:14:04 +01:00
ansible-playbook playbooks/system.yml
```
Alternatively you can use its tag as well:
``` sh
ansible-playbook main.yml --tags "system"
2022-12-07 21:36:08 +01:00
```
### Roles
2022-12-18 21:14:04 +01:00
Playbooks are composed of roles defined in the `roles` directory,
[`playbooks/roles`](playbooks/roles).
To play only a specific role, e.g. `system/base` in the playbook `system`, run:
``` sh
ansible-playbook playbooks/system.yml --tags "system:base"
```
2022-12-18 21:14:04 +01:00
Or from the main playbook:
``` sh
2022-12-18 21:14:04 +01:00
ansible-playbook main.yml --tags "system:base"
```
### Role sub-tasks
Some roles are split into smaller groups of tasks. This can be checked by looking at the
`tasks/main.yml` file of a role, e.g.
2022-12-18 21:14:04 +01:00
[`playbooks/roles/system/base/tasks/main.yml`](playbooks/roles/system/base/tasks/main.yml).
2022-12-07 21:36:08 +01:00
To play only a particular group within a role, e.g. `sshd` in `base` of `system`, run:
``` sh
2022-12-18 21:14:04 +01:00
ansible-playbook playbooks/system.yml --tags "system:base:sshd"
```
Or from the main playbook:
``` sh
ansible-playbook main.yml --tags "system:base:sshd"
```
2023-02-13 21:59:24 +01:00
## Testing backups
Before testing the backups, you may want to shut `yggdrasil` down for extra confidence that it is
not being accessed/modified during this process. It is easy to access `yggdrasil` by accident if
`/etc/hosts` is not modified in the test VM, something that is easy to forget.
### Baldur on Scaleway
2023-02-13 21:59:24 +01:00
1. Create `baldur` by running:
```sh
python scripts/scaleway/baldur.py create --volume-size <size-in-GB>
```
Pick a volume size that's larger than what `yggdrasil` estimates for
`rpool/var/lib/yggdrasil/data`.
2. When done destroy `baldur` by running:
```sh
python scripts/scaleway/baldur.py delete
```
### Baldur on Yggdrasil
1. Create a VM on `yggdrasil` and install the same OS that is running on `yggdrasil`.
- Install the OS on a zvol on `rpool`.
- Prepare a zvol on `hpool` of size that's larger than what `yggdrasil` estimates for
`rpool/var/lib/yggdrasil/data` and mount at `/var/lib/baldur/data`.
- Create non-root user `wojtek` with `sudo` privileges.
2. Configure SSH to use `yggdrasil` as a jump server.
3. Service testing can then be done directly from the VM. To achieve that `/etc/hosts` needs to be
set to directly point at the right proxy server, e.g., `10.66.3.8`, not `localhost`.
### Test
1. Provision `baldur` by running
2023-02-13 21:59:24 +01:00
```sh
ansible-playbook --vault-id @vault-keyring-client.py -i inventory/baldur_production playbooks/baldur.yml
```
2. Restore all the backups by ssh'ing into `baldur` and running (as root):
2023-02-13 21:59:24 +01:00
```sh
/usr/local/sbin/restic-batch --config-dir /etc/restic-batch.d restore
```
2023-07-16 17:08:30 +02:00
3. Once restore has completed, `chown -R <user>:<user>` all the restored directories in
`/var/lib/<hostname>/data`. Restic restores the UID information of the host from which the backup
was performed which may not match that of the new target machine. Note that permissions and
ownership are restored as a second step once all the content is restored. Therefore, the files
will list `root` as owner during the restoration.
4. Start all the pod services with:
2023-02-13 21:59:24 +01:00
```sh
ansible-playbook --vault-id @vault-keyring-client.py -i inventory/baldur_production playbooks/services_start.yml
```
Give them some time to download all the images and start.
2023-07-16 17:08:30 +02:00
5. Once the CPU returns to idling check the state of all the pod services and their `veth`
2023-02-13 21:59:24 +01:00
interfaces. If necessary restart the affected pod. Sometimes they fail to start (presumably due
to issues related to limited CPU and RAM).
2023-07-16 17:08:30 +02:00
6. Boot into a test VM. Ideally, one installed onto a virtual disk since the live system might not
2023-02-13 21:59:24 +01:00
have enough space. A VM is used to make sure that none of the services on the host workstation
connect to `baldur` by accident.
2023-07-16 17:08:30 +02:00
7. Modify `/etc/hosts` in the VM to point at `baldur` for all relevant domains.
8. Test each service manually one by one. Use the Flagfox add-on to verify that you are indeed
2023-02-13 21:59:24 +01:00
connecting to `baldur`.
2023-07-16 17:08:30 +02:00
9. Stop all the pod services with:
2023-02-13 21:59:24 +01:00
```sh
ansible-playbook --vault-id @vault-keyring-client.py -i inventory/baldur_production playbooks/services_stop.yml
```
2023-02-19 23:46:17 +01:00
## Music organisation
The `playbooks/music.yml` playbook sets up tools and configuration for organising music. The process
is manual though. The steps for adding a new CD.
All steps below are to be executed as the `music` user.
2023-02-27 22:04:26 +01:00
### Note on tagging
* For live albums add "YYYY-MM-DD at Venue, City, Country" in the "Subtitle" tag.
* For remasters use original release tags and add "YYYY Remaster" in the "Subtitle" tag.
2023-02-19 23:46:17 +01:00
### Ripping a CD
1. Use a CD ripper and rip the CD to `/var/lib/yggdrasil/home/music/rip` using flac encoding.
2. Samba has been set up to give Windows access to the above directory. Therefore, CD rippers
available only for Windows can also be used, e.g. dBpoweramp.
### Import new music
1. Run `beet import /var/lib/yggdrasil/home/music/rip`. This will move the music files to
`/var/lib/yggdrasil/data/music/collection`.
2023-02-26 00:21:00 +01:00
2. Run `beet convert -a <match>`, where `<match>` is used to narrow down to new music only. This
will convert the flac files into mp3 files for sharing via Nextcloud.
2023-02-19 23:46:17 +01:00
3. Run `nextcloud-upload /var/tmp/music/mp3/<artist>` for every artist to upload to Nextcloud.
4. Remove the `/var/tmp/music/mp3/<artist>` directory.
2023-03-01 20:14:12 +01:00
2023-03-03 20:16:32 +01:00
#### Collections
Every track has a `compilation` tag at track-level as well as at album-level (at least in Beets). To
label the album as a compilation for sorting purposes, run `beet modify -a <album> comp=True`.
2023-03-01 20:14:12 +01:00
### Archive music
#### From rip
2023-03-03 20:16:32 +01:00
1. Run `beet --config .config/beets/archive.yaml import --move /var/lib/yggdrasil/home/music/rip`.
This will move the music files to `/var/lib/yggdrasil/data/music/archive`.
2023-03-01 20:14:12 +01:00
#### From collection
1. Run `beet --config .config/beets/archive.yaml import
/var/lib/yggdrasil/data/music/collection/<artist>/<album>`. This will copy the music files to
`/var/lib/yggdrasil/data/music/archive`.
2023-03-03 20:16:32 +01:00
2. Run `beet remove -d -a "album:<album>"`. This will remove the music files from the collection.