Move FreeBSD CI to cloud (MCS)

[Totktonada]

Now FreeBSD testing looks so: we have a Linux machine (hardware, non-virtual) with GitHub Runner, which serves jobs with freebsd-sh label. Internally it starts a VirtualBox machine with FreeBSD and and run testing inside it bt ssh.

I propose the following scheme:

• Prepare FreeBSD image for MCS.
  • Consult with MCS guys and provide it as one of available images for everyone?
• Investigate how to setup GitHub Runner inside.
• Run several virtual machines with FreeBSD and GitHub Runner.
• Revisit .workflows / .gitlab.mk / .travis.mk part in the repository.
• Don't forget to update internal machines list.

After this we can at least scale those runners manually. In future, I hope, I'll implement some autoscaling and, what is even more important, running a job of always-clean virtual machine.

There is Terraform machinery for manage self-hosted GitHub Runners (this part of the infrastructure is private, keyword: mfactory). Sooner or later, I hope, we'll track and manage our runners this way.

[Totktonada]

What is bad with current approach:

• Hard to reproduce a problem manually (test: stable fail box-tap/curl-build.test.lua tarantool#6547 is not reproduce on another FreeBSD image).
• Hard to setup more runners.
• Complexity of CI scripts (VBoxManage, ssh) — so it is inevitable more fragile.

With FreeBSD image in MCS we can setup a new virtual machine for interactive work in several clicks. And it will be identical to machines in CI.

[0x501D]

Deploy FreeBSD-13.0 to MCS HOWTO:

1. Create virtual drive: (1GB minimum, can be expanded later on VM instance create)
   $ qemu-img create -f qcow2 freebsd.qcow2 1G

2. Download the FreeBSD installer:
   $ wget -c https://download.freebsd.org/ftp/releases/ISO-IMAGES/13.0/FreeBSD-13.0-RELEASE-amd64-disc1.iso

3. Run VM on host system. I recommend use QEMU on Linux host.
   $ qemu-system-x86_64 -boot d -cdrom FreeBSD-13.0-RELEASE-amd64-disc1.iso -drive node-name=hd0,media=disk,if=virtio,file=freebsd.qcow2 -m 1024 -enable-kvm -cpu host -device virtio-net-pci,mac=de:ad:be:ef:00:01,netdev=netdev0 -netdev user,id=netdev0

4. Install FreeBSD as usual except this steps:
   4.1. Use manual disk partitioning:
   
   4.2. Go to shell in "Final config" (Exit) and do the following steps:

$ echo '/root/bsd-cloudinit' >> /etc/rc.local
$ echo 'console="comconsole,vidconsole"' >> /boot/loader.conf
$ echo 'autoboot_delay="1"' >> /boot/loader.conf
$ cat > /root/bsd-cloudinit << EOF
#!/bin/csh
#bsd-cloudinit workaround

#usage
#echo /root/bsd-cloudinit >> /etc/rc.local
#chmod u+x /root/bsd-cloudinit

set PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin
export $PATH

#disk_resize
gpart recover vtbd0
gpart resize -i 2 vtbd0
growfs -y vtbd0p2

#create freebsd user
pw useradd -n freebsd || true

#create home directory and .ssh
mkdir -p /home/freebsd/.ssh/
chown -R freebsd:freebsd /home/freebsd

#set passwordless sudo
echo 'freebsd ALL=(ALL) NOPASSWD: ALL' > /usr/local/etc/sudoers.d/10-cloudinit

#set_key
fetch http://169.254.169.254/openstack/latest/meta_data.json -q -o - | jq -r '.keys[] |.data' > /home/freebsd/.ssh/authorized_keys

#set_hostname
set fqdn=`fetch http://169.254.169.254/openstack/latest/meta_data.json -q -o - | jq '.hostname' | sed 's/.novalocal//g'`
sed -i "" "s/hostname.*/hostname=$fqdn/g" /etc/rc.conf
hostname `echo $fqdn | tr -d '"'`

#set random root passwd
pw mod user root -w random > /dev/null

#run userdata
fetch http://169.254.169.254/latest/user-data -q -o - | sh

#cleanup
set history = 0
ssh-keygen -A
sed -i "" "s/.*bsd-cloudinit//g" /etc/rc.local
rm -f /root/bsd-cloudinit
EOF

$ chmod u+x /root/bsd-cloudinit
$ pkg install sudo jq
$ shutdown -h now

5. Convert VM image to raw format:
   $ qemu-img convert -f qcow2 -O raw freebsd.qcow2 freebsd.raw

6. Upload freebsd.raw image to MCS as usual.

Cloud-init script is taken from here

[0x501D]

Setup FreeBSD github runner HOWTO:

Create FreeBSD instance (MCS), login and run:

$ sudo freebsd-update fetch
$ sudo freebsd-update install
$ sudo pkg install go node bash git screen
$ screen -R ci
$ git clone https://github.com/ChristopherHX/github-act-runner.git --recursive
$ cd github-act-runner
$ go run . configure --url 'https://repo' --name 'Name' -l 'label' --token 'GITHUB-TOKEN'
$ go run . run

Exit from screen with CTRL+a d key.