Talos VM on TrueNAS SCALE

Below are the instructions for creating a Talos VM atop a TrueNAS SCALE host for use with TrueCharts.

Creating a Network Bridge

A network bridge will be needed for running a VM inside of SCALE and to be able to reach the SCALE host machine. To create a Bridge network, open your TrueNAS Scale Web UI and go to the Network tab.

Network

Do not click Test Changes before you finish the whole guide

Acquiring Interface Settings

1. Note down the interface IP Address and the Subnet mask of your currently used interface

2. Click on Edit on the currently used interface

3. Disable “DHCP” and remove all “Aliases” by clicking on the x next to them

4. Create the Network Bridge as described in the following section

The Network Interface should then look like this:

Creating the Network Bridge

1. Click on Add in the top-right corner of the interfaces section

2. Select the type Bridge and give it a Name. Note that this name cannot be changed afterwards, so you might want to add a meaningful description to it

3. Add your network interface as a Bridge Member. (In this example: enp6so)

4. Set the MTU to “1500”

5. In the Alias section, click on Add to create a new IP address field

6. Enter the IP address and Subnet mask you previously took note of

7. Click Test Changes and confirm the prompt. Wait a bit for the UI to become responsive again

8. Click on Save Changes to save the changes you have made

9. If something goes wrong and the UI stays unresponsive, the changes will be reverted after 60 seconds.

The bridge should now look similar to this one when done:

Creating ZVOL storage for the VM

1. Start by going to the Datasets section of your TrueNAS Scale WebUI

2. Select the dataset you want the files for the VM to be stored on, as below

3. Click the Add Zvol button on the top-right

4. You will be presented with a pane of potential options. Leave all of these as default except for Zvol name (call it something recognizable like Talos Zvol 1), Size for this zvol (set to between 256GiB and 512GiB), Force size (check this option), Sparse (check this option) and Block size (change to 128 KiB)

5. We do not recommend to encrypt the dataset

6. After you are done it should look similar to this:

7. Hit save and wait for the system to create the Zvol. The GUI should refresh and then show your Zvol in the list of datasets like so

8. As you can see, I have 2 Zvols in my dataset. This is because we require the presence of multiple disks for the Talos VM. So, repeat the above steps, however this time set Size for this zvol to between 768GiB and 2TiB or however large you desire it to be. The first Zvol you created will be the “system” disk for Talos, and the second Zvol will be the “data” disk for Talos. Instructions on how to attach the second Zvol to the Talos VM will follow below.

GPU Isolation

Skip this step if you do not intend to pass a GPU through to the Talos VM.

1. Go to System Settings > Advanced in your SCALE UI and scroll down to the section Isolated GPU Device(s)

2. Click on Configure on the top right of the section and close the warning

3. Select the GPU you want to Isolate and use for your VM in the dropdown. Like shown here

4. Click Save. This might take a bit

5. It should now show your isolated GPU in the section. You may need to hard-refresh your browser’s page with CTRL + F5 for it to update due to SCALE GUI issues.

Creating the VM

1. Start by going to the Virtualization section of your TrueNAS Scale WebUI

2. If you want to assign a GPU to the VM, make sure to isolate the GPU first

3. Click on Add Virtual Machines

4. We also highly suggest users to set a fixed DHCP-Lease (aka static IP) for your VM inside your modem/router.

Operating System

Select the options like shown below. Make sure to remember the password you have set!

CPU and Memory

1. Select the amount of Cores and set the Threads to 2 if your CPU supports HyperThreading like in the example below. We suggest you to add available host system cores minus 1 for your host system

2. Make sure to allocate enough RAM towards your VM with the Memory Size setting.

Minimum amount of RAM: 16GB

Minimum recommended amount of RAM: 32GB

Disks

Select the first previously created Zvol for your VM as shown below:

Network Interfaces

Change the Attach NIC to the Bridge you created in the beginning of the guide

Installation Media

1. Download the Talos Linux image here and save it to your PC

2. Click on Upload New Image File in the VM Creation Screen and choose Choose File and select the downloaded ISO. Make sure to select a location on your SCALE host where you want to save your ISO to. An example can be seen below:

GPU

Make sure your GPU is isolated before you try to assign it to your VM. If you don’t want to assign a GPU to your VM, just click Next and skip this part of the guide.

Confirm Options

If you followed this guide correctly, the options shown should look similar to the screenshot below. Make sure everything is correct and then click on Save to finish the VM setup.

Attaching Extra Disk

Now that we have created the Talos VM, we need to attach the second Zvol we created earlier to it.

1. Select/Expand the VM

2. Select the Devices button

3. Select the Add button in the top-right

4. Change Type to Disk and click into the Search box under Zvol. It should show the two Zvols you created earlier, select the second. In the below example, this would be “Talos zvol 2”

5. Change Mode to VirtIO and Device Order to 1002. Leave all other options as default.

If you followed this correctly, the options shown should look similar to the screenshot below. Make sure everything is correct and then click on Save to finish the VM setup. Alternatively, follow the instructions here.

Note

Under the Order section of the table, there should be two disks. You want Zvol1 from earlier to be a lower order number than Zvol2. If this is not done by default, change the Device Order numbering so that your first Zvol comes before the second

Fixed DHCP Lease

Below you can find 2 examples on how to set a fixed DHCP Lease for UniFi and Fritzbox hardware.

UniFi

1. Open your UniFi UI and go to the Client View (highlighted in blue on the screenshot)

2. Click on the Client that represents your VM (not your SCALE Host System)

3. Select Fixed IP-Address in the pop-up menu on the right, enter the IP address you’d like to reserve for the VM, and hit Apply Changes. Note that you will need to restart the VM from within SCALE to get it to update its IP address if you changed it.

FritzBox

1. Go to your FritzBox web-interface. Navigate to Home Network> Network and select the running VM (not your SCALE Host System)

2. Click on the Edit button and scroll down to the IP-Address in Home Network section

3. Check Always give this device the same IP.

(screenshot in german)

TalConfig Examples

Here are a few examples you can copy-paste into talconfig to setup the above VMs to run Talos.

Single-Node Cluster or Master-Node

By default we ensure qemu guest additions are always loaded for all master-nodes. It’s important to note that for master nodes, we use static IPs for referencing them after they are initially initialised using a Fixed DHCP lease.

    - hostname: k8s-control-1
      ipAddress: ${MASTER1IP}
      controlPlane: true
      nameservers:
        - 8.8.8.8
        - 1.1.1.1
      installDiskSelector:
        size: <= 100TB
      networkInterfaces:
        - interface: eth0
          addresses:
            - ${MASTER1IP}/24
          routes:
            - network: 0.0.0.0/0
              gateway: ${GATEWAY}
          vip:
            ip: ${VIP}

Worker Node

By default we ensure qemu guest additions are always loaded for all worker-nodes. Workernodes can be pretty basic and should “just work”.

    - hostname: k8s-worker-1
      ipAddress: 192.168.10.111
      nameservers:
        - 8.8.8.8
        - 1.1.1.1
      installDiskSelector:
        size: <= 600GB