...
This is minimal configuration example where only 3 servers are used. Servers and their role are given in below table.
Server# | Role | Purpose |
1 | Installer node | This host is used for remotely installing and configuring master and worker node. This server also hosts bootstrap node on KVM-QEMU using libvirt. Several components like- HAProxy, DNS server, DHCP server for provisioning and baremetal network, CoreDNS, Matchbox, Terraform, IPMItool, TFTPboot are configured on this server. Since cluster coreDNS is running from here, this node will be required later as well. |
2 | Master node | This is control plane or master node of K8s cluster that is based on openshift 4.x. |
3 | Worker node | This is worker node which hosts the application. |
4 | Bootstrap node | Bootstrap node runs as VM on installer node and it exists only during the installation and later automatically deleted by installer. |
High level connectivity
Each server should have 3 Ethernet ports configured, purpose of these is listed below. These three are in addition to IPMI port, which is required for PXE boot.
Interface | Purpose |
Management interface | Remote root login from this interface is used for entire setup. This interface needs to have internet connectivity to download various files. This can be shared with external interface. This only needs to be present on the Installer node |
| External interface | Interface on the installer node that has internet network connectivity. All external traffic from masters/workers is redirected to the external interface of the installer node. |
Baremetal interface | This interface is for baremetal network, also known as SDN network. This interface doesn’t need internet connectivity. |
Provisioning interface | This interface is for PXE boot. This interface doesn’t need internet connectivity. |
Pre-requisites
OS requirements
Node Role | OS requirement |
Installer | CentOS 7.6 and above |
Bootstrap | RHCOS (Redhat CoreOS) |
Master | RHCOS (Redhat CoreOS) |
Worker | RHCOS/RHEL/CentOS/CentOS-rt |
Network requirements
- Configure required network interfaces as explained earlier. Be sure that each server has the NIC for PXE configured properly, matching to the interface that you are setting for this deployment. You can set it by entering the BIOS setup, and entering into the NIC configuration of your BIOS setup menu.
- Collect IPs and MAC addresses of all the nodes, one sample is given below. This information will be required to populate config files:
Role | iDRAC IP/IPMI port IP | Provisioning network IP | Baremetal network IP | Management network IP | Provisioning network port & mac | Baremetal network port & mac | Management network port & mac |
Installer | xx.xx.xx.xx | xx.xx.xx.xx | xx.xx.xx.xx | xx.xx.xx.xx | em1 / 21:02:0E:DC:BC:27 | em2/ 21:02:0E:DC:BC:28 | em3/ 21:02:0E:DC:BC:29 |
master-0 | |||||||
worker-0 |
- Enable IPMI over LAN for all master and worker nodes. This is required for remote PXE boot from installer node. Different servers have different ways to enable it.
In absence of this setting, following kind of errors are thrown from installer.
Error: Error running command ' ipmitool -I lanplus -H x.x.x.x -U xxx -P xxxxx chassis bootdev pxe;
ipmitool -I lanplus -H x.x.x.x -U xxx -P xxxxx power cycle || ipmitool -I lanplus -H x.x.x.x -U xxx -P xxxxx power on;
': exit status 1. Output: Error: Unable to establish IPMI v2 / RMCP+ session
Error: Unable to establish IPMI v2 / RMCP+ session
Error: Unable to establish IPMI v2 / RMCP+ session
Depending on servers, RMCP session needs to be enabled on security settings of the management console.
After enabling this setting, you can run below command to verify that it is working as expected. Give IP address, username and password.
ipmitool -I lanplus -H x.x.x.x -U xxx -P xxxxx chassis status
(where x.x.x.x is IPMI port IP of your master/worker node, this is followed by root username and password for IPMI e.g. iDRAC)
High level steps
Setup installer node
Install CentOS operating system there. Once you have it, configure your NIC/VLANS properly (management/external, provisioning, baremetal, ipmi). Be sure that you collect the information of interfaces/vlans.
Configure the system properly to run knictl on it:
knictl offers two commands to automate the deployment of a baremetal UPI cluster (and only baremetal UPI, at this time). As prerequisites to using these commands, you must ensure the following are true:
...