Introduction
Physical Networking Architecture Considerations
<INSERT DETAILS OF TARGET GEOGRAPHICAL CONSIDERATIONS OF WAN and EDGE SITE LANS TO SUPPORT CENTRALIZED RC AND GEOGRAPHICALLY DISTRIBUTED EDGE PODS>
LAG Configuration
The RC (when installed on bare metal, Rover and Unicycle genesis nodes boot via their VLAN tagged 'host' interfaces which are pre-provisioned on the serving TOR switches with LAG bonding. Since booting occurs before the linux kernel can bring up its LAC-P signaling the TOR switches must be configured to pass traffic on their primary (first) link before the LAG bundle is up.
The switch configuration for the network bond for the 'host' interfaces must be set to bring up the bond on the first interface prior to lacp completing negotiation.
- For Junos OS this option is typically called force-up and should be set on the first interface in the bond.
- For Arista, this option is typically called lacp fallback.
- Please ensure any TOR switch that is to be used supports this functionality and then refer to your network switch documentation to determine the correct configuration.
The 'host' network interfaces on all DHCP clients and servers must be located on the same L2 network so that the DHCP Request messages broadcast from the client reach the server and vice versa. Alternatively DHCP helper/relay functionality may be implemented on the TOR to which the DHCP clients are attached to allow inter subnet DHCP operation between DHCP client and server.
<Update to show RC, Rover, Genesis as Clients and Build server and RC as servers>
eyJleHRTcnZJbnRlZ1R5cGUiOiIiLCJnQ2xpZW50SWQiOiIiLCJjcmVhdG9yTmFtZSI6IkFuZHJldyBXaWxraW5zb24iLCJvdXRwdXRUeXBlIjoiYmxvY2siLCJsYXN0TW9kaWZpZXJOYW1lIjoiQW5kcmV3IFdpbGtpbnNvbiIsImxhbmd1YWdlIjoiZW4iLCJkaWFncmFtRGlzcGxheU5hbWUiOiIiLCJzRmlsZUlkIjoiIiwiYXR0SWQiOiIxMTk5NDgwOCIsImRpYWdyYW1OYW1lIjoiTEFHIDEiLCJhc3BlY3QiOiIiLCJsaW5rcyI6ImF1dG8iLCJjZW9OYW1lIjoiTmV0d29yayBBcmNoaXRlY3R1cmUiLCJ0YnN0eWxlIjoidG9wIiwiY2FuQ29tbWVudCI6ZmFsc2UsImRpYWdyYW1VcmwiOiIiLCJjc3ZGaWxlVXJsIjoiIiwiYm9yZGVyIjp0cnVlLCJtYXhTY2FsZSI6IjEiLCJvd25pbmdQYWdlSWQiOjExOTkzOTA0LCJlZGl0YWJsZSI6ZmFsc2UsImNlb0lkIjoxMTk5NDgxNSwicGFnZUlkIjoiIiwibGJveCI6dHJ1ZSwic2VydmVyQ29uZmlnIjp7ImVtYWlscHJldmlldyI6IjEifSwib2RyaXZlSWQiOiIiLCJyZXZpc2lvbiI6MiwibWFjcm9JZCI6ImZhN2M4ZTJhLWUxZmItNGFlZS1iM2MxLTI3NzBlY2JmY2E1NyIsInByZXZpZXdOYW1lIjoiTEFHIDEucG5nIiwibGljZW5zZVN0YXR1cyI6Ik9LIiwic2VydmljZSI6IiIsImlzVGVtcGxhdGUiOiIiLCJ3aWR0aCI6IjI0NDEiLCJzaW1wbGVWaWV3ZXIiOmZhbHNlLCJsYXN0TW9kaWZpZWQiOjE1NTkwNjEyNjUwMDAsImV4Y2VlZFBhZ2VXaWR0aCI6ZmFsc2UsIm9DbGllbnRJZCI6IiJ9
eyJleHRTcnZJbnRlZ1R5cGUiOiIiLCJnQ2xpZW50SWQiOiIiLCJjcmVhdG9yTmFtZSI6IkFuZHJldyBXaWxraW5zb24iLCJvdXRwdXRUeXBlIjoiYmxvY2siLCJsYXN0TW9kaWZpZXJOYW1lIjoiQW5kcmV3IFdpbGtpbnNvbiIsImxhbmd1YWdlIjoiZW4iLCJkaWFncmFtRGlzcGxheU5hbWUiOiIiLCJzRmlsZUlkIjoiIiwiYXR0SWQiOiIxMTk5NDgwNSIsImRpYWdyYW1OYW1lIjoiTEFHIDIiLCJhc3BlY3QiOiIiLCJsaW5rcyI6ImF1dG8iLCJjZW9OYW1lIjoiTmV0d29yayBBcmNoaXRlY3R1cmUiLCJ0YnN0eWxlIjoidG9wIiwiY2FuQ29tbWVudCI6ZmFsc2UsImRpYWdyYW1VcmwiOiIiLCJjc3ZGaWxlVXJsIjoiIiwiYm9yZGVyIjp0cnVlLCJtYXhTY2FsZSI6IjEiLCJvd25pbmdQYWdlSWQiOjExOTkzOTA0LCJlZGl0YWJsZSI6ZmFsc2UsImNlb0lkIjoxMTk5NDgxNSwicGFnZUlkIjoiIiwibGJveCI6dHJ1ZSwic2VydmVyQ29uZmlnIjp7ImVtYWlscHJldmlldyI6IjEifSwib2RyaXZlSWQiOiIiLCJyZXZpc2lvbiI6MiwibWFjcm9JZCI6ImQ1YTYxMGQ4LWY4OTUtNGYxZS05YTFjLTc0Y2U4MDg1ZDQxNyIsInByZXZpZXdOYW1lIjoiTEFHIDIucG5nIiwibGljZW5zZVN0YXR1cyI6Ik9LIiwic2VydmljZSI6IiIsImlzVGVtcGxhdGUiOiIiLCJ3aWR0aCI6IjI0NDEiLCJzaW1wbGVWaWV3ZXIiOmZhbHNlLCJsYXN0TW9kaWZpZWQiOjE1NTkwNjEzNDMwMDAsImV4Y2VlZFBhZ2VXaWR0aCI6ZmFsc2UsIm9DbGllbnRJZCI6IiJ9
DHCP, HTTP PXE and PXE Booting
Network Cloud family blueprint deployments involve a number of different DHCP servers and HTTP PXE and non HTTP PXE booting procedures during the automatic deployment process.
Regional Controller Deployment
When installed on a bare metal server, the RC is configured by the Redfish API calls issued by the Build Server to make DHCP Requests and then HTTP PXE boot over its vlan tagged 'host' network interface. (When installed in a VM the BIOS setting and operating system install and thus these steps are skipped).
The Build Server acts as the DHCP and HTTP PXE boot server for the RC via its own 'host' network interface.
The 'host' network must provide connectivity for these processes between the RC and the Build Server.
In R1 the 'host' network has been verified as a single L2 broadcast domain.
The RC DHCP/HTTP PXE process flow is summarized below.
eyJleHRTcnZJbnRlZ1R5cGUiOiIiLCJnQ2xpZW50SWQiOiIiLCJjcmVhdG9yTmFtZSI6IkFuZHJldyBXaWxraW5zb24iLCJvdXRwdXRUeXBlIjoiYmxvY2siLCJsYXN0TW9kaWZpZXJOYW1lIjoiQW5kcmV3IFdpbGtpbnNvbiIsImxhbmd1YWdlIjoiZW4iLCJkaWFncmFtRGlzcGxheU5hbWUiOiIiLCJzRmlsZUlkIjoiIiwiYXR0SWQiOiIxMTk5NDQxOCIsImRpYWdyYW1OYW1lIjoiREhDUCAtIFBYRSBSQyBhbmQgUm92ZXIiLCJhc3BlY3QiOiIiLCJsaW5rcyI6ImF1dG8iLCJjZW9OYW1lIjoiTmV0d29yayBBcmNoaXRlY3R1cmUiLCJ0YnN0eWxlIjoidG9wIiwiY2FuQ29tbWVudCI6ZmFsc2UsImRpYWdyYW1VcmwiOiIiLCJjc3ZGaWxlVXJsIjoiIiwiYm9yZGVyIjp0cnVlLCJtYXhTY2FsZSI6IjEiLCJvd25pbmdQYWdlSWQiOjExOTkzOTA0LCJlZGl0YWJsZSI6ZmFsc2UsImNlb0lkIjoxMTk5NDgxNSwicGFnZUlkIjoiIiwibGJveCI6dHJ1ZSwic2VydmVyQ29uZmlnIjp7ImVtYWlscHJldmlldyI6IjEifSwib2RyaXZlSWQiOiIiLCJyZXZpc2lvbiI6MywibWFjcm9JZCI6ImZmMDk0Y2MxLTJmNTktNDAwOC1iYjFkLWNmY2ZhN2NmMTM4ZiIsInByZXZpZXdOYW1lIjoiREhDUCAtIFBYRSBSQyBhbmQgUm92ZXIucG5nIiwibGljZW5zZVN0YXR1cyI6Ik9LIiwic2VydmljZSI6IiIsImlzVGVtcGxhdGUiOiIiLCJ3aWR0aCI6IjQxMSIsInNpbXBsZVZpZXdlciI6ZmFsc2UsImxhc3RNb2RpZmllZCI6MTU1ODgyNTAzMzAwMCwiZXhjZWVkUGFnZVdpZHRoIjpmYWxzZSwib0NsaWVudElkIjoiIn0=
Rover Deployment
The Rover server is configured by the Redfish API calls issued by the RC to make DHCP Requests and then HTTP PXE boot over its vlan tagged 'host' network interface.
The RC acts as the DHCP and HTTP PXE boot server for the Rover server via its own tagged 'host' network interface.
The 'host' network must provide connectivity across the WAN between the Rover edge site and the RC in a more centralized location.
In R1 the 'host' network has been verified as a single L2 broadcast domain.
The Rover DHCP/HTTP PXE process flow is summarized below.
eyJleHRTcnZJbnRlZ1R5cGUiOiIiLCJnQ2xpZW50SWQiOiIiLCJjcmVhdG9yTmFtZSI6IkFuZHJldyBXaWxraW5zb24iLCJvdXRwdXRUeXBlIjoiYmxvY2siLCJsYXN0TW9kaWZpZXJOYW1lIjoiQW5kcmV3IFdpbGtpbnNvbiIsImxhbmd1YWdlIjoiZW4iLCJkaWFncmFtRGlzcGxheU5hbWUiOiIiLCJzRmlsZUlkIjoiIiwiYXR0SWQiOiIxMTk5NDQyMiIsImRpYWdyYW1OYW1lIjoiREhDUCAtIFBYRSBSQyIsImFzcGVjdCI6IiIsImxpbmtzIjoiYXV0byIsImNlb05hbWUiOiJOZXR3b3JrIEFyY2hpdGVjdHVyZSIsInRic3R5bGUiOiJ0b3AiLCJjYW5Db21tZW50IjpmYWxzZSwiZGlhZ3JhbVVybCI6IiIsImNzdkZpbGVVcmwiOiIiLCJib3JkZXIiOnRydWUsIm1heFNjYWxlIjoiMSIsIm93bmluZ1BhZ2VJZCI6MTE5OTM5MDQsImVkaXRhYmxlIjpmYWxzZSwiY2VvSWQiOjExOTk0ODE1LCJwYWdlSWQiOiIiLCJsYm94Ijp0cnVlLCJzZXJ2ZXJDb25maWciOnsiZW1haWxwcmV2aWV3IjoiMSJ9LCJvZHJpdmVJZCI6IiIsInJldmlzaW9uIjoyLCJtYWNyb0lkIjoiODVlNTQ4NGEtYzQxZC00NDE3LTgwNTQtZDVhN2U0YTAxNmVhIiwicHJldmlld05hbWUiOiJESENQIC0gUFhFIFJDLnBuZyIsImxpY2Vuc2VTdGF0dXMiOiJPSyIsInNlcnZpY2UiOiIiLCJpc1RlbXBsYXRlIjoiIiwid2lkdGgiOiI2OTUiLCJzaW1wbGVWaWV3ZXIiOmZhbHNlLCJsYXN0TW9kaWZpZWQiOjE1NTg4MjUwMDgwMDAsImV4Y2VlZFBhZ2VXaWR0aCI6ZmFsc2UsIm9DbGllbnRJZCI6IiJ9
It is possible to split the 'host' network spanning the WAN into multiple routed L2 domains using DHCP helper/relay functionality on the Rover's TOR but this is unverified in R1.
Unicycle Pod Deployment
A Unicycle pod's genesis node is configured by the Redfish API calls issued by the RC to make DHCP Requests and then HTTP PXE boot over its vlan tagged 'host' network interface.
The RC acts as the DHCP and HTTP PXE boot server for the Unicycle genesis node via its own tagged 'host' network interface.
The 'host' network must provide connectivity across the WAN between the Unicycle pods's edge site and the RC in a more centralized location.
In R1 the 'host' network has been verified as a single L2 broadcast domain.
Unicycle masters and workers (all unicycle nodes other than genesis), make DHCP Requests on the edge site's local 'pxe' network's L2 broadcast domain. The genesis node's MaaS acts as the DHCP server.
Unicycle masters and workers (all unicycle nodes other than genesis) are configured by the MaaS server running on the genesis node to PXE boot over the remote site's 'pxe' network.
The unicycle DHCP/HTTP PXE/PXE process flow is summarized below.
eyJleHRTcnZJbnRlZ1R5cGUiOiIiLCJnQ2xpZW50SWQiOiIiLCJjcmVhdG9yTmFtZSI6IkFuZHJldyBXaWxraW5zb24iLCJvdXRwdXRUeXBlIjoiYmxvY2siLCJsYXN0TW9kaWZpZXJOYW1lIjoiQW5kcmV3IFdpbGtpbnNvbiIsImxhbmd1YWdlIjoiZW4iLCJkaWFncmFtRGlzcGxheU5hbWUiOiIiLCJzRmlsZUlkIjoiIiwiYXR0SWQiOiIxMTk5NDQyOCIsImRpYWdyYW1OYW1lIjoiVW5pY3ljbGUgRGVwbG95bWVudCIsImFzcGVjdCI6IiIsImxpbmtzIjoiYXV0byIsImNlb05hbWUiOiJOZXR3b3JrIEFyY2hpdGVjdHVyZSIsInRic3R5bGUiOiJ0b3AiLCJjYW5Db21tZW50IjpmYWxzZSwiZGlhZ3JhbVVybCI6IiIsImNzdkZpbGVVcmwiOiIiLCJib3JkZXIiOnRydWUsIm1heFNjYWxlIjoiMSIsIm93bmluZ1BhZ2VJZCI6MTE5OTM5MDQsImVkaXRhYmxlIjpmYWxzZSwiY2VvSWQiOjExOTk0ODE1LCJwYWdlSWQiOiIiLCJsYm94Ijp0cnVlLCJzZXJ2ZXJDb25maWciOnsiZW1haWxwcmV2aWV3IjoiMSJ9LCJvZHJpdmVJZCI6IiIsInJldmlzaW9uIjoyLCJtYWNyb0lkIjoiYzEzNTMwNTQtZDdjZi00NjBhLTg4M2EtMzgwZTcyNmRlNWYwIiwicHJldmlld05hbWUiOiJVbmljeWNsZSBEZXBsb3ltZW50LnBuZyIsImxpY2Vuc2VTdGF0dXMiOiJPSyIsInNlcnZpY2UiOiIiLCJpc1RlbXBsYXRlIjoiIiwid2lkdGgiOiI5NzQiLCJzaW1wbGVWaWV3ZXIiOmZhbHNlLCJsYXN0TW9kaWZpZWQiOjE1NTg4MjU1MjcwMDAsImV4Y2VlZFBhZ2VXaWR0aCI6ZmFsc2UsIm9DbGllbnRJZCI6IiJ9
It is possible to split the 'host' network spanning the WAN into multiple routed L2 domains using functionality such as DHCP helper/relay on the Unicycle genesis TOR but this is unverified in R1. Note: this is not applicable to the other nodes in the unicycle pod as the DHCP and PXE boot process is supported over the local 'pxe' network.
IP Networking
BGP Networking