Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents
Goals

In ICN's SDWAN usages, SFC (Service Function Chain) is designed to support Corp networks to connect to external internet with security connection. The SFC includes Security VNF (e.g. firewall etc.), WAN Opt CNF and SDWAN VNF/CNF, and SDWAN module is worked as software defined router which can be used to defined the rules when connect to external internet. Below diagram shows where SDWAN module located in the whole system.



Basic Technology

OpenWRT

The OpenWRT Project (https://openwrt.org/) is an open source project based on Linux, and it is primarily used on embedded devices to route network traffic. There are more than 3500 software packages which can be installed on OpenWRT via opkg package management system. OpenWRT provides both docker image and VM image to support virtualization solution (https://openwrt.org/docs/guide-user/virtualization/start). In ICN, we run OpenWRT in container.

OpenWRT Mwan3 package (a replacement for multiwan package) provides the capabilities for multiple WAN management: WAN interfaces management, outbound traffic rules, traffic load balancing etc.

ovn4nfv-k8s-plugin

ovn4nfv-k8s-plugin is a CNI plugin based on ovn. It can work together with Multus CNI to add multiple interfaces for the pod. One of the interfaces is the Multus default interface, it could be flannel, calico, etc. The other interfaces are added by ovn4nfv-k8s-plugin according the the pod annotation. With ovn4nfv-k8s-plugin, we can create virtual network in run-time. Also we can connect the pod with the provider network, this is important for CNF.

Design Proposals

Implementation

SDWAN module is implemented as CNF instead of VNF for better performance and proving of CNF can also be effect way to deploy SDWAN

...

Deployment

Helm will be used to deploy SDWAN CNF as pod on nodes with external network connection with below parameters:

  • Network interfaces: multiple interfaces from difference CNI plugins, e.g. one is Multus default CNI (e.g. flannel or Calico), one is OVN CNI to connect to virtual network1 then connect to corp network, and the last one is OVN CNI which connect to external router
  • WAN configuration: an initial script running in the container will use uci command to do initial WAN configuration based on the input
    • WAN interface definition (e.g. interface, member etc.)

      config interface 'wan'
      	option enabled '1'
      	list track_ip '8.8.4.4'
      	list track_ip '8.8.8.8'
      	list track_ip '208.67.222.222'
      	list track_ip '208.67.220.220'
      	option reliability '2'
      	option count '1'
      	option timeout '2'
      	option interval '5'
      	option down '3'
      	option up '8'
      config member 'wan_m1_w3'
      	option interface 'wan'
      	option metric '1'
      	option weight '3'


    • Initial Traffic rules (e.g. policy, rule etc.)

      config policy 'wan_only'
      	list use_member 'wan_m1_w3'
      config rule 'sticky_even'
      	option src_ip '0.0.0.0/0.0.0.1'
      	option dest_port '443'
      	option proto 'tcp'
      	option use_policy 'wan_only'


Runtime Configuration

SDWAN traffic rules and WAN interfaces are required to be configured at runtime through Restful API interface.

...

  • command: 

    POST /cgi-bin/luci/admin/config/command

    Execute commands in OpenWRT CNF

    Normal Response Code: 200

    Error Response Code: 400

    Request:

    NameInTypeDescription
    commandBodyStringcommands to be executed in CNF, each command is separated by semicolon


SDWAN CNF Controller Implementations

The Akraino-SDEWAN-OutOfBoxNetworking.pptx describes the whole SDEWAN solution design.

...

  • SDEWAN conf mgr deletes Sdewan CR
  • K8s notifies Sdewan controller (run inside Sdewan Conf Agent) the CR delete event
  • Sdewan controller (run inside Sdewan Conf Agent) finds the Sdewan CNF owned by this deleted CR, call k8s API to delete the CNF

Timeline

ModuleTasksOwnerDue Current Status Description
PORs




POCSetup IPSec tunnelRuoyuFeb.26WW09: setup POC environment by manual configuration (Site-2-Site, Initiator-responder, Initiator-responder with vip) - Done
SDEWAN CNF





Service APIHuifeng
Done 

Start/stop/restart/reload SDWAN service, includes: mwan3, firewall/NAT, IpSec.

Reference: SDEWAN CNF#SDEWANService


MWAN3 API Huifeng 
Done 

Support MWAN3 rule/policy configuration.

Reference: SDEWAN CNF#MWAN3 

OpenWRT Reference: https://openwrt.org/docs/guide-user/network/wan/multiwan/mwan3


Firewall API Huifeng 

Design: Feb.26

Implementation: Mar.12 

WW08: Initial design Done

WW09: Implementation - 50%

WW10: 80%

WW11: done

Support firewall configuration for zone (general rule for a group of interfaces), forwarding (iptables forward), rule, redirect (DNAT/SNAT).

Reference: SDEWAN CNF#Firewall  

OpenWRT Reference: https://openwrt.org/docs/guide-user/firewall/firewall_configuration


IPSec API Ruoyu 

Design: Feb.26

Implementation: Mar.18 

WW08: Initial design Done

WW09: design done (to be reviewed)

WW10/11/12: 90%


 

Support IPSec configuration for remote site, proposal.

Reference: https://wiki.akraino.org/display/AK/IPSec+Design#IPSecDesign-IPSecRestAPI

OpenWRT Reference: https://openwrt.org/docs/guide-user/services/vpn/ipsec/strongswan/start

(Note: OpenWRT Wiki page is out-of-date compare to 18.06 implementation which we used and the current design is based on openwrt ipsec code directly)

SDEWAN E2E scenario



E2E demo for SDEWAN solution 

manual stepsAll WW13 -14
manual steps (create CNF, openwrt configuration for Ipsec/NAT rule, manual connectivity test for ms) to verify E2E test scenarios

auto test scripts to enable demo in ICNAll WW15-16

• leverage kud to setup 3 clusters (Hub, edge1, edge2)

• use pre-defined yaml file (with network interface information and rules definition) to create Sdewan CNF

• use linux shell script to call CNF Rest API (e.g. update rule, restart service etc.)

• shell script to verify ms connectivity in different edge cluster

SDEWAN CNF Controller 





POC to verify CR & CNF matching by label
WW17

R3.1

POC to verify the flow for n:m label matching between CR instances and CNF instances (e.g. a CR can apply to multiple CNF and a CNF can have multiple CR)


CRD re-design
WW18-19
R3.1

SDEWAN CRD Cheng WW20

Redesigned in R3.1

Define a SDWAN CNF with mwan3, firewall and IPSec configuration

Reference: Sdewan config AgentCRD Controller 


MWAN3 CRDCheng 

WW21


Redesigned in R3.1

Define MWAN3 configuration (policy, rule)

Reference:  Sdewan config AgentCRD Controller


Firewall CRD Cheng WW22-WW23



Redesigned in R3.1

Define Firewall CRD (zone, forwarding, rule, redirect (NAT)) 


IPSec CRD  Ruoyu WW24-WW25
 

Redesigned in R3.1

Define IPSec CRD (remote site, proposal)

Reference: https://wiki.akraino.org/display/AK/IPSec+Design#IPSecDesign-IPSecCRD 

Scenario design: SD-EWAN Scenarios

Integration



CNF controller and CNF Rest API integration 

MWAN3 Cheng/Huifeng WW26

Redesigned in R3.1

MWAN3 CRD/Restful API integration 


Firewall Cheng/Huifeng WW27

Redesigned in R3.1

Firewall CRD/Restful API integration 


IPSec Ruoyu/Huifeng WW28

Redesigned in R3.1

IPSec CRD/Restful API integration 

Stretch Goals 




SDWAN Hub Controller EWAN Config Manager: call EWAN Conf Agent to configure EWAN CNF Rama 


Key Store key in TPM Cheng 


QAT Support Investigate how to enable QAT support for IPSec (Client library such as OpenSSL configuration, kernel module is not need in CNF) Ruoyu 


SDEWAN CNF

Sdewan

...

CRD Controller

Presentation

ICN Weekly meeting video recordings- Weekly Akraino ICN Engineering Meeting