Intro to OpenFlow Tutorial with Ryu Controller

Part 1: Design and Setup

In this tutorial we are going to use Open vSwitch (OVS) as an OpenFlow switch connected to three hosts. OVS is a software switch running on a compute resource. The other three hosts can only communicate through the OVS switch. The experiment will need (the rspecs for this exercise are provided later in this section):

• 1 Xen VM with a public IP to run an OpenFlow controller
• 1 Xen VM to be the OpenFlow switch
• 3 Xen VMs as hosts

 

Step 1: Reserve Resources

Go to add resources and add RSpec using the URL option

a)  Reserve a VM that runs your OpenFlow controller.

b) Reserve your network, that includes a VM with OVS installed.

Step 2: Configure and Initialize Services

Overview: Although OVS is installed and initialized on the host that is meant to act as a software switch, it has not been configured yet. There are two main things that need to be configured:

(1) configure your software switch with the interfaces as ports

(2) point the switch to an OpenFlow controller.

a) Configure the Software Switch (OVS Window)

1. Login to the OVS host
2. Create an Ethernet bridge that will act as our software switch
1. sudo ovs-vsctl add-br br0
3. Prepare the interfaces to be added as ports to the OVS switch
1. Run ifconfig and write down the interface names that correspond 
1. Interface with IP 10.10.1.11 to host1 - eth3
2. Interface with IP 10.10.1.12 to host2 - eth1
3. Interface with IP 10.10.1.13 to host3 - eth2
2. Remove the IP form your data interfaces
1. sudo ifconfig eth1 0
2. sudo ifconfig eth2 0
3. sudo ifconfig eth3 0
4. Add all the data interfaces to the switch
1. sudo ovs-vsctl add-port br0 eth1
2. sudo ovs-vsctl add-port br0 eth2
3. sudo ovs-vsctl add-port br0 eth3
5. Verify the three ports configured run [This should output eth1, eth2, eth3]
1. sudo ovs-vsctl list-ports br0
1. eth1
2. eth2
3. eth3
   

b) Point your switch to a controller 

An OpenFlow switch will not forward any packet unless instructed by a controller. Basically the forwarding table is empty, until an external controller inserts forwarding rules. The OpenFlow controller communicates with the switch over the control network and it can be anywhere in the Internet as long as it is reachable by the OVS host.

1. Login to the controller host
2. Find the control interface IP of your controller [the IP of eth0]
1. ifconfig
1. 192.86.139.65
3. Point our software OpenFlow switch to the controller in OVS window
1. sudo ovs-vsctl set-controller br0 tcp:192.86.139.65:6633
4. Set your switch to fail-safe-mode
1. sudo ovs-vsctl set-fail-mode br0 secure
5. Verify OVS settings  
1. sudo ovs-vsctl show
1. 89362c3a-1d64-451b-977d-7b2718a1ba20
      Bridge "br0"
        Controller "tcp:192.86.139.65:6633"
        fail_mode: secure
        Port “eth1”
   Interface “eth1”
        Port “eth2”
           Interface “eth2”
        Port “eth3”
           Interface “eth3”
        Port "br0"
          Interface "br0"
            type: internal
      ovs_version: "2.3.1"

c) Standalone vs Secure Mode

The OpenFlow controller is responsible for setting up all flows on the switch, which means that when the controller is not running there should be no packet switching at all. Depending on the setup of your network, such a behavior might not be desired. It might be best that when the controller is down, the switch should default back to being a learning layer 2 switch. In other circumstances however this might be undesirable. In OVS this is a tunable parameter, called fail-safe-mode which can be set to the following parameters:

• standalone [default]: in this case OVS will take responsibility for forwarding the packets if the controller fails

• secure: in this case only the controller is responsible for forwarding packets, and if the controller is down all packets are dropped.

In OVS when the parameter is not set it falls back to the standalone mode. For the purpose of this tutorial we will set the fail-safe-mode to secure, since we want to be the ones controlling the forwarding.

 

Part 2: Execute

Now that the switch is up and running we are ready to start working on the controller. For this tutorial we are going to use the Ryu Controller. The software is already installed in the controller host for running the Ryu controller.

Step 3: Execute Experiment

a) Login to your hosts

• one window with ssh into the controller
• four windows with ssh into OVS
• one window with ssh into host1
• two windows with ssh into host2
• one window with ssh into host3

b) Use a Learning Switch Controller

1. Ping host2 from host1
1. ping host2 -c 10
1. Unable to ping host2
2. Output using 5 pings
1. PING host2-link-0 (10.10.1.2) 56(84) bytes of data.
   From host1-link-3 (10.10.1.1) icmp_seq=1 Destination Host Unreachable
   From host1-link-3 (10.10.1.1) icmp_seq=2 Destination Host Unreachable
   From host1-link-3 (10.10.1.1) icmp_seq=3 Destination Host Unreachable
   From host1-link-3 (10.10.1.1) icmp_seq=4 Destination Host Unreachable
   From host1-link-3 (10.10.1.1) icmp_seq=5 Destination Host Unreachable 
   
   
   --- host2-link-0 ping statistics ---
   5 packets transmitted, 0 received, +5 errors, 100% packet loss, time 4023ms
   pipe 3
       
2. Ping host2 from host1 and let it continue
1. ping host2
3. Start the learning switch controller which is already available by running the following two commands: 
1.  cd /tmp/ryu
2.  ./bin/ryu-manager ryu/app/simple_switch.py
1. lzma module is not available
   Registered VCS backend: git
   Registered VCS backend: hg
   Registered VCS backend: svn
   Registered VCS backend: bzr
   loading app ryu/app/simple_switch.py
   loading app ryu.controller.ofp_handler
   instantiating app ryu.controller.ofp_handler of OFPHandler
   instantiating app ryu/app/simple_switch.py of SimpleSwitch
    
   
4.  Ping host2 from host1
1. ping host2 -c 10
1. Unsuccessful ping
2. Output using 5 pings
1. PING host2-link-0 (10.10.1.2) 56(84) bytes of data.
   From host1-link-3 (10.10.1.1) icmp_seq=1 Destination Host Unreachable
   From host1-link-3 (10.10.1.1) icmp_seq=2 Destination Host Unreachable
   From host1-link-3 (10.10.1.1) icmp_seq=3 Destination Host Unreachable
   From host1-link-3 (10.10.1.1) icmp_seq=4 Destination Host Unreachable
   From host1-link-3 (10.10.1.1) icmp_seq=5 Destination Host Unreachable 
   --- host2-link-0 ping statistics ---
   5 packets transmitted, 0 received, +5 errors, 100% packet loss, time 4023ms
   pipe 3
5. Go to your controller host and take a look at the print outs. You should see that your controller installed flows based on the mac addresses of your packets. 
1. Controller
1. lzma module is not available
   Registered VCS backend: git
   Registered VCS backend: hg
   Registered VCS backend: svn
   Registered VCS backend: bzr
   loading app ryu/app/simple_switch.py
   loading app ryu.controller.ofp_handler
   instantiating app ryu.controller.ofp_handler of OFPHandler
   instantiating app ryu/app/simple_switch.py of SimpleSwitch
   packet in 55847087117385 02:be:af:04:66:ea ff:ff:ff:ff:ff:ff 3
   packet in 55847087117385 02:71:cd:30:b0:68 02:be:af:04:66:ea 1
   packet in 55847087117385 02:be:af:04:66:ea 02:71:cd:30:b0:68 3
   packet in 55847087117385 02:be:af:04:66:ea 02:71:cd:30:b0:68 3
2. Host1
1. From host1-link-3 (10.10.1.1) icmp_seq=11 Destination Host Unreachable
   From host1-link-3 (10.10.1.1) icmp_seq=12 Destination Host Unreachable
   64 bytes from host2-link-0 (10.10.1.2): icmp_seq=13 ttl=64 time=1008 ms
   64 bytes from host2-link-0 (10.10.1.2): icmp_seq=14 ttl=64 time=9.15 ms
   64 bytes from host2-link-0 (10.10.1.2): icmp_seq=15 ttl=64 time=0.826 ms
   64 bytes from host2-link-0 (10.10.1.2): icmp_seq=16 ttl=64 time=0.677 ms
   64 bytes from host2-link-0 (10.10.1.2): icmp_seq=17 ttl=64 time=0.832 ms
   64 bytes from host2-link-0 (10.10.1.2): icmp_seq=18 ttl=64 time=0.872 ms
   64 bytes from host2-link-0 (10.10.1.2): icmp_seq=19 ttl=64 time=0.865 ms
   64 bytes from host2-link-0 (10.10.1.2): icmp_seq=20 ttl=64 time=0.860 ms
   64 bytes from host2-link-0 (10.10.1.2): icmp_seq=21 ttl=64 time=0.828 ms
   
   
   --- host2-link-0 ping statistics ---
   21 packets transmitted, 9 received, +12 errors, 57% packet loss, time 20006ms
   rtt min/avg/max/mdev = 0.677/113.713/1008.508/316.368 ms, pipe 3

c) Look around your OVS switch

1. To see the flow table entries on your OVS switch [using the OVS terminal]
1. sudo ovs-ofctl dump-flows br0
1. NXST_FLOW reply (xid=0x4):
       cookie=0x0, duration=309.110s, table=0, n_packets=8, n_bytes=728, idle_age=302, in_port=3,dl_dst=02:71:cd:30:b0:68 actions=output:1 
       cookie=0x0, duration=309.114s, table=0, n_packets=10, n_bytes=924, idle_age=302, in_port=1,dl_dst=02:be:af:04:66:ea actions=output:3
2. To see messages go between your switch and your controller 
1. sudo tcpdump -i eth0 tcp port 6633 
1. 06:02:38.568516 IP pcvm2-4.genirack.nyu.edu.42638 > pcvm1-3.genirack.nyu.edu.6633: Flags [P.], seq 48:56, ack 49, win 237, options [nop,nop,TS val 908250 ecr 914176], length 8: OpenFlow version 1.0, type ECHO_REQUEST, length 8, xid 0x00000000
   06:02:38.569860 IP pcvm1-3.genirack.nyu.edu.6633 > pcvm2-4.genirack.nyu.edu.42638: Flags [P.], seq 49:57, ack 56, win 235, options [nop,nop,TS val 915426 ecr 908250], length 8: OpenFlow version 1.0, type ECHO_REPLY, length 8, xid 0x00000000
   06:02:38.569888 IP pcvm2-4.genirack.nyu.edu.42638 > pcvm1-3.genirack.nyu.edu.6633: Flags [.], ack 57, win 237, options [nop,nop,TS val 908250 ecr 915426], length 0
   06:02:43.568434 IP pcvm2-4.genirack.nyu.edu.42638 > pcvm1-3.genirack.nyu.edu.6633: Flags [P.], seq 56:64, ack 57, win 237, options [nop,nop,TS val 909500 ecr 915426], length 8: OpenFlow version 1.0, type ECHO_REQUEST, length 8, xid 0x00000000
   06:02:43.569700 IP pcvm1-3.genirack.nyu.edu.6633 > pcvm2-4.genirack.nyu.edu.42638: Flags [P.], seq 57:65, ack 64, win 235, options [nop,nop,TS val 916676 ecr 909500], length 8: OpenFlow version 1.0, type ECHO_REPLY, length 8, xid 0x00000000 
   06:02:43.569728 IP pcvm2-4.genirack.nyu.edu.42638 > pcvm1-3.genirack.nyu.edu.6633: Flags [.], ack 65, win 237, options [nop,nop,TS val 909500 ecr 916676], length 0 
   06:02:48.568688 IP pcvm2-4.genirack.nyu.edu.42638 > pcvm1-3.genirack.nyu.edu.6633: Flags [P.], seq 64:72, ack 65, win 237, options [nop,nop,TS val 910750 ecr 916676], length 8: OpenFlow version 1.0, type ECHO_REQUEST, length 8, xid 0x00000000
   06:02:48.570146 IP pcvm1-3.genirack.nyu.edu.6633 > pcvm2-4.genirack.nyu.edu.42638: Flags [P.], seq 65:73, ack 72, win 235, options [nop,nop,TS val 917926 ecr 910750], length 8: OpenFlow version 1.0, type ECHO_REPLY, length 8, xid 0x00000000 
   06:02:48.570174 IP pcvm2-4.genirack.nyu.edu.42638 > pcvm1-3.genirack.nyu.edu.6633: Flags [.], ack 73, win 237, options [nop,nop,TS val 910750 ecr 917926], length 0
   27 packets captured 
   27 packets received by filter 
   0 packets dropped by kernel
    
3. Kill Ryu Controller
1. ctrl + c
1. Host2 is able to be pinged normally
1. PING host2-link-0 (10.10.1.2) 56(84) bytes of data.
   64 bytes from host2-link-0 (10.10.1.2): icmp_seq=1 ttl=64 time=1.15 ms
   64 bytes from host2-link-0 (10.10.1.2): icmp_seq=2 ttl=64 time=0.875 ms
   64 bytes from host2-link-0 (10.10.1.2): icmp_seq=3 ttl=64 time=0.763 ms
   64 bytes from host2-link-0 (10.10.1.2): icmp_seq=4 ttl=64 time=0.731 ms
   --- host2-link-0 ping statistics ---
   4 packets transmitted, 4 received, 0% packet loss, time 3001ms
   rtt min/avg/max/mdev = 0.731/0.881/1.156/0.168 ms
    
4. Check the flow table entries
1. sudo ovs-ofctl dump-flows br0
1. NXST_FLOW reply (xid=0x4): 
       cookie=0x0, duration=1175.389s, table=0, n_packets=13, n_bytes=1162, idle_age=77, in_port=3,dl_dst=02:71:cd:30:b0:68 actions=output:1 
       cookie=0x0, duration=1175.393s, table=0, n_packets=15, n_bytes=1358, idle_age=77, in_port=1,dl_dst=02:be:af:04:66:ea actions=output:3
5. Delete the entries on the OVS
1. sudo ovs-ofctl del-flows br0
2. Unable to ping host2
1. PING host2-link-0 (10.10.1.2) 56(84) bytes of data.
   From host1-link-3 (10.10.1.1) icmp_seq=9 Destination Host Unreachable
   From host1-link-3 (10.10.1.1) icmp_seq=10 Destination Host Unreachable
   From host1-link-3 (10.10.1.1) icmp_seq=11 Destination Host Unreachable
   --- host2-link-0 ping statistics ---
   14 packets transmitted, 0 received, +3 errors, 100% packet loss, time 13095ms
   pipe 3
6. Soft Timeouts
1.  This determines for how long the flow will remain in the forwarding table of the switch if there are no packets received that match the specific flow. As long as packets from that flow are received the flow remains on the flow table.
7. Hard Timeouts
1. This determines the total time that a flow will remain at the forwarding table, independent of whether packets that match the flow are received; i.e. the flow will be removed after the hard timeout expires.

d) Download the Ryu apps

To help you get started with your controller writing, we will provide:

• skeleton files for the controllers where you only need to complete some missing functionality
• the solution: fully implemented controllers
• a utility library that makes some of the Ryu messages easier to write

1. Execute this line from the controller
1. mkdir /tmp/ryu/ryu/ext
2. cd /tmp/ryu/ryu/ext/
3. sudo wget https://github.com/GENI-NSF/geni-tutorials/raw/master/OVSRyu/ryu-intro-ctrlapps.tar.gz
4. sudo tar xvfz ryu-intro-ctrlapps.tar.gz
2. Tips For Writing a Controller

In order to make this first experience of writing a controller easier, we wrote some helpful functions that will abstract some of the particularities of Ryu away. These functions are located in /tmp/ryu/ryu/ext/utils.py, so while you write your controller consult this file for details.

Functions that are implemented include:

• packetIsIP : Test if the packet is IP
• packetIsARP : Test if the packet is ARP
• packetIsRequestARP : Test if this is an ARP Request packet
• packetIsReplyARP : Test if this is an ARP Reply packet
• packetArpDstIp : Test what is the destination IP in an ARP packet
• packetArpSrcIp : Test what is the source IP in an ARP packet
• packetIsTCP : Test if a packet is TCP
• packetDstIp : Test the destination IP of a packet
• packetSrcIp : Test the source IP of a packet
• packetDstTCPPort : Test the destination TCP port of a packet
• packetSrcTCPPort : Test the source TCP port of a packet
• createOFAction : Create one OpenFlow action
• getFullMatch : get the full match out of a packet
• createFlowMod : create a flow mod
• createArpRequest : Create an Arp Request for a different destination IP
• createArpReply : Create an Arp Reply for a different source IP

e)  Debugging your Controller

1. Print messages
1. Run your controller in verbose mode (--verbose) and add print messages at various places to see what your controller is seeing
2. Check the status in the switch
1. Dump information from your switch
1. sudo ovs-ofctl dump-flows br0  
1. No output
2.  Useful commands to show status of switch
1. sudo ovs-vsctl show
1. 89362c3a-1d64-451b-977d-7b2718a1ba20
      Bridge "br0"
        Controller "tcp:192.86.139.65:6633"
        fail_mode: secure
        Port “eth1”
   Interface “eth1”
        Port “eth2”
           Interface “eth2”
        Port “eth3”
           Interface “eth3”
        Port "br0"
          Interface "br0"
            type: internal
      ovs_version: "2.3.1"
2. sudo ovs-ofctl show br0 
1. OFPT_FEATURES_REPLY (xid=0x2): dpid:000032cae9ca1449 n_tables:254, n_buffers:256
   capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
   actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS
   SET_TP_SRC SET_TP_DST ENQUEUE
   1(eth1): addr:02:57:29:a3:47:61
   config: 0
   state: 0
   speed: 0 Mbps now, 0 Mbps max
   2(eth2): addr:02:e1:ef:aa:68:15
   config: 0
   state: 0
   speed: 0 Mbps now, 0 Mbps max
   3(eth3): addr:02:a1:3b:26:75:80
   config: 0
   state: 0
   speed: 0 Mbps now, 0 Mbps max
   LOCAL(br0): addr:32:ca:e9:ca:14:49
   config: PORT_DOWN
   state: LINK_DOWN
   speed: 0 Mbps now, 0 Mbps max
   OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0
3. Use Wireshark to see the OpenFlow messages

Many times it is useful to see the OpenFlow messages being exchanged between your controller and the switch. This will tell you whether the messages that are created by your controller are correct and will allow you to see the details of any errors you might be seeing from the switch. You can use wireshark on both ends of the connection, in hardware switches you have to rely only on the controller view.

The controller host and OVS has wireshark installed, including the openflow dissector. For more information on wireshark you can take a look at the  wireshark wiki.

Here we have a simple case of how to use the OpenFlow dissector for wireshark

1. ssh to your controller machine using the -Y command line argument
1. ssh -Y <username>@<controller>
2. Assuming that the public IP address on the controller is eth0, run wireshark 
1. sudo wireshark -i eth0&
   

When the wireshark window pops up, you might still have to choose eth0 for a live capture. And you will want to use a filter to cut down on the chatter in the wireshark window. One such filter might be just seeing what shows up on port 6633. To do that type tcp.port eq 6633 in the filter window, assuming that 6633 is the port that the controller is listening on.

f) Run a traffic duplication Controller [IDK What to Do]

In the above example we ran a very simple learning switch controller. The power of OpenFlow comes from the fact that you can decide to forward the packet anyway you want based on the supported OpenFlow actions. A very simple but powerful modification you can do, is to duplicate all the traffic of the switch out a specific port. This is very useful for application and network analysis. You can imagine that at the port where you duplicate traffic you connect a device that does analysis. For this tutorial we are going to verify the duplication by doing tcpdump on two ports on the OVS switch.

1. Use the interfaces that are connected to host2 and host3 
1. Software Switch (OVS): You should have noted down the interfaces for host2 and host3 in section 2a. If you have not noted the interfaces for host2 and host3 down (in section 2a), you can run tcpdump on OVS interfaces to figure out the interfaces for host2 and host3. This will allow you to see all traffic going out the interfaces.

2. sudo tcpdump -i eth1
   sudo tcpdump -i eth2

2. In the controller host directory /tmp/ryu/ryu/ext you should see two files
1. myDuplicateTraffic.py : This is the file that has instructions about how to complete the missing information. Go ahead and try to implement your first controller.
2. DuplicateTraffic.py : This has the actual solution. You can just run this if you don't want to bother with writing a controller.
3. duplicate.config : in this file, you need to specify which port you want to duplicate traffic to. We will be duplicating host2 traffic and send it to host3, so put host3 port number here. We noted down port numbers in section 2a. To figure out which port maps to which interface, use can also use "sudo ovs-ofctl show br0" on ovs node. (You can use any editor to edit the file, e.g. use nano by typing "sudo nano duplicate.config")
3. Run your newly written controller to duplicate the traffic. (update duplicate.config file with host3 port number. We noted it down in section 2a ). You can also use the solution file DuplicateTraffic.py to duplicate traffic: 

1. cd /tmp/ryu
   ./bin/ryu-manager ryu/ext/myDuplicateTraffic.py

4.  To test it go to the terminal of host1 and try to ping host2: 
1. ping host1
5.  Stop the Ryu controller using Ctrl-C.

g) Run a port forward Controller

Now let's do a slightly more complicated controller. OpenFlow gives you the power to overwrite fields of your packets at the switch, for example the TCP source or destination port and do port forwarding. You can have clients trying to contact a server at port 5000, and the OpenFlow switch can redirect your traffic to a service listening on port 6000.

1. Under the /tmp/ryu/ryu/ext directory there are two files: PortForwarding.py and myPortForwarding.py that are similar to the previous exercise. Both of these controllers are configured by a configuration file at ext/port_forward.config. Use myPortForwarding.py to write your own port forwarding controller.
2. To test your controller we are going to use netcat. Go to the two terminals of host2. In one terminal run: 
1. nc -l 5000
2. nc -l 6000
3. Now, start the simple layer 2 forwarding controller. We are doing this to see what happens with a simple controller. 
1. cd /tmp/ryu

   ./bin/ryu-manager ryu/ext/simple_switch.py

4.  Go to the terminal of host1 and connect to host2 at port 5000: 

1. nc host2 5000

5. Type something and you should see it at the terminal of host2 at port 5000. 
1. You should be able to send something from host1 and receive it on what ever port is specified on host2. The controller output
6.  Now, stop the simple layer 2 forwarding controller by Ctrl-C. 
7. And start your port forwarding controller (if you have written your controller then use myPortForwarding in the following command): 
1.  ./bin/ryu-manager ryu/ext/PortForwarding.py
8. Repeat the netcat scenario described above. Now, your text should appear on the other terminal of host2 which is listening to port 6000. 
9. Stop your port forwarding controller using Ctrl-C.

h) Run a Server Proxy Controller 

As our last exercise, instead of diverting the traffic to a different server running on the same host, we will divert the traffic to a server running on a different host and on a different port.

1. As our last exercise, instead of diverting the traffic to a different server running on the same host, we will divert the traffic to a server running on a different host and on a different port.
2. On the terminal of host3 run a netcat server
1. nc -l 7000
3. On your controller host, open the /tmp/ryu/ryu/ext/myProxy.py file, and edit it to implement a controller that will divert traffic destined for host2 to host3. Before you start implementing think about what are the side effects of diverting traffic to a different host.  
1. Is it enough to just change the IP address?
2. Is it enough to just modify the TCP packets?
4. To test your proxy controller run (if you have written your controller then use myProxy in the following command): 

1. cd /tmp/ryu
   ./bin/ryu-manager ryu/ext/Proxy.py

5. Go back to the terminal of host1 and try to connect netcat to host2 port 5000  

1. nc 10.10.1.2 5000

6. If your controller works correctly, you should see your text showing up on the terminal of host3

i) Delete your bridge

Before moving to the next step make sure you delete the bridge you have created, especially if you are using the same reservation for a different exercise:

sudo ovs-vsctl del-br br0

Part 3: Finish

Step 4: Teardown Experiment

 Resources