WANG+Yangyang+BI+Jun

Abstract Software defined networking （SDN） has been applied increasingly in practical networks. Currently， SDN is mainly used to improve the flexibility and efficiency of datacenter networks， enterprise networks and wide?area networks （WAN）. There also emerge some studies that try to deploy SDN to inter?domain settings. In this article， we introduce the progress stages of inter?domain SDN and studies related to each stage. Finally， we discuss the applications and challenges of inter?domain SDN.

Keywords software defined networking； network programmability； inter?domain SDN

1 Introduction

In recent years， the development of software defined networking （SDN） provides an opportunity for networking innovations. SDN decouples the control plane and data plane of networks. The data plane provides common programmable resources and interfaces of network devices， which makes it possible to program network centrally and greatly simplifies and improves network management and control. Many studies of SDN focus on taking logically centralized control plane in single domain in campus networks， enterprise networks， data centers， and private wide?area networks （WANs）. If we extend SDN to inter?domain settings across multiple domains， we can take advantage of its opportunity of programmability and innovation in the Internet. The flexibility of SDN control looks forward to making it possible to optimize scheduling for inter?domain networking resources. However， the inter?domain setting is a scenario of distributed administration. There are more than 500，000 autonomous systems （ASes） in the global Internet. The growth of the number of ASes indicates a super?linear tread based on the statistics from global BGP routing table. Therefore， SDN will meet the scalability issue when applying SDN to the environment of distributed control in inter?domain settings. In a word， there are both challenges and opportunities in the inter?domain SDN researches.

The “SDN domain” in this article generally refers to the SDN control domain， which is a network domain deployed SDN mechanism and administrated by operators to control independently. An SDN domain can be an AS of the Internet， or a network control domain composed of a number of ASes， and may even be a SDN deployment domain with no AS number in the future. The inter?domain SDN mechanisms studied in this paper are around how to solve the problem of inter?domain control cooperation between SDN domains in the global Internet scale， mainly referring to inter?domain SDN applied to the autonomous system level. If this problem cannot be solved， the SDN mechanism can only be used in a single control domain or local scale networks （such as enterprise networks and data networks）， but cannot provide programmability for routing and control applications across multiple domains.endprint

We introduce inter?domain SDN according to the areas of its deployment.

2 Small SDN Networks Interconnected with

Border Gateway Protocol （BGP） Domains

Deploying a new SDN network into an autonomous domain is the most common approach. However， how this newly deployed SDN network is interoperable with other BGP network domains is an important issue. The BTSDN mechanism [1] is an example of how to solve this problem. The principle of BTSDN is to deploy a new SDN area in an AS instead of replacing the existing BGP boundary router. On the control plane， the controller of the SDN region exchanges routing information with the BGP routers located at the AS boundary by running the APP of the internal BGP （iBGP）， and then connects with other ASes via external BGP （eBGP） sessions. The data plane adopts the mechanisms of Address Resolution Protocol （ARP） Proxy and media access control （MAC） address Rewrite to ensure the delivery of IP packets between the BGP border routers and the SDN region. The deployment cost of BTSDN is very small， which does not need to replace the existing BGP routers and only requires to deploy a new SDN area in an AS. It realizes control function on the controller， and complete the necessary control on the data plane.

3 Interconnection Between SDN Domains

and BGP Domains

There are many representative technologies to interconnect SDN domains with BGP domains， including RouteFlow [2]， SDN?IP [3]， and Software Defined Internet Exchange Points （SDXs） [4]. SDN devices are adopted to realize this interconnection. Aiming at various targets， these solutions have their own suited scenarios. RouteFlow [2] focuses on providing services of virtual routers. By replacing a commercial router with an OpenFlow switch that can be remotely programmed to control， the control logic function of BGP routers is moved to the virtual router machine， each OpenFlow switch corresponding to a virtual machine. The SDN?IP technology in [3] focuses on the seamless peering interconnection between SDN control domains and traditional BGP routing domains based on BGP， and the realization of SDN control domains as transit networks of the traditional BGP routing domains， in order to promote the gradual deployment of SDN in the existing networks to replace the traditional BGP ASes. The controller of an SDN control domain still uses BGP protocol to exchange routing information with the neighboring traditional BGP routing domains， but uses SDN centralized mode to control local ASs BGP routing calculation and installation. It is beneficial to the efficient control of BGP routing in a local AS.endprint

Princeton University proposed SDX， which mainly transforms the traditional Internet Exchange Point （IXP） and route server （RS） based on SDN， to achieve flexible policies of interconnection between multiple ASes through SDX （Fig. 1）. The SDX switching infrastructure uses OpenFlow switches to provide flexible， fine?grained inter?domain traffic switching policies for two or more member ASes connected to a SDX. Compared with the traditional IXPs based on RS， the management efficiency and functional flexibility are improved. The limitation of SDX is that SDN control services can only be provided for inter?domain traffic between member ASes. The Swiss Federal Institute of Technology Zurich in Europe proposed the Control Exchange Points （CXPs） [5]， as shown in Fig. 2. Multiple IXPs stitch multiple segments of inter?domain routing paths under SDN centralized control to form a cross?domain end?to?end path that meets the performance requirements of Quality of Service （QoS）.

The study in [6] tried to build a larger SDN control domain， using a super SDN controller to implement centralized control and management of the ASes that belong to the same organizational structure and are distributed around the world， in order to improve the convergence efficiency of the inter?domain routing among these ASes. The characteristics of these researches are using SDN centralized management and control mode to improve the function and performance of the current BGP routing. These solutions make it easier to find inter?domain SDN applications and requirements in the current networks. However， they are limited by the traditional network compatible scenarios and cannot fully take advantages of SDN for fine?grained and flexible control of traffic forwarding.

4 Interconnection and Cooperation Between

SDN Domains

The interconnection and cooperation between SDN domains are more forward?looking and challenging. In RFC draft [7]， BGP was proposed to transfer IP routing reachability messages between federated SDN controllers in data centers. Literatures [8]-[10] propose the hierarchical structure of the controllers in multiple SDN control domains. However， the control plane interconnected by these multi?domain SDN networks is still under centralized management and control， and is not applicable to the distributed and autonomous environment of ASes on the Internet. The literature [11] proposes using NOX controllers and OpenFlow switches to implement a BGP?like distributed inter?domain routing protocol for multiple SDN domains. In some literatures [12]， [13]， multiple SDN domains exchange information to provide end?to?end cross?domain path service satisfying QoS performance. These proposals are appropriate for the scenario of federated SDN domains， and are not appropriate for large?scale Internet?wide environments. In [14]， Extensible Session Protocol （XSP） is proposed. XSP is a high?level session layer protocol located on the transport layer， which is used as the interaction interface between network applications and network services. SDN focuses on the design of a high?level session layer protocol itself that is only for specific application services， but does not address the general underlying interconnection mechanism between domains. In addition， there are a number of projects that try to provide Network Service Interface （NSI） for the upper network applications to realize SDN virtual resource sharing among multiple SDN domains， such as Japan and the EU co?funded project FP7 FELIX [15]. Its main purpose is the high?level inter?domain resource sharing. Although the above researches have their application scenarios and values， they cannot provide a universal inter?domain SDN interconnection mechanism.endprint

The goal of interconnection of SDN domains is to provide general mechanism for global Internet?scale interconnection and cooperation of SDN domains. This kind of research is still in the initial stage. A project team of Tsinghua University has conducted this research earlier in the world， and proposed a new cooperative inter?domain SDN mechanism， West?East Bridge （WE?Bridge） [16].

WE?Bridge mechanism is a new type of SDN East?West interconnection mechanism proposed by Tsinghua University based on the 12th Five?Year “863” national project “Future Network Architecture and Innovation Environment”. Fig. 3 shows the position of the WE?Bridge in the proposed Future Internet Innovation Environment （FINE） architecture in this “863” project. WE?Bridge is an extension of the network operating system layer and virtual platform layer of FINE. The SDN controller exchanges the information of virtual network view of the controlled domain based on WE?Bridge， which provides a collaborative interface for a variety of inter?domain applications （such as new inter?domain routing protocols and path computation）， and realizes the cross?domain collaboration for SDN applications. The East?West interface protocol for cross?domain cooperation of WE?Bridge includes a mechanism of establishing peering connection between SDN domains， optimized distribution mechanism of the exchanged information （routing strategy of each SDN domain， virtual network views） among SDN domains， message format and negotiation process. Based on WE?Bridge， a new type of fine?grained inter?domain routing application is proposed to verify the effectiveness of the proposed mechanisms. The proposal of WE?bridge has begun a preliminary attempt for large?scale SDN inter?domain interconnection.

In 2012， the Chinese?American Network Symposium （CANS） established the future Internet/SDN working group， chaired by the Interent2 CTO Dr. Stephen Wolff and Professor BI Jun of Tsinghua University. The main content of this working group charter is to carry out the research of an inter?domain SDN testbed， and application innovations on the inter?domain SDN testbed. The working group has concluded that the inter?domain SDN testbed was unable to adopt a centralized structure， and therefore adopted the WE?Bridge mechanism proposed by Tsinghua University. At the Interent2 and APAN joint conference in Hawaii in January 2013， the working group discussed and determined the technical solution of Tsinghua University. This meeting was held at the East West Center of University of Hawaii System. Inspired by this， this mechanism is named WE?Bridge. On the basis of the design and implementation of WE?Bridge， Chinese Education and Research Computer Network （CERNET）， the United States Internet2， China Science and Technology Network （CSTNET）， Holland academic network SURFnet， the Japanese academic network （APAN?JP/JGN?X） cooperated and established the first cooperative and international inter?domain SDN test network. The graph user interface shows the topology of this testbed in Fig. 4. This inter?domain SDN testbed had been demonstrated at such conferences as CANS from 2013 to 2015， the International Conference on SuperComputing 2013 and 2014， APAN 2014 and 2015， the global Open Network Summit （ONS） 2014. The test network was also accepted as the IEEE INFOCOM 2014 demo [18]. Dr. Stephen Wolff published the technical evaluation [19]， [20] at the Internet2 website He believes that the current SDN mechanism can only be used in single domain environment， and this pioneering work on inter?domain SDN demonstrates that SDN mechanism can be extended to multi?domain environment in key global scientific cooperation.endprint

5 Deployment Stages of Inter?Domain SDN

SDN network and traditional IP network have a long coexistence transition period. We believe that inter?domain SDN deployment has the processing stages shown in Fig. 5.

The first stage is a new SDN network deployed locally in an autonomous system， which is connected with the domains controlled by traditional BGP. The second stage is to use SDN mechanism to control the whole autonomous system， and to connect with the traditional BGP domain. For example， an AS or IXP uses SDN mechanism to control network traffic forwarding and routing policies. The first and second stages mainly focus on how to implement the interworking between a new SDN domain and a traditional BGP domain.

When SDN has been deployed in many ASes， the third stage forwards further to study how to cooperate control between SDN domains， in order to give full play of SDN programming in the inter?domain routing control. The two ASes may be physically adjacent or across multiple traditional IP networks， and control session negotiation and data traffic forwarding are established by overlay scheme. The third stage faces more opportunities and challenges than the previous stages. This stage enables collaboration between many SDN enabled domains， with fine?grained programmable control over Internet routing in a wider range. This stage will help contribute to the new network business model. Internet service providers can provide users with flexible and efficient network channels， improve the performance of the Internet， security and control efficiency， and provide a possible opportunity for the rapid deployment of new network protocols on the Internet. However， achieving this target needs to overcome some challenges such as scalability of SDN applications in large?scale Internet.

6 Applications and Challenges of

Inter?Domain SDN

Through the introduction of some typical studies， we summarize the prospects and problems of inter?domain SDN research. The mechanism of inter?domain SDN can bring applications as follows： （1） Inter?domain SDN can provide rich and flexible inter?domain traffic control capabilities， such as the applications in inter?domain traffic engineering and distributed denial?of?service （DDoS） attack defense； （2） inter?domain SDN extends SDN to larger?scale Internet to improve operational efficiency， overall networking programmability， and new technology deployment； （3） inter?domain SDN can provide new business models for operators and users， such as meeting the requirements of QoS?aware end?to?end path service.endprint

In the meantime， inter?domain SDN mechanisms are faced with the following challenges：

1） Scalability problems. A large number of network views will lead to poor scalability of the control plane. The large number of routing entries from numerous ASes also bring burden to both memory and lookup efficiency of the forwarding engine of the SDN data plane. How to avoid or solve these scalability problems is a challenge for constructing large inter?domain SDN network.

2） Increasing deployment and incentives. Increasing deployment of the SDN mechanisms among ASes should be critical for operators to be compatible with traditional networks.

3） Security problems. The deployment of new technologies inevitably results in security problems. How the inter?domain SDN mechanism can reduce its own security problems and improve the security control of the Internet is a challenge.

7 Conclusions

In recent years， SDN has received great attention from both academia and industry， and has made considerable progress in the field of centralized SDN control and application. However， how to extend the SDN mechanism to the inter domain to support new inter?domain SDN applications is an opportunity and challenge.

This article introduces the researches of typical inter?domain mechanisms， including the WE?Bridge， an inter?domain SDN interconnection mechanism proposed by Tsinghua University. This article also discuss the deployment stages of inter?domain SDN， and the challenges that need to be resolved in future researches.

With the gradual deployment of SDN domains， inter?domain SDN interconnection will gradually become popular. We hope this article can be of some reference value in this field.

References

[1] P. P. Lin， J. Bi， and H. Y. Hu， “BTSDN： BGP?based transition for the existing networks to SDN， ” Wireless Personal Communications， vol. 86， no. 4， pp. 1829-1843， Feb. 2016. doi：10.1007/s11277?015?3145?0.

[2] M. R. Nascimento， C. E. Rothenberg， M. R. Salvador， et al.， “Virtual routers as a service： the routeflow approach leveraging software?defined networks，” in Proc. 6th International Conference on Future Internet Technologies， New York， USA， Jun. 2011， pp. 34-37. doi： 10.1145/2002396.2002405.

[3] P. Lin， J. Hart， M. Kobayashi， et al.， “Seamless interworking of SDN and IP，” ACM SIGCOMM Computer Communication Review， vol. 43， no. 4， pp. 475-476， Oct. 2013. doi： 10.1145/2534169.2491703.

[4] A. Gupta， L. Vanbever， M. Shahbaz， et al.， “SDX： a software defined internet exchange，” ACM SIGCOMM Computer Communication Review， vol. 44， no. 4， pp. 551-562， Oct. 2014. doi： 10.1145/2740070.2626300.endprint

[5] V. Kotronis， M. Rost， P. Georgopoulos， et al.， “Stitching Inter?Domain Paths over IXPs，” in Proc. Symposium on SDN Research， Santa Clara， USA， Mar. 2016， Article No. 17. doi： 10.1145/2890955.2890960.

[6] H. Kazmi， H. Nawaz， M. A. Gulzar， et al. （2015， Oct. 25）. BGP is High on SDN： Improving BGP Convergence and Security using SDN and Reassertions [Online]. Available： http：//web.cs.ucla.edu/～gulzar/pdf/bgp.pdf

[7] F. Balus， D. Stiliadis， N. Bitar， and K. Ogaki. （2013， Nov.）. Federated SDN?based Controllers for NVO3 （Network Virtualization Overlays）， draft?sb?nvo3?sdn?federation?02 [Online]. Available： https：//tools.ietf.org/html/draft?sb?nvo3?sdn?federation?02

[8] J. McCauley， A. Panda， M. Casado， et al.， “Extending SDN to large?scale networks，” in Open Networking Summit， Santa Clara， USA， 2013， pp. 1-2.

[9] Y. Fu， J. Bi， Z. Chen， et al.， “A hybrid hierarchical control plane for flow?based large?scale software?defined networks，” IEEE Transactions on Network and Service Management， vol. 12， no. 2， pp. 117-131， May 2015. doi： 10.1109/TNSM.2015.2434612.

[10] M. Karakus and A. Durresi， “A scalable inter?AS QOS routing architecture in software defined network （SDN），” in IEEE 29th International Conference on Advanced Information Networking and Applications （AINA）， Gwangju， Korea， 2015， pp. 148-154. doi： 10.1109/AINA.2015.179.

[11] R. Bennesby， P. Fonseca， E. Mota， et al.， “An Inter?AS routing component for software?defined networks，” in IEEE Network Operations and Management Symposium （NOMS）， Maui， USA， 2012， pp. 138-145. doi： 10.1109/NOMS.

2012. 6211892.

[12] K. Phemius， M. Bouet， and J. Leguay， “Disco： distributed multi?domain SDN controllers，” in IEEE Network Operations and Management Symposium （NOMS）， Krakow， Poland， 2014， pp. 1-4. doi： 10.1109/NOMS.2014.6838330.

[13] S. Civanlar， E. Lokman， B. Kaytaz， et al.， “Distributed management of service?enabled flow?paths across multiple SDN domains，” in European Conference on Networks and Communications （EuCNC）， Paris， France， 2015， pp. 360-364. doi： 10.1109/EuCNC.2015.7194099.

[14] E. Kissel， G. Fernandes， M. Jaffee， et al.， “Driving software defined networks with XSP，” in 2012 IEEE International Conference on Communications（ICC）， Ottawa， Canada， 2012， pp. 6616-6621. doi： 10.1109/ICC.2012.6364805.

[15] FELIX Project [Online]. Available： http：//www.ict?felix.eu

[16] P. Lin， J. Bi， and Y. Wang， “East?west bridge for SDN network peering，” in Frontiers in Internet Technologies， J. Su， B. Zhao， Z. Sun， X. Wang， F. Wang， and K. Xu， eds. Berlin/Heidelberg， Germany： Springer， 2013， pp. 170-181. doi： 10.1007/978?3?642?53959?6_16 vol 401.

[17] P. Lin， J. Bi， S. Wolff， et al.， “A west?east bridge based SDN inter?domain testbed，” IEEE Communications Magazine， vol. 53， no. 2， pp. 190-197， 2015. doi： 10.1109/MCOM.2015.7045408.

[18] P. Lin， J. Bi， Z. Chen， et al.， “WE?bridge： west?east bridge for SDN inter?domain network peering，” in 33rd IEEE International Conference on Computer Communications （INFOCOM14） Workshops， Toronto， Canada， 2014， pp. 111-112. doi： 10.1109/INFCOMW.2014.6849180.

[19] S. Wolff. （2013， Sep. 20）. Inter?Domain， Software?Defined Networking Testbed and Application Demonstrated By Chinese and American Network Engineers [Online]. Available： https：//www.internet2.edu/blogs/detail/5020

[20] E. Moynihan. （2014， Oct. 14）. Groundbreaking Demonstration Sets Stage for New Interdomain SDN Applications and Research Collaborations [Online]. Available： https：//www.internet2.edu/blogs/detail/7174endprint