Internet2 Routing for the MOREnet Network

Document Links

Contents Introduction Background Issues, Concerns and Discussion Bandwidth Futures Conclusion Illustrations Participant Router Connectivity UM Routing Relationships Printer friendly version Internet2 Routing (PDF file, 214 KB) Get Acrobat Reader

Introduction

MOREnet has never officially documented its routing policy. Past and current routing techniques have been primarily based on geographic boundaries; Internet connections in a regional hub have been the preferred ingress/egress points for customers served in that region with other regional connections serving as backup paths. Having been presented at various member meetings in the past, this strategy is fairly well understood in the MOREnet community. The techniques have worked well and no changes are anticipated regarding MOREnet's Internet1 routing implementation. However, recent and planned Internet2 activities have introduced some new complexities.

This document proposes a routing strategy supporting the MOREnet community participating in Internet2 activities. It does not address I2 bandwidth allocations, alternate (fail-over) access, or Quality of Service issues. While these are important topics they are outside the scope of this document.

Back to top

Background

MOREnet's current Internet2 routing policy uses the ATM fabric of the network to build a virtual private network (VPN) overlay for those customers participating in Internet2 projects. Currently three MOREnet institutions are Internet2 participants, a fourth will come online later this spring. As currently implemented, the overlay provides separate bandwidth and local routes to Internet2 destinations outside of MOREnet. The VPN is not used for exchange of data between the MOREnet participants; the Internet1 connection is used for all intra-state traffic exchanges. Figure 1 shows connectivity from an Internet2 participant router's point of view.

Figure 1: Participant Router Connectivity

At the Fall 2000 Internet2 members meeting in Atlanta, UCAID announced plans to organize a K-20 initiative (Abilene Update - http://www.internet2.edu/resources/abilene-update-fallmtg00.ppt). The initiative is envisioned as a collaborative venture involving national education and technology organizations, statewide education networks, and educational institutes, among others. Missouri has been selected to participate in the initiative and will be among the first state networks to fully connect.

Back to top

Issues, Concerns and Discussion

There are two issues to be considered regarding MOREnet's current Internet2 routing policy and working with the Internet2 community. The first is how routing should be handled within the state between Internet2 research participants. For example, should inter-organizational traffic use the Internet1 connection or should it flow over the VPN research connection as if it were an exchange of traffic with any other Internet2 institution? The second issue is how Internet2 traffic for the K-20 initiative should be brought into the picture. In particular, should the research VPN be dismantled and a single connection be used where all Internet2 traffic competes? Or should the existing VPN be maintained and a second connection be built to support the rest of the state network?

The concern with intrastate traffic traversing the research VPN is the potential impact of non-research traffic competing directly for the available Internet2 bandwidth and the VPN in general competing with Internet1 traffic. The University of Missouri's inter-campus traffic today flows over their respective Internet1 connections -- even if it is research oriented. Adopting the UCAID policy (Abilene Conditions of Use - http://www.internet2.edu/abilene/html/cou.html) that any traffic between Internet2 primary participants may use the Internet2 path would allow University of Missouri inter-campus traffic to begin using the research VPN. It is important to note that the University of Missouri administrative systems are migrating to network(s) separate from the Columbia campus (Information provided by UM-System, February, 2001). The current plan is to move the IBM MVS from the Columbia campus network to a (non-research VPN) UM-System network system within the next six months. Following implementation of this routing policy and until the move is completed, administrative traffic between the campuses would use the research VPN to reach the MVS system. After completion of the move, traffic destined for the MVS system would use the commodity MOREnet backbone (the Internet1 path). The University of Missouri-Columbia's inter-campus traffic averages less than 500 Kbps inbound or outbound with other inter-campus traffic well below that (According to traffic analysis conducted by MOREnet during February, 2001).

Adopting the UCAID routing policy would result in some changes to the traffic pattern between campuses. Specifically, traffic between any Internet2 participants would traverse the MOREnet managed Internet2 gateway router located in Kansas City. These paths would be visible with any traceroute tool. In no cases would the path length be increased, in some cases the path length would be less. The physical path of packets in some specific cases would change remarkably; for example packets between UM-Columbia and UM-St. Louis would travel to Kansas City for forwarding (but with less routing hops). Clearly the performance and utilization levels along the network's I-70 corridor must continue to be closely monitored. Figure 2 on the following page shows the routing relationship between University of Missouri campuses following adoption of the UCAID Internet2 routing policy.

Figure 2: University of Missouri I2 Routing Relationships

UCAID's K-20 initiative permits the MOREnet community access to all sites connected to Abilene today and to other participants of the initiative. As described earlier, two methods are available to provide the connectivity.

1. Provide a single path shared by all to I2/Abilene destinations, or
2. Provide a second (logical) path to I2/Abilene supporting the K-20 initiative, leaving the existing research VPN in place and functioning as today.

In both cases MOREnet core routers should use Abilene as the preferred path if multiple paths exist. The concern is that the first method does not easily provide the ability to differentiate services. Specifically:

1. Differentiation on a primary participant access circuit. The current research VPN permits up to 45 Mbps of access to I2/Abilene. The participant's available access to Internet1 will typically be less. With no differentiation, available bandwidth leaving a MOREnet research location must either be restricted at the Internet1 level, or opened up to the Internet2 level. Neither are acceptable options.
2. Differentiation on the aggregated I2/Abilene circuit. Without differentiation between "research" traffic and "eligible" traffic, all traffic destined for I2/Abilene locations competes for bandwidth. The potential for aggregated "eligible" traffic to impede "research" traffic is real, but unlikely to be an immediate problem (Reports identifying MOREnet's Abilene "eligible" traffic peak in the 14-16 Mbps range, February, 2001). The potential for aggregated "research" traffic to impede "eligible" traffic is not only very real, but likely. This is compounded by the remote nature of the participants from the gateway; in aggregate they could consume significant amounts of backbone bandwidth as well.

It is clear from this discussion that the ability to differentiate services must continue to be available. Implementing differentiated services over a single connection using existing equipment requires added complexity to participant router configurations as well as configurations supporting the I2/Abilene gateway router(s). MOREnet's most effective delivery of differentiated services today, and with existing equipment, is through the deployment of VPN(s).

The second method described, maintaining the existing research VPN and building a separate connection in support of the K-20 initiative, allows continued delivery of advanced research activity as in the past and permits differentiation between K-20 initiative and research VPN traffic. Easily implemented on current facilities, it also permits the expected utilization reporting requirements of the K-20 initiative to be supported in a simple manner. And in the event the K-20 initiative is dismantled at the end of the project, it is very simple to remove those logical routing interfaces and the necessary support without disrupting traffic on the research VPN.

Back to top

Bandwidth Futures

MOREnet's current commitment to advanced research networking provides for a 45 Mbps infrastructure across the MOREnet backbone and gateway services to the Abilene network. This commitment is implemented as a VPN and is shared equally by participants. The VPN is configured across the MOREnet backbone OC-3 and DS-3 circuits into a gateway router in Kansas City. The Great Plains GigaPOP provides MOREnet's connection point to the Abilene network and an OC-3 provides the physical connectivity between MOREnet and the GigaPOP. A 45 Mbps virtual circuit provides the interconnect from the MOREnet research VPN to the Abilene network. The individual campus connection to the VPN permits the institution to use all 45 Mbps when there is no competition from other participants.

Using "Program Enhancement" funding, the University of Missouri has chosen to augment MOREnet's commitment by funding additional bandwidth for the research VPN. In late-spring/early-summer 2001, an OC-3 funded jointly by the University of Missouri and MOREnet will be added to the MOREnet backbone between Columbia and Kansas City. This will be used to increase the bandwidth available to the research VPN on that portion of the backbone up to a minimum of 90 Mbps. The bandwidth from MOREnet into the Great Plains GigaPOP will be increased as well. The exact bandwidth allocation and engineering details are yet to be finalized. The University of Missouri and MOREnet will jointly determine the specific implementation.

Back to top

Conclusion

It is important as Internet2 projects expand into the education community beyond the higher education research community that implementation and operational issues be clearly understood. This paper specifically describes a routing policy, setting aside other important issues. These other issues need to be resolved in the near term and as such, MOREnet recommends an Internet2 Engineering Committee be formed to provide guidance regarding implementation and operational issues relative to research networking. First topics should include bandwidth allocation and Quality of Service. The committee should have representatives from each primary I2 participant and MOREnet.

As the thrust of this document, MOREnet recommends the following items in support of an Internet2 routing policy:

1. MOREnet will maintain a research VPN supporting institutions involved in I2/Abilene research projects. Available bandwidth shall be allocated according to guidelines developed by the I2 Engineering Committee.
2. By default, all I2/Abilene routes shall be available to participants including other MOREnet I2/Abilene destinations. Unless instructed otherwise by the I2 Engineering Committee, the research path shall be the preferred path among alternate path selections.
3. A separate logical path shall be established to the Abilene network in support of the UCAID K-20 Initiative. It shall be permitted to use that portion of the bandwidth between MOREnet and I2/Abilene not in use by the research VPN, up to and including the entire facility. A minimum bandwidth allocation shall be determined upon further study.

Back to top