More About MOREnet & Internet2

Projects Technical Information FAQ Background & History Additional Resources

MOREnet & Internet2

Technical Information

Internet2 Routing for the Missouri Research and Education Network

MOREnet has never officially documented its routing policy. Past and current routing techniques have been based primarily on geographic boundaries; Internet connections in a regional hub have been the preferred ingress/egress points for customers served in that region with other regional connections serving as backup paths. After being presented at various member meetings, this strategy is fairly well understood in the MOREnet community. The techniques have worked well and no changes in MOREnet's Internet1 routing implementation are anticipated. However, recent and planned Internet2 activities have introduced new complexities.

This document proposes a routing strategy to support the MOREnet community participating in Internet2 activities. It does not address I2 bandwidth allocations, alternate (fail-over) access or Quality of Service issues. While these are important topics, they are outside the scope of this document.

See related research.

Background

MOREnet's current Internet2 routing policy uses the ATM fabric of the network to build a virtual private network (VPN) overlay for those customers participating in Internet2 projects. Currently four MOREnet institutions are Internet2 participants. As currently implemented, the overlay provides separate bandwidth and local routes to Internet2 destinations outside MOREnet. The VPN is not used for exchange of data between MOREnet participants; the Internet1 connection is used for all intrastate traffic exchanges. Figure 1 shows connectivity from an Internet2 participant router's point of view.

Internet2 Participant Router Connectivity

Figure 1

At the Fall 2000 Internet2 Members Meeting, UCAID announced plans to organize a K-20 Initiative (Abilene Update). The initiative is envisioned as a collaborative venture involving national education and technology organizations, statewide education networks and educational institutes, among others. Missouri was selected to participate in the initiative and among the first state networks to fully connect.

Issues, Concerns and Discussion

There are two issues to be considered regarding MOREnet's current Internet2 routing policy and working with the Internet2 community. The first is how routing should be handled within the state between Internet2 research participants. For example, should inter-organizational traffic use the Internet1 connection or should it flow over the VPN research connection as if it were an exchange of traffic with any other Internet2 institution? The second issue is how Internet2 traffic for the K-20 initiative should be brought into the picture. In particular, should the research VPN be dismantled and a single connection be used where all Internet2 traffic competes? Or should the existing VPN be maintained and a second connection be built to support the rest of the state network?

The concern with intrastate traffic traversing the research VPN is the potential impact of non-research traffic competing directly for the available Internet2 bandwidth and the VPN in general competing with Internet1 traffic. The University of Missouri's inter-campus traffic flows over their respective Internet1 connections — even if it is research oriented. Adopting the UCAID policy (Abilene Conditions of Use) that any traffic between Internet2 primary participants may use the Internet2 path would allow University of Missouri inter-campus traffic to begin using the research VPN. It is important to note that the University of Missouri administrative systems are migrating to network(s) separate from the Columbia campus (Information provided by UM System, February 2001). The current plan is to move the IBM MVS from the Columbia campus network to a (non-research VPN) UM System network system within the next six months. Following implementation of this routing policy and until the move is completed, administrative traffic between the campuses would use the research VPN to reach the MVS system. After completion of the move, traffic destined for the MVS system would use the commodity MOREnet backbone (the Internet1 path). The University of Missouri-Columbia's inter-campus traffic averages less than 500 Kbps inbound or outbound with other inter-campus traffic well below that (according to traffic analysis conducted by MOREnet during February 2001).

Adopting the UCAID routing policy would result in some changes to the traffic pattern among campuses. Specifically, traffic between any Internet2 participants would traverse the MOREnet-managed Internet2 gateway router located in Kansas City. These paths would be visible with any traceroute tool. In no cases would the path length be increased; in some cases the path length would be less. The physical path of packets in some specific cases would change remarkably; for example, packets between UM-Columbia and UM-St. Louis would travel to Kansas City for forwarding (but with less routing hops). Clearly the performance and utilization levels along the network's I-70 corridor must continue to be closely monitored. Figure 2 shows the routing relationship among University of Missouri campuses following adoption of the UCAID Internet2 routing policy.

University of Missouri System Internet2 Routing Relationships

Figure 2

UCAID's K-20 Initiative permits the MOREnet community access to all sites connected to Abilene and to other participants of the Initiative. Two methods are available to provide the connectivity:

1. Provide a single path shared by all to Internet2/Abilene destinations, or
2. Provide a second (logical) path to Internet2/Abilene supporting the K-20 Initiative, leaving the existing research VPN in place.

In both cases MOREnet core routers should use Abilene as the preferred path if multiple paths exist. The concern is that the first method does not easily provide the ability to differentiate services. Specifically:

1. Differentiation on a primary participant access circuit. The current research VPN permits up to 45 Mbps of access to I2/Abilene. The participant's available access to Internet1 will typically be less. With no differentiation, available bandwidth leaving a MOREnet research location must either be restricted at the Internet1 level or opened up to the Internet2 level. Neither is an acceptable option.
2. Differentiation on the aggregated I2/Abilene circuit. Without differentiation between "research" traffic and "eligible" traffic, all traffic destined for I2/Abilene locations competes for bandwidth. The potential for aggregated "eligible" traffic to impede "research" traffic is real, but unlikely to be an immediate problem (reports identify MOREnet's Abilene "eligible" traffic peak in the 14-16 Mbps range, February 2001). The potential for aggregated "research" traffic to impede "eligible" traffic is not only very real, but likely. This is compounded by the remote nature of the participants from the gateway; in aggregate they could consume significant amounts of backbone bandwidth as well.
3. It is clear from this discussion that the ability to differentiate services must continue to be available. Implementing differentiated services over a single connection using existing equipment requires added complexity to participant router configurations as well as configurations supporting the I2/Abilene gateway router(s). MOREnet's most effective delivery of differentiated services today, and with existing equipment, is through the deployment of VPN(s).

The second method described, maintaining the existing research VPN and building a separate connection in support of the K-20 Initiative, allows continued delivery of advanced research activity as in the past and permits differentiation between K-20 Initiative and research VPN traffic. Easily implemented on current facilities, it also permits the expected utilization reporting requirements of the K-20 Initiative to be supported in a simple way. And in the event the K-20 Initiative is dismantled at the end of the project, it is very simple to remove those logical routing interfaces and the necessary support without disrupting traffic on the research VPN.

Bandwidth Futures

MOREnet's current commitment to advanced research networking provides for a 45 Mbps infrastructure across the MOREnet backbone and gateway services to the Abilene Network. This commitment is implemented as a VPN and is shared equally by participants. The VPN is configured across the MOREnet backbone OC-3 and DS-3 circuits into a gateway router in Kansas City. The Great Plains GigaPOP provides MOREnet's connection point to the Abilene network and an OC-3 provides the physical connectivity between MOREnet and the GigaPOP. A 45 Mbps virtual circuit provides the interconnection from the MOREnet research VPN to the Abilene Network. The individual campus connection to the VPN permits the institution to use all 45 Mbps when there is no competition from other participants.

Using "Program Enhancement" funding, the University of Missouri has chosen to augment MOREnet's commitment by funding additional bandwidth for the research VPN. In late-spring/early-summer 2001, an OC-3 funded jointly by the University of Missouri and MOREnet was added to the MOREnet backbone between Columbia and Kansas City to increase the bandwidth available to the research VPN on that portion of the backbone up to a minimum of 90 Mbps. The bandwidth from MOREnet into the Great Plains GigaPOP will be increased as well. The exact bandwidth allocation and engineering details are yet to be finalized. The University of Missouri and MOREnet jointly will determine the specific implementation.

Conclusion

It is important as Internet2 projects expand into the education community beyond the higher education research community that implementation and operational issues be clearly understood. This paper specifically describes a routing policy, setting aside other important issues. These other issues need to be resolved in the near term and as such, MOREnet recommends an Internet2 Engineering Committee be formed to provide guidance about implementation and operational issues relative to research networking. First topics should include bandwidth allocation and Quality of Service. The committee should have representatives from each primary Internet2 participant and MOREnet. Read MOREnet reports from these and other research topics.

MOREnet recommends the following items in support of an Internet2 routing policy:

1. MOREnet will maintain a research VPN supporting institutions involved in I2/Abilene research projects. Available bandwidth shall be allocated according to guidelines developed by the I2 Engineering Committee.
2. By default, all I2/Abilene routes shall be available to participants including other MOREnet I2/Abilene destinations. Unless instructed otherwise by the I2 Engineering Committee, the research path shall be the preferred path among alternate path selections.
3. A separate logical path shall be established to the Abilene Network in support of the UCAID K-20 Initiative. It shall be permitted to use that portion of the bandwidth between MOREnet and I2/Abilene not in use by the research VPN, up to and including the entire facility. A minimum bandwidth allocation shall be determined on further study.