TCP Evaluation Suite
Evaluating New Congestion Control Schemes and Its Impact on Standard TCP NewReno

Summary

Sep 1, 2007 by cesar (based on paper [1])

It has been recognized that standard TCP NewReno throughput deteriorates in high-speed networks with large bandwidth-delay product, and new congestion control algorithms have been proposed to address such deterioration.

Assuming that the proposed congestion control schemes would be in general used in the Internet, it is imperative to study the interaction among flows of different schemes, in addition to the interaction among flows using the same congestion control.

Thus, we propose a new TCP evaluation suite that can assess interaction among different congestion control proposals. By applying the same experiment setup, including network configuration, flow parameters, and workload of each flow, to multiple experiment runs for the different schemes, we can assess flow-by-flow behavior of different schemes.

In addition to such realistic simulation environment (provided by the topology generator) that creates networks with multiple bottlenecks, a large number of short-lived and long-lived flows, and a variety of RTTs. We also provide automatic analysis and graphing scripts that enable TCP researchers to explore in-depth questions about efficiency, RTT-fairness, TCP NewReno friendliness, and cwnd dynamics in every scenario tested.

Pre-Requisites

Jun 18, 2007 by cesar

This TCP Evaluation Suite was originally based on the NS-2 Linux [site 2], even though we intend to make it more general in the future. Our main goal was to provide as much realistic congestion control implementations as possible, thus all the simulation code is based on original Linux TCP code (2.6.13 with proper patches applied as to be compatibe to 2.6.20 [site 2]).

# Download ns-2.31 [3] tar xzf ns-allinone-2.31.tar.gz export NSDIR=$HOME/ns-allinone-2.31/ns-2.31 cd ns-allinone-2.31 # download HERE the [ns-2.31 TCP Linux Patch] cd ns-2.31 patch -p1 < ../ns-linux-2.31.patch cd .. ./install # in addition to this patch and ns-2.31 # tcl, perl, gnuplot and graphviz are also needed! # it will be create a directory validation inside /tcp/linux # you can validate comparing results of the linux implementations # and the Caltech ns-2 linux patch website.

TCP Evaluation Suite Download and Usage

July 18, 2007 by cesar
Once, you have a running "ns-2 TCP Linux framework" ready for simulations. You can download our TCP Evaluation Suite. From which we can build different topologies, different traffic scenarios, and automatically generate an extensive comparative analysis among schemes.

# Download TCP Evaluation Suite V1.1 [http://netlab.cs.ucla.edu/tcpsuite/tcp_suite_v1.1.tar.gz] cd $NSDIR/tcp/linux tar xzf tcp_suite_v1.1.tar.gz cd tcp_suite As you can see, the TCP Evaluation Suite is organized in three subdirectories.

  • Topology Generator
  • Experiment Executor
  • Analysis

Topology Generator

This set of tools allows TCP researches to build realistic internet topologies and condition by using randomly assigned link delays and workloads to be used in the simulation execution. In addition, it was included automatic topology drawing scripts (based on Graphviz [download] in order to help researchers see a big picture of the network conditions and understand/debug dynamics of individual flow level.

# Run the execute.sh script to create a pre-configured set of conditions # It will create a set of scenarios based on a 4 core router all 1Gbps network # Each scenario will have the same amount of short lived flows = 100 # And a variable amount of long lived flows = [1 5 10 20 30 40 50 60] # For every combination, 12 differents random seeds are used export NSDIR=$HOME/ns-allinone-2.31/ns-2.31 cd topology_generator sh ./execute.sh # generate automatic graphs of the topology and flow composition # for every condition/seed it will be generated 5 topology planar graphs # 1) forward_flows clusterized by number of hops, # 2) forward_flows organized in a single forward overall view, # 3) backward_flows clusterized by number of hops, # NOTE: we draw only the long-lived flows sh ./process_graphs.sh

Experiment Executor

The next phase is to choose the congestion control schemes to be evaluated. Since this TCP Evaluation Suite was designed to evaluate the interaction of protocols with the standard TCP NewReno as well as the properties of the protocol alone (in an homogeneous environment).

We need to run several protocols combinations in a single topology-generated environment. For the purpose of example, we will use the topology scenario of 5 core routers, RTTs 15 ms exponentially distributed on all links, 2MB buffers, 100 short-lived flows and 30 long-lived flows.

We chose "NewReno, H-TCP and CUBIC" as an example of usage. Thus, we need to execute simulations of the following combinations: (reno vs reno), (htcp vs htcp), (cubic vs cubic), (htcp vs reno) and (cubic vs reno). These 5 combinations are necessary to study the conditons previously mentioned homogeneous and heterogeneous flows. Following is the sequence of commands to execute these simulations.

export NSDIR=$HOME/ns-allinone-2.31/ns-2.31 # copy the directory with the desired topology into the execution directory # the directory with the generated topology must be called "model" cd .. cd experiment_executor cp -R ../topology_generator/long_lived30 model # change the configuration script "change_protocol.sh" # for one of the combinations it will automatically create # execution scripts for every seed (exp-seed1) # one just need to inform which "versions" will be executed # i.e.: reno vs reno (1), cubic vs cubic (2), cubic vs reno (3) ... sh ./change_protocol.sh # the configuration of traffic type and file sizes is defined in "script.tcl" # one can execute every script in separate for every seed # or if you have enough computing power you can just execute the batch script # NOTE: 4 AMD-64 proc 16GB RAM it takes about 5 hs to complete one combination nohup sh ./scenario.sh simulation_name # the scenario script will execute several "submit script" automatically # and it will create directories according to the combinations chosen # note: data and data4 that will keep flows and network statistics # after the execution of all the combinations # NOTE: change the email address inside scenario.sh # in the end, scenario.sh will send an email automatically to the # author informing that the simulation is over # it will take about 25 hours of simulation on a top machine

Analysis

The last phase of the TCP Evaluation Suite is the consolidation of results. We have created a set of scripts to automatically extract all the information necessary for a complete assessment of the congestion control algorithms and its impact on TCP NewReno.

export NSDIR=$HOME/ns-allinone-2.31/ns-2.31 # move all the data and the topology to the analysis directory # make sure to do a backup before analysis cd experiment_executor tar czf data071807.tar.gz data data4 model cd .. cd analysis mv ../experiment_executor/data/ . mv ../experiment_executor/data4/ . mv ../experiment_executor/model/ . # in order to post-process the performance big picture # include in a listing file all the protocol combinations tested vi process_protocol ------------ htcphtcp htcpreno cubiccubic cubicreno renoreno ------------ process_protocol # run the script preparedata.pl to set the correct parameters # of the protocols and graphs to be generated # pass as parameter the total number of flows and # the number of long-lived flows based on the topology-generated sh ./preparedata.pl # change the file processdata # such that it reflect the number of core routers utilized, and # the number of flows and number of long lived flows sh ./processdata # this process takes about 20 minutes to process all the data # separate in another listing file, the protocol combinations that # you would like to plot (we use a different file to give flexibility) vi graph_protocol ------------ htcphtcp htcpreno cubiccubic cubicreno renoreno ------------ graph_protocol # generate the graph gnuplot script and create the graphs # about 12 png plots will be created perl generategraphs.pl > run sh ./run # you can also plot the results in PDF sh ./pdf_results.sh # Finally, in order to plot the cwnd dynamics of the long lived flows # and core router buffer utilization and losses, do the following # the example will plot side-by-side cubic and htcp individual flows # dynamics so one can study the protocols dynamics in depth sh ./flow_dynamics.sh # Note: Change the script flow_dynamics such that you can pass the # correct data4 directories depending on the protocol combination.