Skip to main content

Sensitivity Analysis of Event Driven Simulation Results

Murray Pearson and Tony McGregor
The University of Waikato, Hamilton, New Zealand

One use of passive measurement is to provide traces that can be used as the input to event driven simulation. Such simulations have been used to study a wide range of network problems including strategies to achieve good performance over high bandwidth delay satellite[1], terrestrial[2] and asymmetric satellite/terrestrial[3] international links, without requiring users to upgrade or tune their TCP stacks.

The simulator used in these studies was based on the ATM-TN simulator[4]. ATM-TN's TCP model includes the actual TCP code from 4.4 BSD Lite, modified to suit the simulation environment. Connections are simulated on a packet by packet basis and including slow start, congestion control, fast retransmit, and fast recovery algorithms.[5]

The simulation can be considered as a number of connected components. These are the HTTP requesters, the local proxy, the international link, including the routers that feed it (which is the key component under study), the US network and the HTTP servers.

An HTTP traffic model is responsible for creating TCP connections, sending HTTP GET requests, receiving the request at the destination and returning and results and for recording the time required to complete the HTTP requests. The HTTP model makes use of information about hosts and a URL trace collected from a real network using passive measurement.

The delays in the US part of the network are simulated by the HTTP traffic model which releases the packets that make up the HTTP response at a regulated rate so that the complete response arrives at the US proxy at the same mean rate as applied when the page was fetched over the real network.

Simulations, by their nature, are approximations. Some details of the real system are intentionally omitted to make the simulation more manageable. For example it is possible to drive a simulation from a trace of high level protocol events (e.g. HTTP requests) or from a packet level trace. In the case of TCP packet control level behaviours (such as slow start) are important. They are not always important with UDP based traffic such as a game or mpeg replay. It is part of the art of simulation to omit details that are difficult to provide but make little difference to the final result while maintaining the essential elements of the situation being modelled. In trace-driven simulation we want to omit details that are difficult to collect. This might be either because they are not readily available in passive traces, or because they require a lot of resources to collect, store and simulate. Unfortunately it is not always obvious which aspects of a simulation can be omitted without unduly affecting the result.

In this paper we examine the sensitivity of the simulation results (described above) to the following parameters:

  • Connection establishment and termination Sometimes it is only possible to capture parts of a connection in a trace. This is particularly true for long lived connections. This parameter compares the results with and without the overhead of connection management.
  • The effect of overlaying workloads In many communications contexts, burstiness exists at all layers of traffic abstraction. As a consequence of this, it is not always valid to add together multiple copies of the same workload to generate a higher load. We describe the effect of layering identical workloads, with and without a time offset, compared with a single higher bandwidth workload.
  • Distribution of arrivals from feeder networks A great deal of complexity is contained in the structure and performance of the network that collects data to be carried over the link under study (in these studies these are the local and the US network). Under this heading we show the result of substituting a more detailed model with several alternative simple arrival distributions.
  • MSS While it is well known that MSS has a significant affect on the performance of an individual TCP connection it is less clear how it affects the overall aggregate performance of a link that is carrying many TCP connections.
  • TCP mechanisms. The effect of slow start, fast recovery and fast retransmit on the simulation results are presented.
  • Transmission Errors. Most connections have very low error rates but there are some important exceptions, including satellite and wireless networks.

The goal of this work is to guide the selection of parameters to be included in a measurement program that will feed event driven simulation. In most cases our results indicate that simulations of the behaviour of links with composite traffic, made from many simultaneous TCP connections, are not sensitive to variation of the parameters described above. The main exception is the "TCP mechanism class". There are many possible event driven simulations and the needs many of these simulations vary. However, many simulations have elements in common with ours and, to that extent, we believe our results will be generally useful.

  1. Pearson M. and McGregor A. "A simulation study of network architectures to support HTTP Traffic on Symetric high-bandwidth*delay circuits" Proceedings of the Asia Pacific Web Conference, pp19-25, 2000.
  2. Pearson M.W., McGregor A., and Cleary J.G. "Reducing US/NZ Web Page Latencies" Networks99 pp 54 - 63, Janurary 1999.
  3. A. McGregor, M. Pearson and J. Cleary\\ "The effect of multiplexing HTTP connections over asymmetric high bandwidth-delay product circuits" SPIE Photonics East 1998: Routing in the Internet: New Concepts. pp 398-401, November 1998
  4. M. Arlitt, Y. Chen, R. Gurski, and C. Williamson, "Traffic modeling in the ATM-TN TeleSim project: Design, implementation, and performance evaluation", in Proceedings of the 1995 Summer Computer Simulation Conference, (Ottawa, Ontario), July 1995.
  5. W. Stevens, "TCP slow start, congestion avoidance, fast retransmit, and fast recovery algorithms", Tech. Rep. RFC2001, IETF, Jan. 1997.