Transcription

1 Contents 1 Introduction Requirements for Evaluation Techniques Performance Evaluation Techniques Network Testbeds / Real-World Measurements Network Simulators Analytic Models Hybrid Evaluation Methodologies Challenges in Hybrid Evaluation Synchronization and Integration of Timing Domains Interoperability of Evaluation Techniques and Tools Realistic and Contextual Integration for Hybrid Evaluation Frameworks Research Questions Contributions Thesis Organization 8 2 Fundamentals and Background An Introduction to the Performance Evaluation of Computer Networks Performance Metrics Performance Evaluation Methodologies Hybrid Performance Evaluation Methodologies A BitTorrent Model for Network Emulation The Synchronization Problem of Network Emulation Virtual Machines Implementation Aspects of System Virtualization The Xen Hypervisor 37

2 2.3 Hardware Simulation and Full-System Simulators The Simics Full-System Simulator SystemC Interim Summary 42 3 Synchronized Hybrid Evaluation Time Synchronization Execution of System Representations: Assumptions Synchronization Requirements The Synchronization Scheme The Synchronization Protocol System Representation Components Characteristics of System Representations Virtual Machines (VMs) Discrete Event-based Network Simulation Full-System Simulators Other System Representation Types Hybrid System Representations Packet Exchange between System Representations Non-Synchronized Packet Exchange Integration at the Medium Access Control (MAC) layer Types of Synchronized Hybrid Evaluation setups Synchronized Network Emulation Synchronized Hardware/Network Co-Simulation Other Types of Compositions Interim Summary 66 4 Synchronized Emulation Frameworks SliceTime: A Platform for Scalable and Accurate Network Emulation Conceptual Design Implementation of SliceTime Evaluation of SliceTime Interim Summary 80

3 4.2 Device Driver-enabled Wireless Network Emulation DDWNE Architecture Implementation of DDWNE for Evaluation Compatibility with unmodified Wireless Software Interim Summary Application Studies using SliceTime and DDWNE HTTP Use Case AODV routing daemon study A Hybrid Evaluation Platform for BitTorrent systems Related Work Link Emulation Engines Virtualization-based Emulation Engines Network Simulator-based Emulation Engines Hardware-based Network Emulation Interim Conclusion A hybrid evaluation framework for networked embedded systems Motivation Conceptual Design Synchronization Component Network Simulation Virtual Platform (VP) Implementation The SyncVP Component Virtual Network Chip Evaluation Virtual Platform Performance Influence of Synchronization Accuracy Influences of Network Traffic on the VP Bus Load Influence of Payload Processing on Round Trip Time Related Work Hardware Simulator-based Approaches 139

4 5.5.2 Approaches based 011 Co-Simulation Limitations Non-Optimal Integration of Timing Dependency on Proprietary Software Interim Summary and Conclusion Monitoring and Debugging Communication Software using Virtual Time Motivation A Distributed Analysis Framework Challenges and Solutions Architecture Prototype Implementation Scripting Environment Back-end Server Functional Tests Time Isolation Monitoring State Changes Related Work Virtual Machine-based Debugging Distributed Debugging Online System Monitoring Limitations and Potential Future Work Interim Summary and Conclusion Conclusion Key Results Conceptual Limitations Future Research and Possible Extensions Final Remarks 166 Bibliography 167

5 A Appendix 185 A.l SliceTime: Additional Timing Measurements 180 A.2 BitTorrent Client Model Implementation 187 A.2.1 Component Interaction 187 A.3 Setup and Application of DDWNE 188 A.3.1 Setting up DDWNE for ns A.3.2 Setup of the DDWNE Driver 188 A.4 List of Abbreviations 189