Network simulator survey
Download as PPT, PDF4 likes1,540 views
The document presents a qualitative comparison of network simulation tools, specifically OPNET Modeler and NS-2, emphasizing their differences in design, performance, and suitability for various applications. It discusses the importance of simulator details, the outcomes of real-world testbed comparisons, and highlights the trade-offs involved in achieving accurate simulations. The analysis concludes that each simulator has unique strengths and weaknesses, which influence their usage depending on the specific context of the research.
Network simulator survey
- 1. Qualitative Comparison of Network Simulation Tools Author: Björn Schilling Supervisor: Jörg Hähner Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 2. Contents Introduction 2/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 3. Contents Introduction The Simulators: OPNET Modeler & NS-2 2/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 4. Contents Introduction The Simulators: OPNET Modeler & NS-2 Comparisons NS-2 vs. OPNET Modeler 2/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 5. Contents Introduction The Simulators: OPNET Modeler & NS-2 Comparisons NS-2 vs. OPNET Modeler Simulators vs. RealWorld testbed 2/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 6. Contents Introduction The Simulators: OPNET Modeler & NS-2 Comparisons NS-2 vs. OPNET Modeler Simulators vs. RealWorld testbed Results & Discussion 2/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 7. Contents Introduction The Simulators: OPNET Modeler & NS-2 Comparisons NS-2 vs. OPNET Modeler Simulators vs. RealWorld testbed Results & Discussion Trade-off with simulation details 2/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 8. Contents Introduction The Simulators: OPNET Modeler & NS-2 Comparisons NS-2 vs. OPNET Modeler Simulators vs. RealWorld testbed Results & Discussion Trade-off with simulation details Summary & Conclusions 2/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 9. About Simulation Experiments can be expensive, impossible, complex 3/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 10. About Simulation Experiments can be expensive, impossible, complex Additionally, most measurements are not repeatable 3/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 11. About Simulation Experiments can be expensive, impossible, complex Additionally, most measurements are not repeatable We need simulators 3/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 12. About Simulation Experiments can be expensive, impossible, complex Additionally, most measurements are not repeatable We need simulators Simulators model the real world Ease understanding of it Surge its behavior Research its reactions on particular events 3/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 13. About Simulation Experiments can be expensive, impossible, complex Additionally, most measurements are not repeatable We need simulators Simulators model the real world Ease understanding of it Surge its behavior Research its reactions on particular events „ as real as possible“ 3/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 14. About Simulation Experiments can be expensive, impossible, complex Additionally, most measurements are not repeatable We need simulators Simulators model the real world Ease understanding of it Surge its behavior Research its reactions on particular events „ as real as possible“ Details play an important role 3/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 15. Level of detail in simulation Correct level of detail is one of the most challenging problems 4/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 16. Level of detail in simulation Correct level of detail is one of the most challenging problems Pro: More details -> more realism 4/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 17. Level of detail in simulation Correct level of detail is one of the most challenging problems Pro: More details -> more realism Too less details can lead to wrong or deluding results 4/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 18. Level of detail in simulation Correct level of detail is one of the most challenging problems Pro: More details -> more realism Too less details can lead to wrong or deluding results Contra: Time-consuming implementation & simulation 4/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 19. Level of detail in simulation Correct level of detail is one of the most challenging problems Pro: More details -> more realism Too less details can lead to wrong or deluding results Contra: Time-consuming implementation & simulation Maintenance 4/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 20. Level of detail in simulation Correct level of detail is one of the most challenging problems Pro: More details -> more realism Too less details can lead to wrong or deluding results Contra: Time-consuming implementation & simulation Maintenance In wireless networks two main points are important: Algorithms & protocols have to be implemented detailed 4/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 21. Level of detail in simulation Correct level of detail is one of the most challenging problems Pro: More details -> more realism Too less details can lead to wrong or deluding results Contra: Time-consuming implementation & simulation Maintenance In wireless networks two main points are important: Algorithms & protocols have to be implemented detailed Simulation environment must be realistic 4/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 22. Comparisons Why are they important? Significant differences between simulators 5/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 23. Comparisons Why are they important? Significant differences between simulators Only way to see which simulator is more accurate is to make comparisons 5/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 24. Comparisons Why are they important? Significant differences between simulators Only way to see which simulator is more accurate is to make comparisons Important for users Results can influence the choice of the simulator 5/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 25. Comparisons Why are they important? Significant differences between simulators Only way to see which simulator is more accurate is to make comparisons Important for users Results can influence the choice of the simulator Important for developers Results help to improve the simulator 5/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 26. Contents Introduction The Simulators: OPNET Modeler & NS-2 Comparisons NS-2 vs. OPNET Modeler Simulators vs. RealWorld testbed Results & Discussion Trade-off with simulation details Summary & Conclusions Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 27. OPNET Modeler Core is a finite state machine Analytical model is responsible for speeding up the simulation Licences needed -> no open source product GUI Lots of documentation Many pre-implemented algorithms and protocols 6/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 28. OPNET Modeler Core is a finite state machine Analytical model is responsible for speeding up the simulation Licences needed -> no open source product GUI Lots of documentation Many pre-implemented algorithms and protocols Primal intention was network simulation for companies 6/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 29. OPNET Modeler Core is a finite state machine Analytical model is responsible for speeding up the simulation Licences needed -> no open source product GUI Lots of documentation Many pre-implemented algorithms and protocols Primal intention was network simulation for companies But comes along with some developer tools 6/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 30. NS-2 Second version able to deal with wireless networks Combination of C++ & OTcl Compromise between performance and user-friendly design Event-based simulator Open-source Less documentation Help can be found in one of the numerous user-groups 7/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 31. NS-2 Second version able to deal with wireless networks Combination of C++ & OTcl Compromise between performance and user-friendly design Event-based simulator Open-source Less documentation Help can be found in one of the numerous user-groups Mainly used among developers 7/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 32. Contents Introduction The Simulators: OPNET Modeler & NS-2 Comparisons NS-2 vs. OPNET Modeler Simulators vs. RealWorld testbed Results & Discussion Trade-off with simulation details Summary & Conclusions Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 33. NS-2 vs. OPNET Modeler Simple algorithm needed: Flooding implemented in NS-2, OPNET, GloMoSim 8/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 34. NS-2 vs. OPNET Modeler Simple algorithm needed: Flooding implemented in NS-2, OPNET, GloMoSim Setup 50 nodes, 10 of them initiating 100 broadcasts 1km*1km area 300 seconds runtime Random Waypoint mobility model (controllable) 802.11 MAC layer protocol Freespace propagation 8/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 35. NS-2 vs. OPNET Modeler Effective transmission range Maximum difference of factor 2 Complete different behavior of simulator GloMoSim 9/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 36. NS-2 vs. OPNET Modeler Overhead, i.e. produced duplicates Again different behavior Difference of factor 2,5 10/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 37. Contents Introduction The Simulators: OPNET Modeler & NS-2 Comparisons NS-2 vs. OPNET Modeler Simulators vs. RealWorld testbed Results & Discussion Trade-off with simulation details Summary & Conclusions Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 38. Simulators vs. RealWorld testbed Which simulator comes closest to reality? Simple (wired) testbed: Possibility to create background traffic All the traffic has to pass a 10Mbit link 11/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 39. Simulators vs. RealWorld testbed Which simulator comes closest to reality? Simple (wired) testbed: Possibility to create background traffic All the traffic has to pass a 10Mbit link 2. Two different easy applications make up simulation: CBR & FTP CBR for simplicity, FTP for popularity & common usage 11/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 40. Simulators vs. RealWorld testbed CBR, 5MBit/s, Background = 6MBit/s Network testbed inaccurate NS-2 more constant than OPNET Sudden drop of NS-2 curve after 60 seconds 12/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 41. Simulators vs. RealWorld testbed FTP, 10MB File, no background traffic 2 simulations (normal / after simulator fine-tuning) First results barely accurate Better results after fine-tuning, esp. OPNET Modeler 13/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 42. Contents Introduction The Simulators: OPNET Modeler & NS-2 Comparisons NS-2 vs. OPNET Modeler Simulators vs. RealWorld testbed Results & Discussion Trade-off with simulation details Summary & Conclusions Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 43. Results & Discussion Reasons for the differences 14/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 44. Results & Discussion Reasons for the differences Main factor: Level of detail of physical and MAC layer (1 and 2) These layers are implemented in the simulator Other layers are implemented by the developer/user 14/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 45. Results & Discussion Reasons for the differences Main factor: Level of detail of physical and MAC layer (1 and 2) These layers are implemented in the simulator Other layers are implemented by the developer/user Level of detail in simulation is decisive for the results 14/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 46. Results & Discussion Reasons for the differences Main factor: Level of detail of physical and MAC layer (1 and 2) These layers are implemented in the simulator Other layers are implemented by the developer/user Level of detail in simulation is decisive for the results Identical implementation of algorithms in both simulators is not possible 14/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 47. Results & Discussion Reasons for the differences Main factor: Level of detail of physical and MAC layer (1 and 2) These layers are implemented in the simulator Other layers are implemented by the developer/user Level of detail in simulation is decisive for the results Identical implementation of algorithms in both simulators is not possible Bugs & failures can occur Regular software updates, esp. NS-2 14/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 48. Contents Introduction The Simulators: OPNET Modeler & NS-2 Comparisons NS-2 vs. OPNET Modeler Simulators vs. RealWorld testbed Results & Discussion Trade-off with simulation details Summary & Conclusions Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 49. Trade-off with simulation details Often integration is difficult & can have far reaching consequences: Debugging Later change 15/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 50. Trade-off with simulation details Often integration is difficult & can have far reaching consequences: Debugging Later change Which details do make sense, which do not? Compromise between performance & correctness must be found! 15/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 51. Trade-off with simulation details Often integration is difficult & can have far reaching consequences: Debugging Later change Which details do make sense, which do not? Compromise between performance & correctness must be found! Typically in todays simulators: Protocols rather detailed Developers try to integrate as few details as possible 15/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 52. Example One scenario simulated twice… Without idle energy consumption With idle energy consumption … with 4 different ad-hoc routing protocols AODV DSR DSDV TORA 16/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 53. Example One scenario simulated twice… Without idle energy consumption With idle energy consumption … with 4 different ad-hoc routing protocols AODV DSR DSDV TORA Deluding results in the first scenario 16/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 54. Example One scenario simulated twice… Without idle energy consumption With idle energy consumption … with 4 different ad-hoc routing protocols AODV DSR DSDV TORA Deluding results in the first scenario Idle energy consumption turns out to be an important and necessary detail! 16/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 55. Contents Introduction The Simulators: OPNET Modeler & NS-2 Comparisons NS-2 vs. OPNET Modeler Simulators vs. RealWorld testbed Results & Discussion Trade-off with simulation details Summary & Conclusions Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 56. Summary & Conclusions There are differences between simulators and their simulation results 17/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 57. Summary & Conclusions There are differences between simulators and their simulation results The simulator design influences the simulation 17/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 58. Summary & Conclusions There are differences between simulators and their simulation results The simulator design influences the simulation Too few comparisons have been made to make statements about quality and accuracy of the simulators 17/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 59. Summary & Conclusions There are differences between simulators and their simulation results The simulator design influences the simulation Too few comparisons have been made to make statements about quality and accuracy of the simulators Each of the simulators showed strengths and enervations Usage of a specific simulator depending on application area 17/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 60. Summary & Conclusions Behavior of simulation can depend on small details 18/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 61. Summary & Conclusions Behavior of simulation can depend on small details Necessity of a detail is not known before simulation is done. Solutions for this problem? Use experiences made before for future simulations Share knowledge: User-groups 18/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart
- 62. Thank you for your attention! Are there any questions? 19/19 Qualitative Comparison of Network Simulation Tools University of Stuttgart