A Methodology of Reproducible Research of Simulation Test Models and Communication Protocols Implementations
1 like678 views
Dmitry Chalyy, Yaroslavl State University, Yaroslavl
A Methodology of Reproducible Research of Simulation Test Models and Communication Protocols Implementations
- 1. Ìåòîäîëîãèÿ âîñïðîèçâîäèìûõ èññëåäîâàíèé èìèòàöèîííûõ ìîäåëåé è ðåàëèçàöèé êîììóíèêàöèîííûõ ïðîòîêëîâ Äìèòðèé Þðüåâè÷ ×àëûé ßðîñëàâñêèé ãîñóäàðñòâåííûé óíèâåðñèòåò, ßðîñëàâëü 13 íîÿáðÿ 2014 ã. ×àëûé Ä.Þ.: Âîñïðîèçâîäèìûå èññëåäîâàíèÿ êîììóíèêàöèîííûõ ïðîòîêîëîâ 1 / 8
- 2. Èñòî÷íèê: Google Ngram Viewer. Äàòà äîñòóïà: 12.11.2014. url: https://books.google.com/ngrams ×àëûé Ä.Þ.: Âîñïðîèçâîäèìûå èññëåäîâàíèÿ êîììóíèêàöèîííûõ ïðîòîêîëîâ 2 / 8
- 3. Ïðèìèòèâíûé ïîäõîä ê âîñïðîèçâîäèìîñòè Ìîæíî ëè îòêîìïèëèðîâàòü è çàïóñòèòü èñõîäíûé êîä ýêñïåðèìåíòà çà 30 ìèíóò? 1 Äîñòóïåí ëè êîä è áèáëèîòåêè îò êîòîðûõ îí çàâèñèò? 2 Êîìïèëèðóåòñÿ? Çàïóñêàåòñÿ? 1Äîñòóïåí ïî àäðåñó: http://reproducibility.cs.arizona.edu/v1/tr.pdf. Äàòà äîñòóïà: 12.11.2014. Âåðñèÿ äîêóìåíòà îò 21 ìàðòà 2014 ã. ×àëûé Ä.Þ.: Âîñïðîèçâîäèìûå èññëåäîâàíèÿ êîììóíèêàöèîííûõ ïðîòîêîëîâ 3 / 8
- 4. Ïðèìèòèâíûé ïîäõîä ê âîñïðîèçâîäèìîñòè Ìîæíî ëè îòêîìïèëèðîâàòü è çàïóñòèòü èñõîäíûé êîä ýêñïåðèìåíòà çà 30 ìèíóò? 1 Äîñòóïåí ëè êîä è áèáëèîòåêè îò êîòîðûõ îí çàâèñèò? 2 Êîìïèëèðóåòñÿ? Çàïóñêàåòñÿ? Christian Collberg, Todd Proebsting, Gina Moraila, Akash Shankaran, Zuoming Shi, Alex M. Warren Measuring Reproducibility in Computer Systems Research1 1Äîñòóïåí ïî àäðåñó: http://reproducibility.cs.arizona.edu/v1/tr.pdf. Äàòà äîñòóïà: 12.11.2014. Âåðñèÿ äîêóìåíòà îò 21 ìàðòà 2014 ã. ×àëûé Ä.Þ.: Âîñïðîèçâîäèìûå èññëåäîâàíèÿ êîììóíèêàöèîííûõ ïðîòîêîëîâ 3 / 8
- 5. Ïðèìèòèâíûé ïîäõîä ê âîñïðîèçâîäèìîñòè Ìîæíî ëè îòêîìïèëèðîâàòü è çàïóñòèòü èñõîäíûé êîä ýêñïåðèìåíòà çà 30 ìèíóò? 1 Äîñòóïåí ëè êîä è áèáëèîòåêè îò êîòîðûõ îí çàâèñèò? 2 Êîìïèëèðóåòñÿ? Çàïóñêàåòñÿ? Christian Collberg, Todd Proebsting, Gina Moraila, Akash Shankaran, Zuoming Shi, Alex M. Warren Measuring Reproducibility in Computer Systems Research1 Ìåòîäîëîãèÿ èññëåäîâàíèÿ: àíàëèç 613 ñòàòåé (ASPLOS’12, CCS’12, OOPSLA’12, OSDI’12, PLDI’12, SIGMOD’12, SOSP’11, TACO’9, TISSEC’15, TOCS’30, TODS’37, TOPLAS’34, VLDB’12). 1Äîñòóïåí ïî àäðåñó: http://reproducibility.cs.arizona.edu/v1/tr.pdf. Äàòà äîñòóïà: 12.11.2014. Âåðñèÿ äîêóìåíòà îò 21 ìàðòà 2014 ã. ×àëûé Ä.Þ.: Âîñïðîèçâîäèìûå èññëåäîâàíèÿ êîììóíèêàöèîííûõ ïðîòîêîëîâ 3 / 8
- 6. Âîñïðîèçâîäèìîñòü â íàñòîÿùåå âðåìÿ Èòîã: âñåãî 25% ðåçóëüòàòîâ êîìïèëèðóþòñÿ (èç êîòîðûõ êàæäûé ÷åòâåðòûé ïðîåêò íå çàïóñêàåòñÿ)2 2Èñòî÷íèê: http://reproducibility.cs.arizona.edu/v1/index.html. Äàòà äîñòóïà: 12.11.2014. ×àëûé Ä.Þ.: Âîñïðîèçâîäèìûå èññëåäîâàíèÿ êîììóíèêàöèîííûõ ïðîòîêîëîâ 4 / 8
- 8. ×àëûé Ä.Þ.: Âîñïðîèçâîäèìûå èññëåäîâàíèÿ êîììóíèêàöèîííûõ ïðîòîêîëîâ 5 / 8
- 9. Ïðèìåð: òðàíñïîðòíûå ïðîòîêîëû. Èñõîäíûå äàííûå 1 Îðèãèíàëüíûå ìîäåëè (ARTCP). 2 Ïàêåòû èìèòàöèîííîãî ìîäåëèðîâàíèÿ: ns-3, OMNeT++, Cisco Packet Tracer (TCP TIPS). 3 Çàõâàò ñåòåâîãî òðàôèêà íåïîñðåäñòâåííî ñ ñåòåâîãî óñòðîéñòâà: tcpdump (îðèãèíàëüíûé Trickles). ×àëûé Ä.Þ.: Âîñïðîèçâîäèìûå èññëåäîâàíèÿ êîììóíèêàöèîííûõ ïðîòîêîëîâ 6 / 8
- 10. Ïðèìåð: òðàíñïîðòíûå ïðîòîêîëû. Àíàëèç è îáðàáîòêà äàííûõ 1 Ñêðèïòîâûå ÿçûêè ïðîãðàììèðîâàíèÿ (shell, awk è ò.ä.). 2 R. 3 Âîçìîæíîñòè ñèñòåì ìîäåëèðîâàíèÿ (íàïð. ns-3 statistical framework, wireshark). ×àëûé Ä.Þ.: Âîñïðîèçâîäèìûå èññëåäîâàíèÿ êîììóíèêàöèîííûõ ïðîòîêîëîâ 7 / 8
- 11. Ïðîåêòíûé ïîäõîä 1 Êàæäîå èññëåäîâàíèå - ýòî ïðîåêò. 2 Êàê ìîæíî ìåíüøå íåñòàíòàðäíûõ ñðåäñòâ ðàçðàáîòêè. 3 Îáùåäîñòóïíûé ðåïîçèòîðèé êîäà - bitbucket, github. 4 Äîêóìåíòàöèÿ - Doxygen. 5 Àíàëèç äàííûõ - R+{Sweave, knitr}+RPubs. ×àëûé Ä.Þ.: Âîñïðîèçâîäèìûå èññëåäîâàíèÿ êîììóíèêàöèîííûõ ïðîòîêîëîâ 8 / 8