Synthesizing Program Execution Time Discrepancies in Julia Used for Scientific Software

Synthesizing Program Execution
Time Discrepancies in Julia
Used for Scientific Software
Effat Farhana Nasif Imtiaz Akond Rahman

Scientific Software in Julia
• Julia: Designed for scientific and numeric computing
o Scripting : syntax as Python
o Compiled : speed as C

Program Execution Time Discrepancies in Julia

Motivation
“Discrepancies in program execution time is a risk for scientific
software development, which should be mitigated by stakeholders”

Methodology
Filtering Qualitative
rating
Categories
Causes
SOTorrent Dataset
(Baltes et al.)

Category: Parallel Programming

Category: Filesystem Operation

Category: Scope Concern
Global
Local

Category Frequency
Category Q AQ VQ
Data Structure Concern (103) 0.36 1.68 556.38
Mathematics/Statistics/Machine Learning (57) 0.20 1.48 384.43
Language Transfer (42) 0.15 1.86 1678.12
Function/Package/Compilation (36) 0.13 1.39 581.50
Parallel Programming (34) 0.12 1.41 580.09
Data Type Concern (21) 0.07 1.52 239.00
Memory Allocation (16) 0.06 1.69 237.56
File Operation (12) 0.04 1.25 454.67
Scope Concern (7) 0.02 1.57 288.71

Cause: Inefficient Program Construct

Cause: Knowledge Gap
Memory Management Julia Knowledge Parallel Programming

Cause: Knowledge Gap
Macro Data Structure Foundational CS

Vectorization
Loop

Cause: Vectorization

Summary
Scientific Software
2
Cause#10: Data Structure Knowledge Gap
Category#9: Scope Concern
Global
Local
arahman@tntech.edu
akondrahman.github.io
@akondrahman

https://phys.org/news/2019-08-astronomers-earth-fingerprint-habitable-planets.html
https://www.icr.ac.uk/our-research/centres-and-collaborations/strategic-collaborations/cancer-research-uk-centre
https://www.wired.com/2017/01/self-driving-cars-approach-auto-industry-races-rebuild/
http://hardpict.pw/darwin-op2.html
https://stackoverflow.com/questions/36694459/optimising-a-julia-one-liner-to-make-it-as-fast-as-python
https://www.illinoispolicy.org/daily-comics/rauner-expectation-vs-reality/
https://miro.medium.com/max/3000/1*Dyu63sMUVL-gYEZISOE2BQ.jpeg
https://storage.googleapis.com/ehimages/2017/8/26/img_5edaf03d322353fae687ea03350ce096_1503760338828_original.jpg
https://stackoverflow.com/questions/47491713/julia-best-way-to-sample-from-successively-shrinking-range
https://storage.googleapis.com/groundai-web-prod/media%2Fusers%2Fuser_272673%2Fproject_252412%2Fimages%2Fx3.png
https://stackoverflow.com/questions/36694459/optimising-a-julia-one-liner-to-make-it-as-fast-as-python
https://www.codementor.io/innat_2k14/preferable-tools-for-machine-learning-python-matlab-r-jfozzpphz
https://stackoverflow.com/questions/41639237/best-way-to-eval-in-a-given-scope-context-which-is-in-the-form-of-a-dict
http://blog.rogerluo.me/images/julia-generated-compile-diagram.png
https://computing.llnl.gov/tutorials/parallel_comp/images/parallelProblem2.gif
https://stackoverflow.com/questions/46941164/julia-parallel-code-slower-than-sequential-code-are-there-alternatives-to-remo
https://stackoverflow.com/questions/38399478/huge-memory-allocation-running-a-julia-function
https://miro.medium.com/max/4000/1*nkLi2YLBO3xys_dehSXoQw.png
https://stackoverflow.com/questions/26810171/is-there-an-equivalent-or-close-to-numpy-loadtxt-for-julia
https://www.cs.cmu.edu/link/setting-standards
https://media.geeksforgeeks.org/wp-content/uploads/directory.jpg
https://i2.wp.com/blog.codeanalogies.com/wp-content/uploads/2017/11/474b7-1ywpubaj-_gmws4jedvbufa.png?w=730&ssl=1
https://stackoverflow.com/questions/29877563/performance-of-for-loops-in-julia
https://www.ics.com/sites/default/files/styles/blog_detail/public/images/anti_pattern.jpg?itok=R12fhxzm
https://stackoverflow.com/questions/34980556/memory-computation-efficient-creation-of-array-with-repeated-elements
https://stackoverflow.com/questions/41203425/why-is-julia-allocating-so-much-memory
https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcS_jXTu1cDHU6U9vY1OrNeDnsneCfhQz_CflaOFfDF75WwwsG8l
https://www.wikihow.com/images/thumb/9/97/Check-Memory-Usage-Step-3-Version-3.jpg/aid4366551-v4-728px-Check-Memory-Usage-Step-3-Version-3.jpg
https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcSUBRX72hofbNLaGmIrXX77wT0NEEy3CoXSK4G2qHAJcdyq1uRPiA
https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcSBUcLoGPKruhzguEeWnJagLNdIfJ7AHJKChMBDeQ1dAwlza_HJVg
https://stackoverflow.com/questions/42043590/why-is-this-very-simple-vectorized-code-orders-of-magnitude-slower-than-numpy
https://miro.medium.com/max/1648/1*Q_UZeToStz8YY2oQGiUPqw.png
https://stackoverflow.com/questions/50975707/parallel-computing-in-julia-and-mis-allocating-of-cores
References

https://towardsdatascience.com/python-vectorization-5b882eeef658
https://stackoverflow.com/questions/20613817/julia-julia-lang-performance-compared-to-fortran-and-python
https://impsciuw.org/wp-content/uploads/2018/07/know-do-gap-1.jpg
https://blog.brakmic.com/wp-content/uploads/2015/11/julia_intro_own_macros2.png
https://stackoverflow.com/questions/25009072/how-to-write-good-julia-code-when-dealing-with-multiple-types-and-arrays-mult
https://www.google.com/imgres?imgurl=https%3A%2F%2Fi.ytimg.com%2Fvi%2FbpHWYdx74jo%2Fmaxresdefault.jpg&imgrefurl=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D
bpHWYdx74jo&docid=flCJB17HxKtsOM&tbnid=Z2_24-z15v-ahM%3A&vet=10ahUKEwjxwN-
gzOrkAhWtnuAKHZUoBfQQMwhVKAUwBQ..i&w=1280&h=720&bih=793&biw=1600&q=Array%20or%20link%20list&ved=0ahUKEwjxwN-
gzOrkAhWtnuAKHZUoBfQQMwhVKAUwBQ&iact=mrc&uact=8
https://fvmstatic.s3.amazonaws.com/maps/m/US-OH-EPS-02-6001.pnghttps://softwareengineeringdaily.com/wp-content/uploads/2016/11/julia-language-inside-the-corporation-21-638.jpg
https://stackoverflow.com/questions/27745003/performance-of-the-first-vs-second-call-of-a-function
https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQvWf6n2-sraeWSiHEJkctMtfpXN5M5f9PEiqWnp75WHKDeQ1c-
https://softwareengineeringdaily.com/wp-content/uploads/2016/11/julia-language-inside-the-corporation-21-638.jpg
https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQhtmdIU4fuy9B4UP44PhYZ9z0hcVkVAT2OIzBfJj5sHM4qk7Y8OA
https://stackoverflow.com/questions/31321810/gini-coefficient-in-julia-efficient-and-accurate-code
https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQkBvlPIV2ocl5akiLksJn0yhE1Nnes3VYCVPT47qVshEzSG2Qoyg
https://www.iconfinder.com/icons/3694677/accountancy_accounting_online_accounting_software_financial_services_online_banking_icon
https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcRQAEzS4Ivk-f5xPU29nCPIituE5IOVmJ4RfjeIHSIyM9biFSpz
https://www.channelfutures.com/desktop/the-dos-and-donts-of-using-microsoft-teams
https://stackoverflow.com/questions/35799521/julia-efficient-memory-allocation
https://stackoverflow.com/company/logos
https://icon-library.net/icon/data-mining-icon-1.html
http://www.softicons.com/system-
icons/refresh-cl-icons-by-tpdk/java-icon
https://www.amazon.com/Introduction-Algorithms-Eastern-Economy-Thomas/dp/8120340078/ref=asc_df_8120340078/?tag=hyprod-20&linkCode=df0&hvadid=312176709
100&hvpos=1o1&hvnetw=g&hvrand=16435165872965472153&hvpone=&hvptwo=&hvqmt=&hvdev=c&hvdvcmdl=&hvlocint=&hvlocphy=1025954&hvtargid=pla-385636858931&psc=1&tag=&ref=&a
dgrpid=60258872537&hvpone=&hvptwo=&hvadid=312176709100&hvpos=1o1&hvnetw=g&hvrand=16435165872965472153&hvqmt=&hvdev=c&hvdvcmdl=&hvlocint=&hvlocphy=1025954&hvtargid=pla-385636858931
https://www.youtube.com/watch?v=q7sgzDH1cR8
http://www.zverovich.net/2016/05/13/giving-up-on-julia.html

Synthesizing Program Execution Time Discrepancies in Julia Used for Scientific Software

More Related Content

Recently uploaded (20)

Featured (20)

Synthesizing Program Execution Time Discrepancies in Julia Used for Scientific Software

Editor's Notes