![OpenMP - Directives
• For Parallel work-sharing
• parallel - # pragma omp parallel
• for - #pragma omp [parallel] for [clauses]
• sections - #pragma omp [parallel] sections [clauses]
• single - #pragma omp single [clauses]
• For Master and Synchronization
• master
• critical
• barrier
• atomic
• flush
• Ordered
• #pragma omp directive](https://image.slidesharecdn.com/parallelcomputing-webminar-220509105738-19db2e6a/85/Parallel-Computing-Webminar-ppsx-25-320.jpg)
![omp_schedule
• Syntax
omp_schedule [= type[,size]]
• Type
• Dynamic
• Guided
• Runtime
• static
• Size
• Iterations
• Integer
• Not valid – Runtime](https://image.slidesharecdn.com/parallelcomputing-webminar-220509105738-19db2e6a/85/Parallel-Computing-Webminar-ppsx-29-320.jpg)
![omp_nested
• Syntax
omp_nested [= true | false]
• True - enabled
• False – disabled
• Default - false](https://image.slidesharecdn.com/parallelcomputing-webminar-220509105738-19db2e6a/85/Parallel-Computing-Webminar-ppsx-30-320.jpg)
![Matrix Multiplication
•Serial coding
for(int i=0;i<n;i++)
for(int k=0;k<n;k++)
for(int j=0;j<m;j++)
c[i][j]=c[i][j]+a[i][k]*b[k][j];](https://image.slidesharecdn.com/parallelcomputing-webminar-220509105738-19db2e6a/85/Parallel-Computing-Webminar-ppsx-34-320.jpg)
![Matrix Multiplication
• Parallel coding
#include<omp.h>
omp_set_num_threads(4);
#pragma omp parallel for private(i,j,k)
{
for(int i=0;i<n;i++)
for(int k=0;k<n;k++)
for(int j=0;j<m;j++)
c[i][j]=c[i][j]+a[i][k]*b[k][j];
}](https://image.slidesharecdn.com/parallelcomputing-webminar-220509105738-19db2e6a/85/Parallel-Computing-Webminar-ppsx-35-320.jpg)
![Sum of an Array
•Serial coding
sum=0;
for(int i=0;i<n;i++)
for(int j=0;j<m;j++)
sum+=a[i][j];](https://image.slidesharecdn.com/parallelcomputing-webminar-220509105738-19db2e6a/85/Parallel-Computing-Webminar-ppsx-36-320.jpg)
![Sum of an Array
• Parallel coding
#include<omp.h>
omp_set_num_threads(4);
#pragma omp parallel for private(i,j) reduction(+:sum)
{
for(int i=0;i<n;i++)
for(int j=0;j<m;j++)
sum+=a[i][j];
}](https://image.slidesharecdn.com/parallelcomputing-webminar-220509105738-19db2e6a/85/Parallel-Computing-Webminar-ppsx-37-320.jpg)
Parallel Computing--Webminar.ppsx
- 1. Parallel Computing 30.09.2020 Holy Cross College, Puttady Kerala International Webinar Dr.A.Bharathi Lakshmi Head of IT Department, VVVC, VNR
- 2. Content •What •Why •Architecture •Software and Processors •Parallel Programming •Research Work
- 3. Preliminary
- 4. Preliminary
- 5. Preliminary
- 6. Parallel Computing Serial Computing Parallel Computing
- 7. Why Parallel Computing •Save Time •Memory Usage •Concurrency
- 8. Architecture
- 9. Architecture •Flynn’s Taxonomy •Feng’s Classification •Handler Classification
- 10. Flynn’s Taxonomy
- 11. Flynn’s Taxonomy - SISD
- 12. Flynn’s Taxonomy – SIMD
- 13. Flynn’s Taxonomy - MISD
- 14. Flynn’s Taxonomy - MIMD
- 15. Memory Architecture •Shared Memory •Uniform Memory Access (UMA) •Non Uniform Memory Access (NUMA) •Distributed Memory •Hybrid Memory
- 16. Memory Architecture - UMA
- 17. Memory Architecture - NUMA
- 20. Type of Parallel Computing •Data Parallel •Task Parallel •Pipeline Parallel
- 21. OS & Processor • Multiprocessing • Multitasking • Multithreading • AMD • 4 – 32 Cores • 4 – 64 Threads • Intel • 2 – 7 Cores • Duo – multithreading • I7 – 8 Cores
- 22. Programming Languages • Apache Hadoop • Apache Spark • Apache Flink • Apache Beam • CUDA • OpenCL • OpenHMPP • OpenMP for C, C++ and Fortran (Shared Memory) • Message Passing Interface (MPI) for C, C++ and Fortran (Distributed Memory)
- 23. OpenMP •Thread Modeling •Converting Serial to Parallel program is easy •Unix pThread •Compiler directives •Runtime Library •Environmental Variables •Fork-Join Model
- 24. OpenMP – Fork-Join Model
- 25. OpenMP - Directives • For Parallel work-sharing • parallel - # pragma omp parallel • for - #pragma omp [parallel] for [clauses] • sections - #pragma omp [parallel] sections [clauses] • single - #pragma omp single [clauses] • For Master and Synchronization • master • critical • barrier • atomic • flush • Ordered • #pragma omp directive
- 26. OpenMP •omp.h – Runtime Library •Environmental Variable •omp_dynamic •omp_num_threads •omp_schedule •omp_nested
- 27. omp_dynamic •Syntax omp_dynamic = boolean value •value – true | false •True – allow users to adjust number of threads •False – users can’t adjust number of threads •Default value - false
- 28. omp_num_threads • Syntax omp_num_threads = num_list • Num_list – positive integer values • Single value • True & parallel construct without num_threads • false & parallel construct without num_threads • Multiple values • True & parallel construct without num_threads • false & parallel construct without num_threads
- 29. omp_schedule • Syntax omp_schedule [= type[,size]] • Type • Dynamic • Guided • Runtime • static • Size • Iterations • Integer • Not valid – Runtime
- 30. omp_nested • Syntax omp_nested [= true | false] • True - enabled • False – disabled • Default - false
- 31. Functions • omp_set_num_threads(int num_threads) • int omp_get_num_threads • int omp_get_max_threads • int omp_get_thread_num • int omp_get_num_procs • void omp_set_dynamic • int omp_get_dynamic • void omp_set_nested • int omp_get_nested
- 32. OpenMP - Clauses • For General attributes • if - if(expression) • num_threads – num_threads(num) • ordered - ordered • schedule • nowait - nowait • For data-sharing attributes • private – private(var) • firstprivate – firstprivate(var) • lastprivate – lastprivate(var) • shared – shared(var) • default – default(shared | none) • reduction – reduction(operator:list)
- 33. Paradigm for using OMP • Write a sequential program • Identify the portion to be parallelized •Add directive/pragmas • In addition to this call runtime library routines and modify environment variables • Parallel programming is ready. • Use OpenMP’s compiler to compile •Run the program
- 34. Matrix Multiplication •Serial coding for(int i=0;i<n;i++) for(int k=0;k<n;k++) for(int j=0;j<m;j++) c[i][j]=c[i][j]+a[i][k]*b[k][j];
- 35. Matrix Multiplication • Parallel coding #include<omp.h> omp_set_num_threads(4); #pragma omp parallel for private(i,j,k) { for(int i=0;i<n;i++) for(int k=0;k<n;k++) for(int j=0;j<m;j++) c[i][j]=c[i][j]+a[i][k]*b[k][j]; }
- 36. Sum of an Array •Serial coding sum=0; for(int i=0;i<n;i++) for(int j=0;j<m;j++) sum+=a[i][j];
- 37. Sum of an Array • Parallel coding #include<omp.h> omp_set_num_threads(4); #pragma omp parallel for private(i,j) reduction(+:sum) { for(int i=0;i<n;i++) for(int j=0;j<m;j++) sum+=a[i][j]; }
- 38. Image Reconstruction - Pseducode
- 39. Image Reconstruction – Time complexity Time complexity Time complexity Graph Speedup Graph 10 12 15 20 30 FBP 0.008455 0.007704 0.00781 0.015839 0.021083 SIRT 75.6588 76.6664 91.628 56.3881 176.8353 SART 50.5161 57.6855 56.3243 56.3881 202.067 ART 1609.1 1699.73 1889.8 918.3131 723.983 MLEM 522.894 462.973 750.215 709.861 2134.4 MAPEM 726.522 532.309 727.098 532.317 771.465 2 Core 502.087 332.341 502.65 332.347 463.192 4 Core 398.953 297.146 399.495 297.143 447.483 8 Core 198.488 145.926 199.045 145.934 259.513
- 40. Square Naïve Matrix Multiplication Time complexity Time complexity Graph Speedup Graph 0 200 400 600 800 1000 1200 1400 1000 × 1000 2000 × 2000 3000 × 3000 4000 × 4000 5000 × 5000 1 Core 2 Cores 4 Cores 8 Cores 12 Cores 16 Cores 18 cores 20 Cores 0.0000 2.0000 4.0000 6.0000 8.0000 10.0000 12.0000 14.0000 16.0000 18.0000 1000 × 1000 2000 × 2000 3000 × 3000 4000 × 4000 5000 × 5000 2 4 8 12 16 18 20 Cores 1000 x 1000 2000 x 2000 3000 x 3000 4000 x 4000 5000 x 5000 1 4.1621 54.4683 233.965 639.153 1282.2257 2 2.1539 25.9129 118.3784 316.9857 641.8125 4 1.0993 17.0027 64.2888 172.9639 329.7763 8 0.5822 8.5168 34.0960 81.1930 163.0.773 12 0.5074 5.8074 22.6845 62.6338 135.0753 16 0.5061 4.7371 19.455 57.1035 126.0156 18 0.4708 4.6368 18.6277 52.2070 120.5529 20 0.4487 0.5695 0.6275 0.5502 0.5177
- 41. Hot research •Nividia •Data mining – tremendous data •Lacking in techniques and Computational power •AI/Machine learning •Image Processing •Medical Field • Image Reconstruction