Randomised Optimisation Algorithms in DAPHNE

Introduction Backgrounds ROA DAPHNE Results Conclusion
Austrian-Slovenian HPC Meeting 2024
Grundlsee, 10–13 June 2024
ASHPC24 Session 5
Eugenia-Schwarzwald-Saal 9:45–10:00 Wednesday, 6 June 2024
Randomised
Optimisation Algorithms
in
DAPHNE
Aleš Zamuda
University of Maribor <ales.zamuda@um.si>
Acknowledgement. This work is supported by project DAPHNE (Integrated Data Analysis Pipelines for Large-Scale Data Management, HPC and Machine Learning) funded by the European Union’s Horizon 2020
research and innovation programme under grant agreement No 957407and ARIS (Slovenian Research And Innovation Agency) programme P2-0041 (Computer Systems, Methodologies, and Intelligent Services).
-50
-40
-30
-20
-10
0
10
0 50 100 150 200 250 300
Fitness, run 1
Fitness, run 2
Fitness, run 3
Fitness, run 4
Fitness, run 5
Fitness, run 6
Fitness, run 7
Fitness, run 8
Fitness, run 9
-50
-40
-30
-20
-10
0
10
0 50 100 150 200 250 300
Fitness, run 1
Fitness, run 2
Fitness, run 3
Fitness, run 4
Fitness, run 5
Fitness, run 6
Fitness, run 7
Fitness, run 8
Fitness, run 9
-50
-40
-30
-20
-10
0
10
0 50 100 150 200 250 300
Fitness, run 1
Fitness, run 2
Fitness, run 3
Fitness, run 4
Fitness, run 5
Fitness, run 6
Fitness, run 7
Fitness, run 8
Fitness, run 9
Aleš Zamuda 7@aleszamuda Randomised Optimisation Alg. in DAPHNE @ ASHPC, Grundlsee, 10-13 June 2024 1/57

ASHPC24 Session 5
Randomised
Optimisation
Algorithms
in DAPHNE
—
Introduction & Outline
—

Introduction & Outline: Aims of this Talk
1 (5 minutes) Part I: Background — Randomised
Optimisation Algorithms (ROA)
2 (3 minutes) Part II: DAPHNE
3 (3 minutes) Part III: ASHPC24: ROA in DAPHNE
4 (1 minutes) Part IV: Conclusion with Takeaways
5 (1 minute) Questions, Misc
6 (Appendix) Business, Marketing

Warmup Highlights on (Generative) AI
w/ ChatGPT+Synthesia: visiting Canaries, ASHPC
Photo/video: 1) HPC generated introduction (ASHPC23); 2) with underwater glider at
ULPGC SIANI; 3) infront SIANI; 4) with autonomous sailboat at SIANI; 5) rebooting in
March 2023 (digital green); 6) generative animation
If 2023 was about Generative AI, is 2024 on CI omnia Robotics?

Introduction: ASHPC21 — Supercomputing and HPC
— Vectorized Benchmarking Opportunities
A closer observation of execution times
for workloads processed in [2] is
provided in Fig. 1, where it is seen that
the execution time (color of the patches)
changes for different benchmark
executions.
Fig. 1: Execution time of full benchmarks
for different instances of optimization
algorithms. Each patch presents one full
benchmark execution to evaluate an
optimization algorithm.
• Therefore, it is useful to consider speeding up of
benchmarking through vectorization of the tasks that a
benchmark is comprised of.
• These include e.g.,
• parallell data cleaning part of an individual
ML tile [1] or
• synchronization between tasks when
executing parallell geospatial processing [3].
• To enable the possibilities of data cleaning (preprocessing) as
well as geospatial processing in parallell, such opportunities
ﬁrst need to be found or designed, if none yet exist for a
problem tackled.
• Therefore, this contribution will highlight some experiences
with ﬁnding and designing parallell ML pipelines for
vectorization and observe speedup gained from that.
• The speeding up focus will be on optimization
algorithms within such ML pipelines, but some more
future work possibilities will also be provided.
[2] Zamuda, A, Brest, J., Self-adaptive control parameters’ randomization frequency and propagations in differential evolution. Swarm and
Evolutionary Computation 25C, 72-99 (2015).
[3] Zamuda, A., Hernández Sosa, J. D., Success history applied to expert system for underwater glider path planning using differential
evolution. Expert Systems with Applications 119, 155-170 (2019).

ASHPC24 Session 5
Randomised
Optimisation
Algorithms
in DAPHNE
—
Background — Randomised
Optimisation Algorithms (ROA)
—

Differential Evolution (DE)
• A ﬂoating point encoding EA for global optimization over
continuous spaces,
• through generations,
the evolution process improves population of vectors,
• iteratively by combining a parent individual and
several other individuals of the same population,
using evolutionary operators.
• We choose the strategy jDE/rand/1/bin
• mutation: vi,G+1 = xr1,G + F × (xr2,G − xr3,G),
• crossover:
ui,j,G+1 =
(
vi,j,G+1 if rand(0, 1) ≤ CR or j = jrand
xi,j,G otherwise
,
• selection: xi,G+1 =
(
ui,G+1 if f(ui,G+1) < f(xi,G)
xi,G otherwise
,

Background: ROA in DAPHNE & ASHPCs
• June 2021, ASHPC211
; June 2022, ASHPC22
• extended abstract submitted (DAPHNE vision on Integrated pipelines), lightning talk page, and
printed poster presented (listing syntax where communication between nodes is abstracted)
• June 2023, ASHPC232
: 2 extended abstract submitted (ROA are sought in DAPHNE; ROA role in GenAI3
),
lightning talk page, printed poster presented, and a talk with slides
• August 2023, Euro-Par 20234
: initial demo created for ROA, conﬁrming that ROA can run in DAPHNE
(Apple M1)
• September 2023, seminars at Alicante (UA — in Ada Lovelace room, Conferencia Invitada) and Barcelona
(BSC — SORS: Severo Ochoa Research Seminar): role of ROA in NLP (incl. GenAI), ROA deployment
preparations base start for Mare Nostrum 5 (not opened yet at that time)
• January 2024, HiPEAC 2024: presentation of the initial benchmarking of ROA in DAPHNE (Apple M3)
1
A. Zamuda. “Parallelization of benchmarking using HPC: text summarization in natural language processing
(NLP), glider piloting in deep-sea missions, and search algorithms in computational intelligence (CI)”. In:
Austrian-Slovenian HPC Meeting 2021 - ASHPC21. 2021, p. 35.
2
Aleš Zamuda. “Generative AI Using HPC in Text Summarization and Energy Plants”. In: Austrian-Slovenian
HPC Meeting 2023–ASHPC23. 2023, p. 5.
3
Zamuda, “Generative AI Using HPC in Text Summarization and Energy Plants”.
4
A. Vontzalidis et al. “DAPHNE Runtime: Harnessing Parallelism for Integrated Data Analysis Pipelines”. In:
Euro-Par 2023: Parallel Processing Workshops. Ed. by Demetris Zeinalipour et al. Lecture Notes in Computer
Science, vol. 14352. Cham: Springer, 2024, pp. 242–246.

ASHPC24 Session 5
Randomised
Optimisation
Algorithms
in DAPHNE
—
DAPHNE
—

DAPHNE Partners
https://daphne-eu.eu
Project Consortium
13 partner institutions
from 7 countries
• DM, ML, HPC
• Academia & industry
• Different application
domains
14
• Technical University Berlin
University of Maribor (UM): UM FERI research team I lead (DAPHNE), SLING connection (EuroHPC Vega).
https://feri.um.si/en/research/international-and-structural-funds-projects/
integrated-data-analysis-pipelines-for-large-scale-data-management-hpc-and-machine-learning/

DAPHNE: Overview
Overview: Generic Aspect of the Project
• Deployment Challenges
• Hardware Challenges
• DM+ML+HPC share compilation
and runtime techniques /
converging cluster hardware
• End of Dennard scaling:
P = α CFV2 (power density 1)
• End of Moore’s law
• Amdahl’s law: sp = 1/s
 Increasing Specialization
#1 Data
Representations
Sparsity Exploitation
from Algorithms to HW
dense
graph
sparse
compressed
#2 Data
Placement
Local vs distributed
CPUs/
NUMA
GPUs
FPGAs/
ASICs
#3 Data
(Value) Types
FP32, FP64, INT8,
INT32, INT64, UINT8,
BF16, TF32, FlexPoint
[NVIDIA
A100]
 DAPHNE Overall Objective:
Open and extensible system infrastructure
Different
Systems/
Libraries
Dev Teams Programming Models
Resource
Managers
Cluster
Under-
utilization
Data/File
Exchange
3 lessons learnt so far
choices made, methodology

DAPHNE: Functionalities
y
Functionality Introduction: from Language Abstractions to
Distributed Vectorized Execution and Use Cases
• Federated matrices/frames + distribution primitives
• Hierarchical vectorized pipelines and scheduling
• Coordinator
(spawns distributed fused pipeline)
• #1 Prepare Inputs
(N/A, repartition, broadcasts,
slices broadcasts as necessary)
• #2 Coarse-grained Tasks
(tasks run vectorized pipeline)
• #3 Combine Outputs
(N/A, all-reduce, rbind/cbind)
Node 1
X
[1:
100M]
Node 2
X
[100M:
200M]
colmu
colsd
y
y
(X)
XTX
XTy
dc = DaphneContext()
G = dc.from_numpy(npG)
G = (G != 0)
c = components(G, 100, True).compute()
Python API DaphneLib
def components(G, maxi, verbose) {
n = nrow(G); // get the number of vertexes
maxi = 100;
c = seq(1, n); // init vertex IDs
diff = inf; // init diff to +Infinity
iter = 1;
// iterative computation of connected components
while(diff>0 & iter<=maxi) {
u = max(rowMaxs(G * t(c)), c); // neighbor prop
diff = sum(u != c); // # of changed vertexes
c = u; // update assignment
iter = iter + 1;
}
}
Domain-specific Language DaphneDSL
Multiple dispatch of functions/kernels
Example Use Cases
• DLR Earth Observation
• ESA Sentinel-1/2 datasets  4PB/year
• Training of local climate zone classifiers on So2Sat LCZ42
(15 experts, 400K instances, 10 labels each, ~55GB HDF5)
• ML pipeline: preprocessing,
ResNet-20, climate models
• IFAT Semiconductor Ion Beam Tuning
• KAI Semiconductor Material Degradation
• AVL Vehicle Development Process (ejector geometries, KPIs)
• ML-assisted simulations, data cleaning, augmentation
• Cleaning during exploratory query processing

HPC Sites & Visits
Demo at Euro-PAR 20235, also on EuroHPC Vega6 & OpenStack
ASHPC23: EuroHPC Vega tour
Ales Zamuda
@a�eszamuda
While visiting today I had the honor visiting the spectacular
MareNostrum supercomputers and their installation. From observing
afar in ‘06 as v1&v2 were deployed w/ 42&94 Tflops and v3 passing the
>Pflop, this tour ‘23 to v4 and v5 was sourcely. Thanks
#sors
�BSC_CNS
@rosabadia
Ales Zamuda ·
@a�eszamuda Sep 12
Show this thread
Today I am delighted to present a Severo Ochoa Research Seminar (SORS) at
Barcelona Supercomputing Center #BSC, titled: "EuroHPC AI in DAPHNE"
(host: Rosa Badia @rosabadia, Workflows and Distributed Computing Group
Manager, CS, BSC) bsc.es/research-and-d… #presenting @daphne_eu
4:20 PM · Sep 12, 2023 · Views
165
View post engagements
5
Post your reply Reply
Post
Ales Zamuda on X: "While visiting @BSC_CNS today I had the hono... https://twitter.com/aleszamuda/status/17016017
5
Vontzalidis et al., “DAPHNE Runtime: Harnessing Parallelism for Integrated Data Analysis Pipelines”.
6
Aleš Zamuda and et al. “Deploying DAPHNE Computational Intelligence on EuroHPC Vega for Benchmarking
Randomised Optimisation Algorithms”. In: International Conference on Broadband Communications for Next
Generation Networks and Multimedia Applications (CoBCom). 2024, (to appear).

ASHPC24 Session 5
Randomised
Optimisation
Algorithms
in DAPHNE
—
ASHPC24: ROA in DAPHNE —
Contribution, Results & Discussion
—

ASHPC24: Contribution
• Deploying Randomised Optimisation Algorithms (ROA) in
DAPHNE [1] on EuroHPC Vega [2] allows
• benchmarking and research in novel and innovative
models for Artificial Intelligece (AI) [3].
• The methods that are supported in DAPHNE allow
seamless distribution of AI memory
• that is required when an AI algorithm run requires a
large memory that can be distributed across different
HPC nodes.
• Using DAPHNE, the benchmarking can be not only run in
• the Runtime Environment [2] on an HPC that is much
larger than a regular laptop computer,
• but also gather monitoring data of the workload while the
algorithm is running,
• to obtain a benchmarking profile, allowing an informed
scientific observation of a novel algorithm under test.

ASHPC24: Results — ROA in DAPHNE Benchmarked
Testing: convergence of a ML system
ROAR: Randomised Optimisation Algorithm
-50
-40
-30
-20
-10
0
10
0 50 100 150 200 250 300
Fitness, run 1
Fitness, run 2
Fitness, run 3
Fitness, run 4
Fitness, run 5
Fitness, run 6
Fitness, run 7
Fitness, run 8
Fitness, run 9
-350
-300
-250
-200
-150
-100
-50
0 50 100 150 200 250 300
Fitness, run 1
Fitness, run 2
Fitness, run 3
Fitness, run 4
Fitness, run 5
Fitness, run 6
Fitness, run 7
Fitness, run 8
Fitness, run 9
0
1
2
3
4
5
6
7
0 50 100 150 200 250 300
Fitness, run 1
Fitness, run 2
Fitness, run 3
Fitness, run 4
Fitness, run 5
Fitness, run 6
Fitness, run 7
Fitness, run 8
Fitness, run 9

ASHPC24: Discussion — ROA in DAPHNE
As seen from the plots, the ﬁtness values are convergent, optimizing.
-50
-40
-30
-20
-10
0
10
0 50 100 150 200 250 300
Fitness, run 1
Fitness, run 2
Fitness, run 3
Fitness, run 4
Fitness, run 5
Fitness, run 6
Fitness, run 7
Fitness, run 8
Fitness, run 9
-350
-300
-250
-200
-150
-100
-50
0 50 100 150 200 250 300
Fitness, run 1
Fitness, run 2
Fitness, run 3
Fitness, run 4
Fitness, run 5
Fitness, run 6
Fitness, run 7
Fitness, run 8
Fitness, run 9
Figure 1 (left) shows an example ROA run, with convergence plot for the HappyCat
function, f1 =
P
x2, 0

− 10
2
0.125
+
P
x2, 0

/2 +
P
(x)

/10 + 0.5 on a
sample of different independent runs.
Figure 2 (right) shows a convergence plot for the function
f2 =
P
x5 + 1 + max(x, 0), 0

, also on different independent runs.

ASHPC24 Session 5
Randomised
Optimisation
Algorithms
in DAPHNE
—
IV: Conclusion
with Takeaways
—

ASHPC24: Conclusion Takeaways
• The results show an early demonstration of feasibility to compute
ROA in DAPHNE.
• This initial implementation was run within DAPHNE v0.2.
• At the moment (June 2024): v0.3 release preparations.
• A linear time increase was observed with an increasing problem
dimension size.
• The linear time increase is a good candidate for scaling and parallel
execution of this workload,
• because the implementation uses matrix operations which are scalable
in DAPHNE language over tiles of rows.
• Also, by changing the ﬁtness evaluation function of the ROA:
• the system is applicable to various AI challenges
• like standard evolutionary computation (EC) test functions, or
• other more elaborated AI scenarios (see Appendix) .

Further Research — After ASHPC24
• Further research can include:
• running on EuroHPC Vega with larger workloads7
,
• applying optimization algorithms to larger problems,
• with larger and distributed memory and storage,
• beneﬁtting from distributed processing with row-wise data
in DAPHNE (distributed kernels execution).
• Examples of these include vehicle navigation
• e.g. Underwater Glider Path Planning — UGPP) and
• production (e.g. engineering design optimization,
• ROA algorithms self-design (e.g. through whole
benchmark autoconﬁgurations), and
• more generative AI integration (such as Human
Language Technologies — HLT).
7
CoBCom 2024 Graz.

AI Challenges Shortlist HPC Initiatives EuroHPC Vega ,Deploying DAPHNE References
Thank you. Questions?
Thanks!
Acknowledgement: this work is supported by DAPHNE, funded by the European Union’s Horizon 2020
research and innovation programme under grant agreement No 957407.
10000
1x106
1x108
1x1010
1x1012
0 1 2 3 4 5 6 7 8 9
FES
Digits
f1
f2
f3
f4
f5
f6
f7
f8
f9
f10
-50
-40
-30
-20
-10
0
10
0 50 100 150 200 250 300
Fitness, run 1
Fitness, run 2
Fitness, run 3
Fitness, run 4
Fitness, run 5
Fitness, run 6
Fitness, run 7
Fitness, run 8
Fitness, run 9
0 1 2 3 4 5 6 7 8 9
Combined power from 110 MW to 975 MW, step 0.01 MW#104
0
100
200
300
400
500
600
700
Individual
output
(power
[MW]
or
unit
total
cost
[$])
Cost, TC / 3
Powerplant P1 power
Powerplant P2 power
Powerplant P3 power
150
200
250
300
350
400
450
500
550
16 32 48 64 80
Seconds
to
compute
a
workload
Number of tasks (equals 16 times the SLURM --nodes parameter)
Summarizer workload
Real examples: science and HPC

ASHPC24 Session 5
Randomised
Optimisation
Algorithms
in DAPHNE
—
Appendix Part I: Backgrounds
—

Background A:
HPC Workloads and
Cloud Computing

Conclusion
(Vega supercomputer in TOP500)
— A Multimedia Tour —

TOP500: EuroHPC Vega (tour at ASHPC23)

AI Challenges Shortlist
(Part II: First subpart)
Faced 5 types of challenges, leading to the needs to apply HPC
architectures for benchmarking state-of-the-art topics in
1 text summarization,
2 forest ecosystem modeling, simulation, and
visualization,
3 underwater robotic mission planning,
4 energy production scheduling for hydro-thermal power
plants, and
5 understanding evolutionary algorithms.

Challenges 1: Text Summarization (Language)
For NLP (Natural Language Processing),
part of ”Big Data”.
Terms across sentences are determined
using a semantic analysis using both:
• coreference resolution (using WordNet) and
• a Concept Matrix (from Freeling).
INPUT
CORPUS
NATURAL
LANGUAGE
PROCESSING
ANALYSIS
CONCEPTS
DISTRIBUTION
PER
SENTENCES
MATRIX OF
CONCEPTS
PROCESSING
CORPUS
PREPROCESSING
PHASE
OPTIMIZATION
TASKS
EXECUTION
PHASE
ASSEMBLE
TASK
DESCRIPTION
SUBMIT
TASKS
TO PARALLEL
EXECUTION
OPTIMIZER
+
TASK DATA
ROUGE
EVALUATION
The detailed new method called
CaBiSDETS is developed in the HPC
approach comprising of:
• a version of evolutionary algorithm
(Differential Evolution, DE),
• self-adaptation, binarization,
constraint adjusting, and some more
pre-computation,
• optimizing the inputs to deﬁne the
summarization optimization model.
Aleš Zamuda, Elena Lloret. Optimizing Data-Driven
Models for Summarization as Parallel Tasks. Journal
of Computational Science, 2020, vol. 42, pp. 101101.
DOI: 10.1016/j.jocs.2020.101101

Challenges 2: Forest Ecosystem Modeling,
Simulation, and Visualization (Real World / Video)
• HPC need to process spatial data and add procedural
content, generating real-world items for producing a
video of 3D space.
Videos: https://www.youtube.com/watch?list=PL7pmTW8neV7tZf2qx1wV5zbD74sUyHL3Bv=V9YJgYO_sIA

Challenges 3: Underwater Robotic Mission Planning
• Computational Fluid Dynamics (CFD) spatio-temporal model of the
ocean currents for autonomous vehicle navigation path planning.
• Constrained Differential Evolution Optimization
for Underwater Glider Path Planning
in Sub-mesoscale Eddy Sampling.
• Corridor-constrained optimization: eddy
border region sampling — new challenge
for UGPP DE.
• Feasible path area is constrained —
trajectory in corridor around the border of
an ocean eddy.
The objective of the glider here is to sample the
oceanographic variables more efﬁciently,
while keeping a bounded trajectory.
HPC: develop new methods and evaluate them.
Video: https://www.youtube.com/watch?v=4kCsXAehAmU

Challenges 4: Energy Production Scheduling for
Hydro-thermal Power Plants
A. Glotić, A. Zamuda. Short-term combined economic and emission
hydrothermal optimization by surrogate differential evolution.
Applied Energy, 1 March 2015, vol. 141, pp. 42-56.
DOI: 10.1016/j.apenergy.2014.12.020

Challenges 5: Understanding Evolutionary Algorithms
• Evolutionary algorithms benchmarking to
understand computational intelligence of
these algorithms (→ storage requirement!),
• aim: Machine Learning to design
an optimization algorithm
(learning to learn).
• Example CI Algorithm Mechanism Design:
Control Parameters Self-Adaptation (in DE).
Video: https://www.youtube.com/watch?v=R244LZpZSG0
Application stacks for real code:
inspired by previous computational
optimization competitions in
continuous settings that used
test functions for
optimization application domains:
• single-objective: CEC 2005,
2013, 2014, 2015
• constrained: CEC 2006, CEC 2007, CEC 2010
• multi-modal: CEC 2010, SWEVO 2016
• black-box (target value): BBOB 2009, COCO
2016
• noisy optimization: BBOB 2009
• large-scale: CEC 2008, CEC 2010
• dynamic: CEC 2009, CEC 2014
• real-world: CEC 2011
• computationally expensive: CEC 2013, CEC
2015
• learning-based: CEC 2015
• 100-digit (50% targets): 2019 joined CEC,
SEMCCO, GECCO
• multi-objective: CEC 2002, CEC 2007, CEC
2009, CEC 2014
• bi-objective: CEC 2008
• many objective: CEC 2018
Tuning/ranking/hyperheuristics use. → DEs as usual
winner algorithms.

Challenges 6: new DAPHNE Use Cases
Example Use Cases
• DLR Earth Observation
• ESA Sentinel-1/2 datasets  4PB/year
• Training of local climate zone classifiers on So2Sat LCZ42
(15 experts, 400K instances, 10 labels each, ~55GB HDF5)
• ML pipeline: preprocessing,
ResNet-20, climate models
• IFAT Semiconductor Ion Beam Tuning
• KAI Semiconductor Material Degradation
• AVL Vehicle Development Process (ejector geometries, KPIs)
• ML-assisted simulations, data cleaning, augmentation
• Cleaning during exploratory query processing
[So2Sat LC42: https://mediatum.ub.tum.de/1454690]
[Xiao Xiang Zhu et al: So2Sat LCZ42: A
Benchmark Dataset for the Classification of
Global Local Climate Zones. GRSM 8(3) 2020]

HPC Initiatives
(Part II: Second subpart)
Timeline (as member) of recent impactful HPC initiatives including Slovenia:
• SLING: Slovenian national supercomputing network, 2010-05-03–,
• SIHPC: Slovenian High-Performance Computing Centre, 2016-03-04–
• ImAppNIO: Improving Applicability of Nature-Inspired Optimisation
by Joining Theory and Practice, 2016-03-09–2020-10-31
• cHiPSet: High-Performance Modelling and Simulation for Big Data
Applications, 2015-04-08–2019-04-07,
• HPC RIVR: UPGRADING OF NATIONAL RESEARCH INFRASTRUCTURES,
Investment Program, 2018-03-01–2020-09-15,
• TFoB: IEEE CIS Task Force on Benchmarking, January 2020–,
• EuroCC: National Competence Centres in the framework of EuroHPC,
2020-09-01–(2022-08-31),
• DAPHNE: Integrated Data Analysis Pipelines for Large-Scale Data
Management, HPC, and Machine Learning, 2020-12-01–(2024-11-30).

Initiatives: SLING, SIHPC, HPC RIVR, EuroCC
• There is a federated and orchestrated aim
towards HPC infrastructure in Slovenia,
especially through:
• SLING: Slovenian national supercomputing network
→ has federated the initiative push towards
orchestration of HPC resources across the country.
• SIHPC: Slovenian High-Performance Computing Centre
→ has orchestrated the ﬁrst EU funds application
towards HPC Teaming in the country
(and Participation of Slovenia in PRACE 2).
• HPC RIVR: UPGRADING OF NATIONAL RESEARCH
INFRASTRUCTURES, Investment Program
→ has provided an investment in experimental HPC
infrastructure.
• EuroCC: National Competence Centres in the framework of
EuroHPC
→ has secured a National Competence Centre (EuroHPC).
Vega supercomputer online
Consortium Slovenian High-Performance Computing Centre
Aškerčeva ulica 6
SI-1000 Ljubljana
Slovenia
Ljubljana, 22. 2. 2017
prof. dr. Anwar Osseyran
PRACE Council Chair
Subject: Participation of Slovenia in PRACE 2
Dear professor Osseyran,
In March 2016 a consortium Slovenski superračunalniški center (Slovenian High-Performance
Computing Centre - SIHPC) was established with a founding act (Attachment 1) where article 6
claims that the consortium will join PRACE and that University of Ljubljana, Faculty of
mechanical engineering (ULFME) will represent the consortium in the PRACE. The legal
representative of ULFME in the consortium and therefore, also in the PRACE (i.e., the delegate
in PRACE Council with full authorization) is prof. dr. Jožef Duhovnik, as follows from
appointment of the dean of ULFME (Attachment 2).
The consortium has also agreed that it joins PRACE 2 optional programme as contributing GP.
With best regards,
assist. prof. dr. Aleš Zamuda, vice-president of SIHPC

Initiatives: ImAppNIO, cHiPSet, TFoB, DAPHNE
Aim towards software to run HPC and improve capabilities:
• ImAppNIO: Improving Applicability of Nature-Inspired Optimisation
by Joining Theory and Practice,
→ improve capabilities through benchmarking (to understand (and to
learn to learn)) CI algorithms
• cHiPSet: High-Performance Modelling and Simulation for Big Data
Applications,
→ include HPC in Modelling and Simulation (of the process to be
learned)
• TFoB: IEEE CIS Task Force on Benchmarking,
→ includes CI benchmarking opportunities, where HPC would enable
new capabilities.
• DAPHNE: Integrated Data Analysis Pipelines for Large-Scale Data
Management, HPC, and Machine Learning.
→ to deﬁne and build an open and extensible system infrastructure
for integrated data analysis pipelines, including data management and
processing, high-performance computing (HPC), and machine learning
(ML) training and scoring https://daphne-eu.github.io/

EuroHPC Vega
Deploying DAPHNE
(Part II: Third subpart)
MODA (Monitoring and Operational Data Analytics) tools for
• collecting, analyzing, and visualizing
• rich system and application data, and
• my opinion on how one can make sense of the data for
actionable insights.
• Explained through previous examples:
from a HPC User Perspective.

MODA Actionable Insights, Explained From a HPC
User Perspective, Through the Example of
Summarization
Most interesting findings of summarization on HPC example
are
• the fitness of the NLP model keeps increasing with prolonging
the dedicated HPC resources (see below) and that
• the fitness improvement correlates with ROUGE evaluation in
the benchmark, i.e. better summaries.
-0.05
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
1 10 100 1000 10000
ROUGE-1R
ROUGE-2R
ROUGE-LR
ROUGE-SU4R
Fitness (scaled)
Hence, the use of HPC
significantly
contributes to
capability of this NLP
challenge.
However, the MODA insight also provided the
useful task running times and resource
usage.

Running the Tasks on HPC: ARC Job Preparation
Parallel summarization tasks on grid through ARC.

Running the Tasks on HPC: ARC Job Submission,
Results Retrieval Merging [JoCS2020]
Through an HPC
approach and by
parallelization of tasks,
a data-driven
summarization model
optimization yields
improved benchmark
metric results (drawn
using gnuplot merge).
MODA is needed
to run again and
improve upon, to
forecast how to
set required task
running time and
resources
(predicting system
response).

Monitoring and Operational Data Analytics
• Monitor used (jobs, CPU/wall time, etc.):
Smirnova, Oxana. The Grid Monitor. Usage manual,
Tech. Rep. NORDUGRID-MANUAL-5, The NorduGrid
Collaboration, 2003.
http://www.nordugrid.org/documents/
http://www.nordugrid.org/manuals.html
http://www.nordugrid.org/documents/monitor.pdf
• Deployed at:
www.nordugrid.org/monitor/
• NorduGrid Grid Monitor
Sampled: 2021-06-28 at 17-57-08
• Nation-wide in Slovenia:
https://www.sling.si/gridmonitor/loadmon.php
http://www.nordugrid.org/monitor/index.php?
display=vo=Slovenia

MODA Example From: ARC at Jost
Example experiments from DOI: 10.1016/j.swevo.2015.10.007 (SPSRDEMMS)
– job YYULDmGOXpmnmmR0Xox1SiGmABFKDmABFKDmrtMKDmABFKDm66faPo.
Sample ARC ﬁle gridlog/diag (2–3 day Wall Times).
runtimeenvironments=APPS/ARNES/MPI−1.6−R ;
CPUUsage=99%
MaxResidentMemory=5824kB
AverageUnsharedMemory=0kB
AverageUnsharedStack=0kB
AverageSharedMemory=0kB
PageSize=4096B
MajorPageFaults=4
MinorPageFaults=1213758
Swaps=0
ForcedSwitches=36371494
WaitSwitches=170435
Inputs=45608
Outputs=477168
SocketReceived=0
SocketSent=0
Signals =0
nodename=wn003 . arnes . s i
WallTime=148332s
Processors=16
UserTime=147921.14s
KernelTime =2.54 s
AverageTotalMemory=0kB
AverageResidentMemory=0kB
LRMSStartTime=20150906104626Z
LRMSEndTime=20150908035838Z
exitcode=0

EuroCC HPC: Vega (TOP500 #106, HPCG #56 — June 2021)
• Researchers apply to EuroHPC JU calls for access.
• Regular calls opened in 2021 fall (Benchmark Development).
• https://prace-ri.eu/benchmark-and-development-access-information-for-applicants/
• 60% capacities for national share (70% OA, 20% commercial, 10% host (community, urgent
priority of national importance, maintenance)) + 35% EuroHPC JU share (approved applications)
• Has a SLURM dev partition for SSH login (SLURM partitions w/ CPUs: login[0001-0004]=128;
login[0005-0008]=64; cn[0001-384,0577-0960]=256; cn[0385-0576]=256; gn[01-60]=256).
Listing 1: Setting up at Vega — slurm dev partition access (login).
1 [ ales . zamuda@vglogin0007 ˜]$ s i n g u l a r i t y pull qmake . s i f docker : / / ak352 /qmake−opencv
2 [ ales . zamuda@vglogin0007 ˜]$ s i n g u l a r i t y run qmake . s i f bash
3 cd sum; qmake ; make clean ; make
4
5 [ ales . zamuda@vglogin0007 ˜]$ cat runme . sh
6 # ! / bin / bash
7 cd sum time mpirun
8 −
−mca btl openib warn no device params found 0
9 . / summarizer
10 −
−useBinaryDEMPI −
−i n p u t f i l e mRNA−1273−t x t
11 −
−withoutStatementMarkersInput
12 −
−printPreprocessProgress calcInverseTermFrequencyndTermWeights
13 −
−printOptimizationBestInGeneration
14 −
−summarylength 600 −
−NP 200
15 −
−GMAX 400
16 summarizer . out . $SLURM PROCID
17 2 summarizer . err . $SLURM PROCID
Text summarization/generation systems
are getting more and more useful
and accessible on deployed systems
(e.g. OpenAI’s ChatGPT, Microsoft’s Bing AI part,
NVIDIA’s (Fin)Megatron, BLOOM,
LaMDA, BERT, VALL-E, Point-E, etc.). -0.65
-0.6
-0.55
-0.5
-0.45
-0.4
-0.35
-0.3
-0.25
-0.2
-0.15
-0.1
1 10 100
Evaluation

MODA at First EuroCC HPC Vega Supercomputer
Listing 2: Runnig at Vega MODA.
1 ===================================================================== GMAX=200 =====
2 [ ales . zamuda@vglogin0002 ˜]$ srun −
−cpu−bind=cores −
−nodes=1 −
−ntasks−per−node=101
3 −
−cpus−per−task=2 −
−
mem=180G s i n g u l a r i t y run qmake . s i f bash
4 srun : job 4531374 queued and waiting for resources
5 srun : job 4531374 has been allocated resources
6 [ ”$SLURM PROCID” = 0 ] . / runme . sh
7 real 5m22.475 s
8 user 484m42.262 s
9 sys 1m38.304 s
10 ===================================================================== NODES=51 =====
−nodes=1 −
12 −
−
16 real 13m57.851 s
17 user 431m25.833 s
18 sys 0m29.272 s
19 ===================================================================== GMAX=400 =====
−nodes=1 −
21 −
−
25 real 6m14.687 s
26 user 590m45.641 s
27 sys 1m40.930 s

More Output: SLURM accounting
Listing 3: Example accounting tool at Vega: sacct.
[ ales . zamuda@vglogin0002 ˜]$ sacct
4531374. ext+ extern vega−users 202 COMPLETED 0:0
[ ales . zamuda@vglogin0002 ˜]$ sacct −j 4531374 −j 4531746 −j 4532697
−o MaxRSS , MaxVMSize , AvePages
MaxRSS MaxVMSize AvePages
−
−
−
−
−
−
−
−
−
− −
−
−
−
−
−
−
−
−
− −
−
−
−
−
−
−
−
−
−
0 217052K 0
26403828K 1264384K 22
0 217052K 0
13325268K 1264380K 0
0 217052K 0
26404356K 1264384K 30
Future MODA testings:
• testing the web interface for job analysis (as available from HPC RIVR);
• proﬁling MPI inter-node communication;
• use proﬁlers and monitoring tools available
— in the context of heterogeneous setups, like e.g.
• TAU Performance System — http://www.cs.uoregon.edu/research/tau/home.php,
• LIKWID Performance Tools — https://hpc.fau.de/research/tools/likwid/.

Deploying DAPHNE on Vega
Main documentation ﬁle:
Deploy.md

SLURM

More HPC User Perspective Nation-wide in Slovenia
More: at University of Maribor, Bologna study courses for
teaching (training) of Computer Science at cycles — click URL:
• level 1 (BSc)
• year 1: Programming I – e.g. C++ syntax
• year 2: Computer Architectures – e.g. assembly, microcode, ILP
• year 3: Parallell and Distributed Computing – e.g. OpenMP, MPI, CUDA
• level 2 (MSc)
• year 1: Cloud Computing Deployment and Management – e.g. arc, slurm,
Hadoop, containers (docker, singularity) through
virtualization
• level 3 (PhD)
• EU and other national projects research:
HPC RIVR, EuroCC, DAPHNE, ... – e.g. scaling new systems
of CI Operational Research of ... over HPC
• IEEE Computational Intelligence Task Force on Benchmarking
• Scientiﬁc Journals (e.g. SWEVO, TEVC, JoCS, ASOC, INS)
These contribute towards Sustainable Development of HPC.

Biography and References: Organizations
• Associate Professor at University of Maribor, Slovenia
• Continuous research programme funded by Slovenian Research Agency, P2-0041: Computer
Systems, Methodologies, and Intelligent Services
• EU H2020 Research and Innovation project, holder for UM part: Integrated
Data Analysis Pipelines for Large-Scale Data Management, HPC, and
Machine Learning (DAPHNE), https://cordis.europa.eu/project/id/957407
• IEEE (Institute of Electrical and Electronics Engineers) SM
• IEEE Computational Intelligence Society (CIS), senior member
• IEEE CIS ISATC (Intelligent Systems Applications Technical Committee) —
Vice Chair Conferences; IEEE CIS ECTC (Evolutionary Computation TC)
• IEEE CIS Task Force on Benchmarking, chair Website link
• IEEE CIS, Slovenia Section Chapter (CH08873), founding chair
• IEEE Slovenia Section, 2018–2021 vice chair, 2018-21
• IEEE Young Professionals Slovenia, 2016-19 chair
• ACM SIGEVO (Special Interest Group on Genetic and Evolutionary Computation); EurAI; SLAIS
• Associate Editor: Swarm and Evolutionary Computation (2016-’22), Human-centric Computing and
Information Sciences, Frontiers in robotics and AI (section Robot Learning and Evolution)
• Co-operation in Science and Techology (COST) Association Management Committee, member:
• CA COST Action CA22137: Randomised Optimisation Algorithms Research
Network (ROAR-NET), 2.10.2023–1.10.1027
• CA COST Action CA15140: Improving Applicability of Nature-Inspired Optimisation by Joining
Theory and Practice (ImAppNIO), WG3 VC
• ICT COST Action IC1406 High-Performance Modelling and Simulation for Big Data Applications
(cHiPSet);
• More: SI-HPC vice-chair; HPC-RIVR user; EuroHPC Vega user; SLAIS Honorary Tribunal; SLING CB member;

Biography and References: Top Publications
• Aleš Zamuda, Elena Lloret. Optimizing Data-Driven Models for Summarization as Parallel Tasks.
Journal of Computational Science, 2020, vol. 42, pp. 101101. DOI 10.1016/j.jocs.2020.101101.
• A. Zamuda, J. D. Hernández Sosa. Success history applied to expert system for underwater glider path
planning using differential evolution. Expert Systems with Applications, 2019, vol. 119, pp. 155-170. DOI
10.1016/j.eswa.2018.10.048
• C. Lucas, D. Hernández-Sosa, D. Greiner, A. Zamuda, R. Caldeira. An Approach to Multi-Objective Path
Planning Optimization for Underwater Gliders. Sensors, 2019, vol. 19, no. 24, pp. 5506. DOI
10.3390/s19245506.
• A. Viktorin, R. Senkerik, M. Pluhacek, T. Kadavy, A. Zamuda. Distance Based Parameter Adaptation for
Success-History based Differential Evolution. Swarm and Evolutionary Computation, 2019, vol. 50, pp.
100462. DOI 10.1016/j.swevo.2018.10.013.
• A. Zamuda, J. Brest. Self-adaptive control parameters’ randomization frequency and propagations
in differential evolution. Swarm and Evolutionary Computation, 2015, vol. 25C, pp. 72-99.
DOI 10.1016/j.swevo.2015.10.007.
• A. Zamuda, J. D. Hernández Sosa, L. Adler. Constrained Differential Evolution Optimization for
Underwater Glider Path Planning in Sub-mesoscale Eddy Sampling. Applied Soft Computing, 2016,
vol. 42, pp. 93-118. DOI 10.1016/j.asoc.2016.01.038.
• A. Zamuda, J. D. Hernández Sosa. Differential Evolution and Underwater Glider Path Planning
Applied to the Short-Term Opportunistic Sampling of Dynamic Mesoscale Ocean Structures.
Applied Soft Computing, vol. 24, November 2014, pp. 95-108. DOI 10.1016/j.asoc.2014.06.048.
• A. Zamuda, J. Brest. Vectorized Procedural Models for Animated Trees Reconstruction using
Differential Evolution. Information Sciences, vol. 278, pp. 1-21, 2014. DOI 10.1016/j.ins.2014.04.037.
• A. Zamuda, J. Brest. Environmental Framework to Visualize Emergent Artiﬁcial Forest Ecosystems.
Information Sciences, vol. 220, pp. 522-540, 2013. DOI 10.1016/j.ins.2012.07.031.
• A. Glotić, A. Zamuda. Short-term combined economic and emission hydrothermal optimization by
surrogate differential evolution. Applied Energy, 1 March 2015, vol. 141, pp. 42-56.
DOI 10.1016/j.apenergy.2014.12.020.
• H. Hamann, Y. Khaluf, J. Botev, M. Divband Soorati, E. Ferrante, O. Kosak, J.-M. Montanier, S. Mostaghim,
R. Redpath, J. Timmis, F. Veenstra, M. Wahby and A. Zamuda. Hybrid Societies: Challenges and
Perspectives in the Design of Collective Behavior in Self-organizing Systems. Frontiers in Robotics
and AI, 2016, vol. 3, no. 14. DOI 10.3389/frobt.2016.00014.
• J. Šilc, A. Zamuda. Special Issue on ”Bioinspired Optimization” (guest editors). Informatica - An
International Journal of Computing and Informatics, 2015, vol. 39, no. 2, pp. 1-122.

Biography and References: Bound Speciﬁc to HPC
PROJECTS:
• DAPHNE: Integrated Data Analysis Pipelines for Large-Scale Data Management, HPC, and Machine Learning
• ICT COST Action IC1406 High-Performance Modelling and Simulation for Big Data Applications
• SLING: Slovenian national supercomputing network
• SI-HPC: Slovenian corsortium for High-Performance Computing
• UM HPC-RIVR: Supercomputer at UM, https://www.hpc-rivr.si/
• SmartVillages: Smart digital transformation of villages in the Alpine Space
• Interreg Alpine Space, https://www.alpine-space.eu/projects/smartvillages/en/home
• Interactive multimedia digital signage (PKP, Adin DS)
EDITOR:
• Associate Editor in journals:
• Swarm and Evolutionary Computation (2016-2022),
• Human-centric Computing and Information Sciences (2020-2023),
• Frontiers in robotics and AI, section Robot Learning and Evolution (2021-2023),
• etc.
• Mathematics-MDPI, Special Issue Guest Editor: ”Innovations in High-Performance Computing”
• Mathematics-MDPI, Special Issue Guest Editor: ”Evolutionary Algorithms in Engineering Design
Optimization”
• Journal of advanced engineering and computation (member of editorial board since 2019). Viet Nam: Ton
Duc Thang University, 2017-. ISSN 2588-123X.
• Cloud Computing and Data Science (Associate Editor, since 2019). Universal Wiser Publisher Pte.Ltd.
• D. Gleich, P. Planinšič, A. Zamuda. 2018 25th International Conference on Systems, Signals and Image
Processing (IWSSIP). IEEE Xplore, Maribor, 20-22 June 2018.
• General Chair: 7-th Joint International Conferences on Swarm, Evolutionary and Memetic Computing
Conference (SEMCCO 2019) Fuzzy And Neural Computing Conference (FANCCO 2019), Maribor, Slovenia,
EU, 10-12 July 2019; co-editors: Aleš Zamuda, Swagatam Das, Ponnuthurai Nagaratnam Suganthan, Bijaya
Ketan Panigrahi.
• Organizers member: GECCO 2022, GECCO 2023

Biography and References: More Publications on HPC
• Aleš Zamuda, Elena Lloret. Optimizing Data-Driven Models for Summarization as Parallel Tasks.
Journal of Computational Science, 2020, vol. 42, pp. 101101. DOI 10.1016/j.jocs.2020.101101.
• Patrick Damme, Marius Birkenbach, Constantinos Bitsakos, Matthias Boehm, Philippe Bonnet, Florina
Ciorba, Mark Dokter, Pawel Dowgiallo, Ahmed Eleliemy, Christian Faerber, Georgios Goumas, Dirk Habich,
Niclas Hedam, Marlies Hofer, Wenjun Huang, Kevin Innerebner, Vasileios Karakostas, Roman Kern, Tomaž
Kosar, Alexander Krause, Daniel Krems, Andreas Laber, Wolfgang Lehner, Eric Mier, Marcus Paradies,
Bernhard Peischl, Gabrielle Poerwawinata, Stratos Psomadakis, Tilmann Rabl, Piotr Ratuszniak, Pedro
Silva, Nikolai Skuppin, Andreas Starzacher, Benjamin Steinwender, Ilin Tolovski, Pınar Tözün, Wojciech
Ulatowski, Yuanyuan Wang, Izajasz Wrosz, Aleš Zamuda, Ce Zhang, Xiao Xiang Zhu. DAPHNE: An Open
and Extensible System Infrastructure for Integrated Data Analysis Pipelines. 12th Conference on
Innovative Data Systems Research, CIDR 2022, Chaminade, CA, January 9-12, 2022.
• Aleš Zamuda, Vincenzo Crescimanna, Juan C. Burguillo, Joana Matos Dias, Katarzyna Wegrzyn-Wolska,
Imen Rached, Horacio González-Vélez, Roman Senkerik, Claudia Pop, Tudor Cioara, Ioan Salomie, Andrea
Bracciali. Forecasting Cryptocurrency Value by Sentiment Analysis: An HPC-Oriented Survey of the
State-of-the-Art in the Cloud Era. Kołodziej J., González-Vélez H. (eds) High-Performance Modelling and
Simulation for Big Data Applications. Lecture Notes in Computer Science, vol 11400, 2019, pp. 325-349.
DOI 10.1007/978-3-030-16272-6 12.
• Nenad Karolija, Aleš Zamuda. On cloud-supported web-based integrated development environment
for programming dataﬂow architectures. MILUTINOVIĆ, Veljko (ur.), KOTLAR, Milos (ur.). Exploring the
DataFlow supercomputing paradigm: example algorithms for selected applications, (Computer
communications and networks (Internet), ISSN 2197-8433), 2019, pp. 41-51. DOI
10.1007/978-3-030-13803-5 2.
• Simone Spolaor, Marco Gribaudo, Mauro Iacono, Tomas Kadavy, Zuzana Komı́nková Oplatková, Giancarlo
Mauri, Sabri Pllana, Roman Senkerik, Natalija Stojanovic, Esko Turunen, Adam Viktorin, Salvatore Vitabile,
Aleš Zamuda, Marco S. Nobile. Towards Human Cell Simulation. Kołodziej J., González-Vélez H. (eds)
High-Performance Modelling and Simulation for Big Data Applications. Lecture Notes in Computer
Science, vol 11400, 2019, pp. 221-249. DOI 10.1007/978-3-030-16272-6 8.
• A. Zamuda, J. D. Hernandez Sosa, L. Adler. Improving Constrained Glider Trajectories for Ocean Eddy
Border Sampling within Extended Mission Planning Time. IEEE Congress on Evolutionary Computation
(CEC) 2016, 2016, pp. 1727-1734.
• A. Zamuda. Function evaluations upto 1e+12 and large population sizes assessed in distance-based
success history differential evolution for 100-digit challenge and numerical optimization scenarios
(DISHchain 1e+12): a competition entry for ”100-digit challenge, and four other numerical optimization
competitions” at the genetic and evolutionary computation conference (GECCO) 2019. Proceedings of the
Genetic and Evolutionary Computation Conference Companion (GECCO 2019), 2019, pp. 11-12.
• ... several more experiments for papers run using HPCs.
• ... also, pedagogic materials in Slovenian and English — see Conclusion .

Promo materials: Calls for Papers, Websites
CS FERI WWW
CIS TFoB
CFPs WWW LI Twitter

Randomised Optimisation Algorithms in DAPHNE

More Related Content

Similar to Randomised Optimisation Algorithms in DAPHNE (20)

More from University of Maribor (18)

Recently uploaded (20)

Randomised Optimisation Algorithms in DAPHNE