SlideShare a Scribd company logo
Integrating practical
     Author/Presenter: David Fergusson
            Technology Masters –
Alain Roy, Ben Clifford, Rebbeca Breu, Carlos
 Aranda, Emidio Giorgio, Tony Calanduci,
        Steve Crouch, Tilaye Alemu

           SFK Master – Ted Wen
SfK = simulation of typical
    e-Science Research

    Collaboration between scientists (your group)
    Exploring large amounts of Data to find particular patterns of interest
         Astronomy
         Particle physics
         Biomedicine
         Geophysics

           ….
    Using results of other researchers’ work
The Pillars of Wisdom

                                         background
                                         Pillar –
                                         Overrides background
                                         Rectangular
                                         Constant height
                                         Aligned with x-y axis
                                            Plaque –
                                            Overrides background
                                            Rectangular
                                            Constant height
                                            > Or < pillar height
                                            Aligned with x-y axis

                                              Word or phrase –
                                              Overrides background
                                              Rectangular
                            Wise Words        Constant height
Total of 20 Pillars                           > Or < plaque height
                                              Aligned with x-y axis
Hints




                                                            Hints are found
                                                            using OGSA-DAI

                                       Hints look like this (in relational form):

                                 x       y          form    technology
                                 12.554886          2.295809        CALCULATE
                                 764.082765         91.932643       DATA OMII

  Boundary of the Surface
 Hints, (Xi, Yi), obtained by “previous research teams” - may not be
  completely trustworthy!
 Hints tell you which technology to use and the form of the data
  (calculated on the fly or stored as files and accessed via metadata)
Real search space
characteristics
 In real life
      Noise is much larger and pervasive (ie. Top of pillars)
         Ratio of signal to noise is usually larger
      Search spaces generally larger
      Don’t normally know the complete parameters of your search
       space
         E.G. Boundaries & alignment

 So, here, we do not need statistics to analyse the
  patterns
 Do not need to run complex models
 You are being given the searching tool (may often be the
  case in real life)
tex
 t




      6
tex
 t




      7
tex
 t




      8
Bounding box

                     X2, y2


               tex
                t        Step size
  x1, y1




                                     9
tex
 t




      10
tex
 t




      11
Text can have white space – make sure you find all of it



tex
 t




                                                            12
Search space
(conceptual view, not actual)

                                          10000



                                          Computed data area


 -10000                                                        10000

          GridSAM   GT4         UNICORE    Condor      gLite

                                           Stored data area




                                -10000
Framework



Expected to
 Write program(s) / script(s)
   To   run explorations across Surface,
      E.G. interfacing tools for displaying result with
       technologies that deliver those results
   to   find
      Pillars, then plaques
 Do visualisation on Plaques to read Wisdom
  Words
 Recognise the pattern
 Making full use of capabilities
Scanner tool
 Use the Scanner tool to search for a pillar in a given area
 From the command line,
 java -jar sfkscan-XXX.jar <x1> <y1> <x2>
  <y2> <step_size>
      (XXX = technology name: glite globus condor
       unicore gridsam)
      X1y1 = bottom left, x2y2 = top right
 If a pillar is found in the given area, the word on the
  plaque will be printed on the screen.
 It can be saved in a text file to view better if the word
  wraps on the screen.
 This can be done through redirecting the output to a file.
                                                              15
Parameters
 Area = -10,000 to 10,000
 Step size range = 0.0001 -> 0.1
 Each technology has sample pillar that
  you will be given.

 30 pillars in total



                                           16
Semantic Grid Integrating Practical
 Objectives
      Use the lessons learned in the Semantic Grid Practical
      Query the metadata stored in a Globus container
 Procedure
      After you find all the words hidden in the pillars…
      Connect to issgc-client-01.polytech.unice.fr
      Use the query-all-notes command in the Globus installation
      Instructions: http://www.dia.fi.upm.es/~ocorcho/ISSGC2009/
       web/index_integrating.html
 Results
      The results are the name of the elements you were querying
       to the Metadata Query Service: 8 words
      Combining these words with the previous ones from the
       pillars you get the final solution
      http://www.dia.fi.upm.es/~ocorcho/ISSGC2009/Inte
       gratingWeb/integrating.html
Reporting colums

 Browse to
 http://dc06.nesc.ed.ac.uk:8080/sfk/
 To enter discovered pillars:
   Group   - this is your group number, (do not
    add for other groups!)
   Text - the whole text you found on the pillar
    (need to find all of it)
   x1, y1, x2, y2 - specify a bounding box that
    includes all of that text

                                                    18
Reporting your results
 Each team will get
    5 minutes during session 56
    2 minutes to change over and get started!
 Maximum number of slides
    Title
    4 others
 What you learnt and insights gained
    Results of the search
    Technologies used
    Evaluation of technologies
    Evaluation of your strategies
       Team organisation, roles and how they worked
Instructions
Technology-specific instructions for the integrating practical:

Submission: (When a pillar is found, submit it to this website)
http://dc06.nesc.ed.ac.uk:8080/sfk/
   Condor:
http://pages.cs.wisc.edu/~roy/grid_school_2009/integrating.html
2. GridSAM:
http://www.ecs.soton.ac.uk/~stc/ISSGC09/GridSAMIntegratingPractical.htm
3. gLite:
http://issgc-server-01.polytech.unice.fr/glite/issgc09/glite-integrating-practical.html
4. Globus:
http://www.ci.uchicago.edu/~benc/issgc09/integrating.html
5. UNICORE:
http://www.fz-juelich.de/jsc/unicore/ISSGC09/
6. OGSA-DAI:
http://homepages.nesc.ac.uk/~elias/issgc09/html/hints.html
7. Semantic Grid
http://www.dia.fi.upm.es/~ocorcho/ISSGC2009/IntegratingWeb/integrating.html
                                                                                          20

More Related Content

PDF
cvpr2009: class specific hough forest for object detection
PDF
Lecture 03: Machine Learning for Language Technology - Linear Classifiers
PPTX
TechEvent Machine Learning
PPT
Lec3
PDF
Streaming Random Forest Learning in Spark and StreamDM with Heitor Murilogome...
PDF
Decision Trees and Bayes Classifiers
PPT
Submodularity slides
PDF
An Importance Sampling Approach to Integrate Expert Knowledge When Learning B...
cvpr2009: class specific hough forest for object detection
Lecture 03: Machine Learning for Language Technology - Linear Classifiers
TechEvent Machine Learning
Lec3
Streaming Random Forest Learning in Spark and StreamDM with Heitor Murilogome...
Decision Trees and Bayes Classifiers
Submodularity slides
An Importance Sampling Approach to Integrate Expert Knowledge When Learning B...

Viewers also liked (6)

PDF
PDF
Session 58 - Cloud computing, virtualisation and the future
PDF
Session 58 :: Cloud computing, virtualisation and the future Speaker: Ake Edlund
PPT
Session 48 - Principles of Semantic metadata management
PPTX
Session 50 - High Performance Computing Ecosystem in Europe
PPT
Session 49 Practical Semantic Sticky Note
Session 58 - Cloud computing, virtualisation and the future
Session 58 :: Cloud computing, virtualisation and the future Speaker: Ake Edlund
Session 48 - Principles of Semantic metadata management
Session 50 - High Performance Computing Ecosystem in Europe
Session 49 Practical Semantic Sticky Note
Ad

Similar to Integrating Practical2009 (20)

PPT
Web Data Extraction Como2010
PDF
Software Testing:
 A Research Travelogue 
(2000–2014)
PPT
Easy edd phd talks 28 oct 2008
PDF
Getting Started with Keras and TensorFlow - StampedeCon AI Summit 2017
PDF
Standardizing on a single N-dimensional array API for Python
PDF
High-Performance Graph Analysis and Modeling
PDF
Using Deep Learning to do Real-Time Scoring in Practical Applications
PDF
AI and Deep Learning
PPTX
Principal Component Analysis For Novelty Detection
PPT
Claire98
PPTX
Distributed Deep Learning + others for Spark Meetup
PPT
lecture1.ppt
PDF
Deep Learning And Business Models (VNITC 2015-09-13)
PDF
Greg Hogan – To Petascale and Beyond- Apache Flink in the Clouds
PDF
Software Defined Visualization (SDVis): Get the Most Out of ParaView* with OS...
PDF
Computational decision making
PDF
Machine Learning and Go. Go!
PDF
Neural Networks: Support Vector machines
PDF
Fosdem 2013 petra selmer flexible querying of graph data
PPTX
24-TensorFlow-Clipper.pptxnjjjjnjjjjjjmm
Web Data Extraction Como2010
Software Testing:
 A Research Travelogue 
(2000–2014)
Easy edd phd talks 28 oct 2008
Getting Started with Keras and TensorFlow - StampedeCon AI Summit 2017
Standardizing on a single N-dimensional array API for Python
High-Performance Graph Analysis and Modeling
Using Deep Learning to do Real-Time Scoring in Practical Applications
AI and Deep Learning
Principal Component Analysis For Novelty Detection
Claire98
Distributed Deep Learning + others for Spark Meetup
lecture1.ppt
Deep Learning And Business Models (VNITC 2015-09-13)
Greg Hogan – To Petascale and Beyond- Apache Flink in the Clouds
Software Defined Visualization (SDVis): Get the Most Out of ParaView* with OS...
Computational decision making
Machine Learning and Go. Go!
Neural Networks: Support Vector machines
Fosdem 2013 petra selmer flexible querying of graph data
24-TensorFlow-Clipper.pptxnjjjjnjjjjjjmm
Ad

More from ISSGC Summer School (20)

PPT
Session 49 - Semantic metadata management practical
PPT
Session 46 - Principles of workflow management and execution
PPT
Session 42 - GridSAM
PPT
Session 37 - Intro to Workflows, API's and semantics
PPT
Session 43 :: Accessing data using a common interface: OGSA-DAI as an example
PDF
Session 40 : SAGA Overview and Introduction
PPT
Session 36 - Engage Results
PDF
Session 23 - Intro to EGEE-III
PPTX
Session 33 - Production Grids
PDF
Social Program
PPT
Session29 Arc
PDF
Session 24 - Distribute Data and Metadata Management with gLite
PDF
Session 23 - gLite Overview
PPTX
General Introduction to technologies that will be seen in the school
PPT
Session 3-Distributed System Principals
PDF
Session10part2 Servers Detailed
PPT
Session18 Madduri
PDF
Session6 Security Emidio
PDF
Session9part1
PDF
Session19 Globus
Session 49 - Semantic metadata management practical
Session 46 - Principles of workflow management and execution
Session 42 - GridSAM
Session 37 - Intro to Workflows, API's and semantics
Session 43 :: Accessing data using a common interface: OGSA-DAI as an example
Session 40 : SAGA Overview and Introduction
Session 36 - Engage Results
Session 23 - Intro to EGEE-III
Session 33 - Production Grids
Social Program
Session29 Arc
Session 24 - Distribute Data and Metadata Management with gLite
Session 23 - gLite Overview
General Introduction to technologies that will be seen in the school
Session 3-Distributed System Principals
Session10part2 Servers Detailed
Session18 Madduri
Session6 Security Emidio
Session9part1
Session19 Globus

Recently uploaded (20)

PPTX
Cell Structure & Organelles in detailed.
PDF
Business Ethics Teaching Materials for college
PPTX
Introduction_to_Human_Anatomy_and_Physiology_for_B.Pharm.pptx
PDF
Abdominal Access Techniques with Prof. Dr. R K Mishra
PPTX
PPH.pptx obstetrics and gynecology in nursing
PDF
VCE English Exam - Section C Student Revision Booklet
PPTX
Institutional Correction lecture only . . .
PDF
O7-L3 Supply Chain Operations - ICLT Program
PDF
The Lost Whites of Pakistan by Jahanzaib Mughal.pdf
PPTX
BOWEL ELIMINATION FACTORS AFFECTING AND TYPES
PPTX
Final Presentation General Medicine 03-08-2024.pptx
PPTX
human mycosis Human fungal infections are called human mycosis..pptx
PPTX
Week 4 Term 3 Study Techniques revisited.pptx
PDF
Pre independence Education in Inndia.pdf
PDF
Complications of Minimal Access Surgery at WLH
PDF
TR - Agricultural Crops Production NC III.pdf
PDF
Classroom Observation Tools for Teachers
PDF
Mark Klimek Lecture Notes_240423 revision books _173037.pdf
PDF
ANTIBIOTICS.pptx.pdf………………… xxxxxxxxxxxxx
PDF
Microbial disease of the cardiovascular and lymphatic systems
Cell Structure & Organelles in detailed.
Business Ethics Teaching Materials for college
Introduction_to_Human_Anatomy_and_Physiology_for_B.Pharm.pptx
Abdominal Access Techniques with Prof. Dr. R K Mishra
PPH.pptx obstetrics and gynecology in nursing
VCE English Exam - Section C Student Revision Booklet
Institutional Correction lecture only . . .
O7-L3 Supply Chain Operations - ICLT Program
The Lost Whites of Pakistan by Jahanzaib Mughal.pdf
BOWEL ELIMINATION FACTORS AFFECTING AND TYPES
Final Presentation General Medicine 03-08-2024.pptx
human mycosis Human fungal infections are called human mycosis..pptx
Week 4 Term 3 Study Techniques revisited.pptx
Pre independence Education in Inndia.pdf
Complications of Minimal Access Surgery at WLH
TR - Agricultural Crops Production NC III.pdf
Classroom Observation Tools for Teachers
Mark Klimek Lecture Notes_240423 revision books _173037.pdf
ANTIBIOTICS.pptx.pdf………………… xxxxxxxxxxxxx
Microbial disease of the cardiovascular and lymphatic systems

Integrating Practical2009

  • 1. Integrating practical Author/Presenter: David Fergusson Technology Masters – Alain Roy, Ben Clifford, Rebbeca Breu, Carlos Aranda, Emidio Giorgio, Tony Calanduci, Steve Crouch, Tilaye Alemu SFK Master – Ted Wen
  • 2. SfK = simulation of typical e-Science Research  Collaboration between scientists (your group)  Exploring large amounts of Data to find particular patterns of interest  Astronomy  Particle physics  Biomedicine  Geophysics ….  Using results of other researchers’ work
  • 3. The Pillars of Wisdom background Pillar – Overrides background Rectangular Constant height Aligned with x-y axis Plaque – Overrides background Rectangular Constant height > Or < pillar height Aligned with x-y axis Word or phrase – Overrides background Rectangular Wise Words Constant height Total of 20 Pillars > Or < plaque height Aligned with x-y axis
  • 4. Hints Hints are found using OGSA-DAI Hints look like this (in relational form): x y form technology 12.554886 2.295809 CALCULATE 764.082765 91.932643 DATA OMII Boundary of the Surface  Hints, (Xi, Yi), obtained by “previous research teams” - may not be completely trustworthy!  Hints tell you which technology to use and the form of the data (calculated on the fly or stored as files and accessed via metadata)
  • 5. Real search space characteristics  In real life  Noise is much larger and pervasive (ie. Top of pillars)  Ratio of signal to noise is usually larger  Search spaces generally larger  Don’t normally know the complete parameters of your search space  E.G. Boundaries & alignment  So, here, we do not need statistics to analyse the patterns  Do not need to run complex models  You are being given the searching tool (may often be the case in real life)
  • 6. tex t 6
  • 7. tex t 7
  • 8. tex t 8
  • 9. Bounding box X2, y2 tex t Step size x1, y1 9
  • 10. tex t 10
  • 11. tex t 11
  • 12. Text can have white space – make sure you find all of it tex t 12
  • 13. Search space (conceptual view, not actual) 10000 Computed data area -10000 10000 GridSAM GT4 UNICORE Condor gLite Stored data area -10000
  • 14. Framework Expected to  Write program(s) / script(s)  To run explorations across Surface,  E.G. interfacing tools for displaying result with technologies that deliver those results  to find  Pillars, then plaques  Do visualisation on Plaques to read Wisdom Words  Recognise the pattern  Making full use of capabilities
  • 15. Scanner tool  Use the Scanner tool to search for a pillar in a given area  From the command line,  java -jar sfkscan-XXX.jar <x1> <y1> <x2> <y2> <step_size>  (XXX = technology name: glite globus condor unicore gridsam)  X1y1 = bottom left, x2y2 = top right  If a pillar is found in the given area, the word on the plaque will be printed on the screen.  It can be saved in a text file to view better if the word wraps on the screen.  This can be done through redirecting the output to a file. 15
  • 16. Parameters  Area = -10,000 to 10,000  Step size range = 0.0001 -> 0.1  Each technology has sample pillar that you will be given.  30 pillars in total 16
  • 17. Semantic Grid Integrating Practical  Objectives  Use the lessons learned in the Semantic Grid Practical  Query the metadata stored in a Globus container  Procedure  After you find all the words hidden in the pillars…  Connect to issgc-client-01.polytech.unice.fr  Use the query-all-notes command in the Globus installation  Instructions: http://www.dia.fi.upm.es/~ocorcho/ISSGC2009/ web/index_integrating.html  Results  The results are the name of the elements you were querying to the Metadata Query Service: 8 words  Combining these words with the previous ones from the pillars you get the final solution  http://www.dia.fi.upm.es/~ocorcho/ISSGC2009/Inte gratingWeb/integrating.html
  • 18. Reporting colums  Browse to http://dc06.nesc.ed.ac.uk:8080/sfk/  To enter discovered pillars:  Group - this is your group number, (do not add for other groups!)  Text - the whole text you found on the pillar (need to find all of it)  x1, y1, x2, y2 - specify a bounding box that includes all of that text 18
  • 19. Reporting your results  Each team will get  5 minutes during session 56  2 minutes to change over and get started!  Maximum number of slides  Title  4 others  What you learnt and insights gained  Results of the search  Technologies used  Evaluation of technologies  Evaluation of your strategies  Team organisation, roles and how they worked
  • 20. Instructions Technology-specific instructions for the integrating practical: Submission: (When a pillar is found, submit it to this website) http://dc06.nesc.ed.ac.uk:8080/sfk/  Condor: http://pages.cs.wisc.edu/~roy/grid_school_2009/integrating.html 2. GridSAM: http://www.ecs.soton.ac.uk/~stc/ISSGC09/GridSAMIntegratingPractical.htm 3. gLite: http://issgc-server-01.polytech.unice.fr/glite/issgc09/glite-integrating-practical.html 4. Globus: http://www.ci.uchicago.edu/~benc/issgc09/integrating.html 5. UNICORE: http://www.fz-juelich.de/jsc/unicore/ISSGC09/ 6. OGSA-DAI: http://homepages.nesc.ac.uk/~elias/issgc09/html/hints.html 7. Semantic Grid http://www.dia.fi.upm.es/~ocorcho/ISSGC2009/IntegratingWeb/integrating.html 20