SlideShare a Scribd company logo

e-Infrastructure & Data Service

Download as PPTX, PDF
R
riround

This document describes an e-infrastructure and data service that was designed to provide modular access to different computing resources from applications. The service allows reading and writing data to multiple e-infrastructures in a straightforward way. It identifies four types of e-infrastructures - local, remote, grid, and cloud - and allows defining policies and quotas for each. The data service implements common file operations independently of the underlying technologies through storage interfaces. This provides a virtual file system view of data stored across different infrastructure implementations.

Education
1 of 23
Download to read offline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
e-Infrastructure & Data Services Features and implementations Catania, 14/03/2014Riccardo Rotondo riccardo.rotondo@ct.infn.it
Outline  Requirements  e-Infrastructure Service  Data Service  Workflow  Javadocs  Implementations  Demo 2
e-Infrastructure & Data Service  We designed the architecture to provide a Service accessible from your application.  Modular attachable resources available to the SG thanks to the straight coupled service that allows read/write data from several e-Infrastructures. 3
e-Infrastructure Service  Four different kind of e-Infrastructure identified:  Local  Remote  Grid  Cloud  Possibility of defining different Policy and Quota according e-infrastructures/users availability/needs  Default and favorite e-Infrastructures selectable  Disaster recovery and (cross) file replica. 4
e-Infrastructure Service  If you want to copy a file on a file system you plug-in an external storage to it. 5 HFS ext4 NFS FAT
e-Infrastructure Service  On a SG now you are able to plug-in an e- Infrastructure. 6 Local Storage Remote Storage Grid Storage Cloud Storage
Data Services  Business logic related to file management totally independent from the underneath technology  All the common features associated to file provided: create, copy, delete, move and sharing.  For each features, we implemented a different Storage Interface that translates the single commands according to the technology  All data are stored in different tables in order to let users organise files on a virtual file system. 7
Data Service 8 Local Storage Remote Storage Grid Storage Cloud Storage Different implementations of the same storage interface
Upload workflow 9 1. Sign in Database 5. File Upload 4. e-Infra found 7. Update 6. Upload on e-Infras 2. Upload request 3. e-Infra Discovery
Data Management on various e-Infrastructures 10 Dec ’13: release 0.1
Service Implementation  Package name  Namespace  Entity  Interface type  Finder  Ordered value  One-to-many relation  Many-to-many relation  Fake entity Riccardo RotondoTutorial on Science Gateways, Roma, service.xml
Database created (automatically by service buider) 12 mysql> show tables like 'eInfrSrv%'; +---------------------------------+ | Tables_in_lportal61 (eInfrSrv%) | +---------------------------------+ | eInfrSrv_File | | einfrsrv_files_tags | | einfrsrv_files_virtualpaths | | einfrsrv_infrastructure | | einfrsrv_properties | | einfrsrv_storageurls | | einfrsrv_tag | | einfrsrv_virtualpath | +---------------------------------+ File Directory File url on storage e-Infrastructures File tags Properties (reffered to e-Infra)
Javadocs 13  http://www.catania-science-gateways.it/javadocs/einfrsrv-portlet/index.html
Model & Service Implementation 14 Custom model implementation Custom service implementation
Main features 15  e-Infrastructure:  Define several e-Infrastructures;  Associate each e-Infra with Policy and Quota  Storage interface:  A standard interface defined to support any storage solution simply implementing the abstract method  Directory:  File tree data organisation
e-Infrastructure Service (it.infn.ct.einfrsrv.sb.service.impl.InfrastructureLocalServiceImpl) 16  Infrastructure getDefaultInfrastructure(long companyId, long userId, java.lang.String portletId, java.lang.String type)
Data Service Storage (it.infn.ct.einfrsrv.data.storage) 17  java.lang.StringgetStoragePath()  boolean isURLAvailable(java.lang.String url)  java.lang.StringputFile(java.lang.String filePath)  java.lang.StringputFile(java.lang.String source, java.lang.String destination)  boolean writeFileToStream(java.lang.String filePath, java.io.OutputStream out)  boolean removeFile(java.lang.String filePath)
Data Service File (it.infn.ct.einfrsrv.sb.service.impl.FileLocalServiceImpl) 18  boolean upload(long companyId, java.lang.String companyName, long userId, java.lang.String portletId, Infrastructure infraParam, java.lang.String fileName, java.lang.String source, java.lang.String virtualPathParam, java.lang.String[] tagsParam)  java.util.List<File> listFiles(long userId, java.lang.String virtualPathParam)  boolean writeFileToStream(long userId, long fileId, java.io.OutputStream out)  java.net.URL getURL(long userId, long fileId)
Data Service Directory (it.infn.ct.einfrsrv.sb.service.impl.VirtualPathLocalServiceImpl) 19  VirtualPath makeDir(java.lang.String parentPath, java.lang.String path, long userId)
Portlet accessing the service 20  Add to liferay-plugin-package.properties:  required-deployment-contexts=einfrsrv-portlet
Source & Binary 21  Sourceforge: http://sourceforge.net/p/ctsciencegtwys/einfsrv/sourc es/HEAD/tree/  Rel 0.1: http://sourceforge.net/p/ctsciencegtwys/einfsrv/sources/H EAD/tree/tags/release-0.1/  Binaryhttps://sourceforge.net/projects/ctsciencegtwys/files /eInfrastructureService/Release/0.1/
Demo 22  Whishing to see e-Infra & Data service in action? Request access to GARR Science Gateway (https://sgw.garr.it)  Remember the service is still under development so your feedback is welcome!
Questions ? 23

More Related Content

PPTX
Job Managment Portlet
riround
 
PPTX
Catania Science Gateway Framework
riround
 
PPTX
Gromacs on Science Gateway
riround
 
PPTX
Introduction on Science Gateway
riround
 
PPTX
Web Application Development
riround
 
PPT
Portlets 2.0 JSR286
C2B2 Consulting
 
PPTX
Science Gateways: one portal, many e-Infrastructures and related services
riround
 
PPTX
BigDataTech 2016 How to manage authorization rules on Hadoop cluster with Apa...
Krzysztof Adamski
 
Job Managment Portlet
riround
 
Catania Science Gateway Framework
riround
 
Gromacs on Science Gateway
riround
 
Introduction on Science Gateway
riround
 
Web Application Development
riround
 
Portlets 2.0 JSR286
C2B2 Consulting
 
Science Gateways: one portal, many e-Infrastructures and related services
riround
 
BigDataTech 2016 How to manage authorization rules on Hadoop cluster with Apa...
Krzysztof Adamski
 

Similar to e-Infrastructure & Data Service (20)

PDF
Microservice and Service Fabric talk
Daniel Kreuzhofer
 
PDF
Splunk app for stream
csching
 
PPTX
Optimizing windows 8 for virtual desktops - teched 2013 Jeff Stokes
Jeff Stokes
 
PDF
IT Network Asset Discovery & Inventory
ikirmer
 
PDF
Automated Out-of-Band management with Ansible and Redfish
Jose De La Rosa
 
PDF
Linux Desktop Automation
Rui Lapa
 
DOC
Resume_Appaji
Appaji K
 
PPTX
Hyper v and live migration on cisco unified computing system - virtualized on...
Louis Göhl
 
PDF
Better Network Management Through Network Programmability
Cisco Canada
 
PPTX
Cloud Computing
Adhish Pendharkar
 
DOCX
Cv
Virendra Gupta
 
PPT
Linux Based Advanced Routing with Firewall and Traffic Control
sandy_vasan
 
PDF
Matrix - One-year in, Matthew Hodgson, Matrix.org
Alan Quayle
 
PPTX
Service Fabric and Azure Service Fabric Mesh introduction
Mikkel Mørk Hegnhøj
 
PDF
DriverPack Solution Download Full ISO free
blouch112kp
 
PDF
Atlantis Word Processor 4.4.5.1 Free Download
shanbahikp01
 
PDF
Neo4j Vision and Roadmap
Neo4j
 
PDF
Adobe After Effects 2025 v25.1.0 Free Download
alihamzakpa070
 
PDF
iTop VPN Crack 6.3.3 serial Key Free 2025
blouch86kp
 
PDF
"Wie passen Serverless & Autonomous zusammen?"
Volker Linz
 
Microservice and Service Fabric talk
Daniel Kreuzhofer
 
Splunk app for stream
csching
 
Optimizing windows 8 for virtual desktops - teched 2013 Jeff Stokes
Jeff Stokes
 
IT Network Asset Discovery & Inventory
ikirmer
 
Automated Out-of-Band management with Ansible and Redfish
Jose De La Rosa
 
Linux Desktop Automation
Rui Lapa
 
Resume_Appaji
Appaji K
 
Hyper v and live migration on cisco unified computing system - virtualized on...
Louis Göhl
 
Better Network Management Through Network Programmability
Cisco Canada
 
Cloud Computing
Adhish Pendharkar
 
Cv
Virendra Gupta
 
Linux Based Advanced Routing with Firewall and Traffic Control
sandy_vasan
 
Matrix - One-year in, Matthew Hodgson, Matrix.org
Alan Quayle
 
Service Fabric and Azure Service Fabric Mesh introduction
Mikkel Mørk Hegnhøj
 
DriverPack Solution Download Full ISO free
blouch112kp
 
Atlantis Word Processor 4.4.5.1 Free Download
shanbahikp01
 
Neo4j Vision and Roadmap
Neo4j
 
Adobe After Effects 2025 v25.1.0 Free Download
alihamzakpa070
 
iTop VPN Crack 6.3.3 serial Key Free 2025
blouch86kp
 
"Wie passen Serverless & Autonomous zusammen?"
Volker Linz
 
Ad

Recently uploaded (20)

PDF
RTP_AR_KS1_Tutor's Guide_English [FOR REPRODUCTION].pdf
RobertMentino1
 
PDF
The Lost Whites of Pakistan by Jahanzaib Mughal.pdf
jahanzaibmughal6
 
PDF
Black Hat USA 2025 - Micro ICS Summit - ICS/OT Threat Landscape
Chris Sistrunk
 
PPTX
Tissue processing ( HISTOPATHOLOGICAL TECHNIQUE
mathiasmelwyn7
 
PDF
Weekly quiz Compilation Jan -July 25.pdf
Nitin Suresh
 
PPTX
UNIT III MENTAL HEALTH NURSING ASSESSMENT
Sonali Gupta
 
PDF
OBE - B.A.(HON'S) IN INTERIOR ARCHITECTURE -Ar.MOHIUDDIN.pdf
ArchitectAFMMohiuddi
 
PDF
RMMM.pdf make it easy to upload and study
sajedul2
 
PPTX
PPT- ENG7_QUARTER1_LESSON1_WEEK1. IMAGERY -DESCRIPTIONS pptx.pptx
KMDee
 
PPTX
History, Philosophy and sociology of education (1).pptx
tmdth1
 
PDF
LDMMIA Reiki Yoga Finals Review Spring Summer
LDMMia Reiki Lectures
 
PDF
GENETICS IN BIOLOGY IN SECONDARY LEVEL FORM 3
ElishamichaelSamwel
 
PDF
Updated Idioms and Phrasal Verbs in English subject
surbaraoye
 
PDF
Paper A Mock Exam 9_ Attempt review.pdf.
SameeraNaveed2
 
PPTX
school management -TNTEU- B.Ed., Semester II Unit 1.pptx
NihumathunnisaH
 
PDF
Trump Administration's workforce development strategy
Mebane Rash
 
PPTX
master seminar digital applications in india
ananyaray35
 
PDF
2.FourierTransform-ShortQuestionswithAnswers.pdf
Raji Murugan
 
PDF
Microbial disease of the cardiovascular and lymphatic systems
KeyaSharma
 
PDF
Module 4: Burden of Disease Tutorial Slides S2 2025
Jonathan Hallett
 
RTP_AR_KS1_Tutor's Guide_English [FOR REPRODUCTION].pdf
RobertMentino1
 
The Lost Whites of Pakistan by Jahanzaib Mughal.pdf
jahanzaibmughal6
 
Black Hat USA 2025 - Micro ICS Summit - ICS/OT Threat Landscape
Chris Sistrunk
 
Tissue processing ( HISTOPATHOLOGICAL TECHNIQUE
mathiasmelwyn7
 
Weekly quiz Compilation Jan -July 25.pdf
Nitin Suresh
 
UNIT III MENTAL HEALTH NURSING ASSESSMENT
Sonali Gupta
 
OBE - B.A.(HON'S) IN INTERIOR ARCHITECTURE -Ar.MOHIUDDIN.pdf
ArchitectAFMMohiuddi
 
RMMM.pdf make it easy to upload and study
sajedul2
 
PPT- ENG7_QUARTER1_LESSON1_WEEK1. IMAGERY -DESCRIPTIONS pptx.pptx
KMDee
 
History, Philosophy and sociology of education (1).pptx
tmdth1
 
LDMMIA Reiki Yoga Finals Review Spring Summer
LDMMia Reiki Lectures
 
GENETICS IN BIOLOGY IN SECONDARY LEVEL FORM 3
ElishamichaelSamwel
 
Updated Idioms and Phrasal Verbs in English subject
surbaraoye
 
Paper A Mock Exam 9_ Attempt review.pdf.
SameeraNaveed2
 
school management -TNTEU- B.Ed., Semester II Unit 1.pptx
NihumathunnisaH
 
Trump Administration's workforce development strategy
Mebane Rash
 
master seminar digital applications in india
ananyaray35
 
2.FourierTransform-ShortQuestionswithAnswers.pdf
Raji Murugan
 
Microbial disease of the cardiovascular and lymphatic systems
KeyaSharma
 
Module 4: Burden of Disease Tutorial Slides S2 2025
Jonathan Hallett
 
Ad

e-Infrastructure & Data Service

  • 1. e-Infrastructure & Data Services Features and implementations Catania, 14/03/2014Riccardo Rotondo riccardo.rotondo@ct.infn.it
  • 2. Outline  Requirements  e-Infrastructure Service  Data Service  Workflow  Javadocs  Implementations  Demo 2
  • 3. e-Infrastructure & Data Service  We designed the architecture to provide a Service accessible from your application.  Modular attachable resources available to the SG thanks to the straight coupled service that allows read/write data from several e-Infrastructures. 3
  • 4. e-Infrastructure Service  Four different kind of e-Infrastructure identified:  Local  Remote  Grid  Cloud  Possibility of defining different Policy and Quota according e-infrastructures/users availability/needs  Default and favorite e-Infrastructures selectable  Disaster recovery and (cross) file replica. 4
  • 5. e-Infrastructure Service  If you want to copy a file on a file system you plug-in an external storage to it. 5 HFS ext4 NFS FAT
  • 6. e-Infrastructure Service  On a SG now you are able to plug-in an e- Infrastructure. 6 Local Storage Remote Storage Grid Storage Cloud Storage
  • 7. Data Services  Business logic related to file management totally independent from the underneath technology  All the common features associated to file provided: create, copy, delete, move and sharing.  For each features, we implemented a different Storage Interface that translates the single commands according to the technology  All data are stored in different tables in order to let users organise files on a virtual file system. 7
  • 8. Data Service 8 Local Storage Remote Storage Grid Storage Cloud Storage Different implementations of the same storage interface
  • 9. Upload workflow 9 1. Sign in Database 5. File Upload 4. e-Infra found 7. Update 6. Upload on e-Infras 2. Upload request 3. e-Infra Discovery
  • 10. Data Management on various e-Infrastructures 10 Dec ’13: release 0.1
  • 11. Service Implementation  Package name  Namespace  Entity  Interface type  Finder  Ordered value  One-to-many relation  Many-to-many relation  Fake entity Riccardo RotondoTutorial on Science Gateways, Roma, service.xml
  • 12. Database created (automatically by service buider) 12 mysql> show tables like 'eInfrSrv%'; +---------------------------------+ | Tables_in_lportal61 (eInfrSrv%) | +---------------------------------+ | eInfrSrv_File | | einfrsrv_files_tags | | einfrsrv_files_virtualpaths | | einfrsrv_infrastructure | | einfrsrv_properties | | einfrsrv_storageurls | | einfrsrv_tag | | einfrsrv_virtualpath | +---------------------------------+ File Directory File url on storage e-Infrastructures File tags Properties (reffered to e-Infra)
  • 14. Model & Service Implementation 14 Custom model implementation Custom service implementation
  • 15. Main features 15  e-Infrastructure:  Define several e-Infrastructures;  Associate each e-Infra with Policy and Quota  Storage interface:  A standard interface defined to support any storage solution simply implementing the abstract method  Directory:  File tree data organisation
  • 17. Data Service Storage (it.infn.ct.einfrsrv.data.storage) 17  java.lang.StringgetStoragePath()  boolean isURLAvailable(java.lang.String url)  java.lang.StringputFile(java.lang.String filePath)  java.lang.StringputFile(java.lang.String source, java.lang.String destination)  boolean writeFileToStream(java.lang.String filePath, java.io.OutputStream out)  boolean removeFile(java.lang.String filePath)
  • 18. Data Service File (it.infn.ct.einfrsrv.sb.service.impl.FileLocalServiceImpl) 18  boolean upload(long companyId, java.lang.String companyName, long userId, java.lang.String portletId, Infrastructure infraParam, java.lang.String fileName, java.lang.String source, java.lang.String virtualPathParam, java.lang.String[] tagsParam)  java.util.List<File> listFiles(long userId, java.lang.String virtualPathParam)  boolean writeFileToStream(long userId, long fileId, java.io.OutputStream out)  java.net.URL getURL(long userId, long fileId)
  • 19. Data Service Directory (it.infn.ct.einfrsrv.sb.service.impl.VirtualPathLocalServiceImpl) 19  VirtualPath makeDir(java.lang.String parentPath, java.lang.String path, long userId)
  • 20. Portlet accessing the service 20  Add to liferay-plugin-package.properties:  required-deployment-contexts=einfrsrv-portlet
  • 21. Source & Binary 21  Sourceforge: http://sourceforge.net/p/ctsciencegtwys/einfsrv/sourc es/HEAD/tree/  Rel 0.1: http://sourceforge.net/p/ctsciencegtwys/einfsrv/sources/H EAD/tree/tags/release-0.1/  Binaryhttps://sourceforge.net/projects/ctsciencegtwys/files /eInfrastructureService/Release/0.1/
  • 22. Demo 22  Whishing to see e-Infra & Data service in action? Request access to GARR Science Gateway (https://sgw.garr.it)  Remember the service is still under development so your feedback is welcome!