Session11 Ucc Intro
0 likes358 views
The document discusses the UNICORE Command Line Client (UCC) which allows users to submit jobs to UNICORE from the command line. It provides an overview of basic UCC commands for connecting, submitting jobs, managing files and directories, and more advanced features like batch processing and scripting with Groovy. The presentation is intended to introduce users to using UCC to execute single jobs and prepare them for submitting workflow jobs using the Eclipse client in a subsequent session.
![UCC job format: file imports and export
Example: Script job that produces additional file output.txt:
{
ApplicationName : Bash shell ,
Environment : [ " SOURCE = input . sh " ] ,
Imports : [ { File : " / home / rbreu / myscript . sh " ,
To : input . sh } ] ,
# If script produces output ( optional ):
Exports : [ { File : output . txt ,
To : output . txt } ]
}
File import from local machine to job’s working directory
File export from job’s working directory to local machine
07.07.2009 Slide 9](https://image.slidesharecdn.com/session11uccintro-090709051139-phpapp01/85/Session11-Ucc-Intro-9-320.jpg)
Session11 Ucc Intro
- 1. Mitglied der Helmholtz-Gemeinschaft The UNICORE Command Line Client 07.07.2009 Rebecca Breu
- 2. UNICORE Clients Several clients for different purposes: Rich Client: Eclipse-based graphical client for complex tasks UCC: Command line client GPE Application Client: Easy-to-use graphical client for simple tasks Portal Clients: Web based client to use with a browser 07.07.2009 Slide 2
- 3. UNICORE Clients Clients are . . . intuitive Java based → platform independent easy to install Prerequisites for users: Sun Java 1.5 A user certificate Outline This session: Single jobs via ucc Next session: Workflow jobs via Eclipse Client 07.07.2009 Slide 3
- 4. Basic Client Configuration Create a password-protected keystore that holds all keys needed Add keys to keystore: User’s private and public key Public keys of trusted CAs (must trust signer of involved gateway keys!) Add URI of global registry 07.07.2009 Slide 4
- 5. Job excecution and file transfer For each job, an own directory is created on the target system (uspace) Job executes within uspace directory File import: File tranfer from somewhere (e.g. user’s local computer) into uspace File export: File tranfer from uspace to somewhere (e.g. user’s local computer) Every input file a job needs has to be imported Every output file a user wants to see has to be exported Applications (GridBeans) import and export some files autmatically 07.07.2009 Slide 5
- 6. Basic usage of UCC Connect to UNICORE site: > ucc connect You can access 2 target system ( s ). For every ucc command, you can use -v for verbose output: > ucc connect -v Getting help: > ucc help Help for each ucc command with -h: > ucc connect -h 07.07.2009 Slide 6
- 7. Synchronous Job Submission Use run command to submit a job: > ucc run ucc -1.1/ samples / date . u RUNNING SUCCESSFUL exit code : 0 / tmp / c61e1d7c -92 d4 -4 be0 -87 f8 -6 a7dddc4c728 . stdout / tmp / c61e1d7c -92 d4 -4 be0 -87 f8 -6 a7dddc4c728 . stderr / tmp / c61e1d7c -92 d4 -4 be0 -87 f8 -6 a7dddc4c728 . properties UCC waits until job is finished Output files are fetched automatically Use -a option for asynchronous job submission → UCC returns prompt after submit 07.07.2009 Slide 7
- 8. UCC job format: simple example UCC jobs are given in JSON format. Simple example: All-time favourite date application: # Simple Date job { ApplicationName : Date , Version : 1.0 # Optional } Date takes no input and produces stdout and stderr. 07.07.2009 Slide 8
- 9. UCC job format: file imports and export Example: Script job that produces additional file output.txt: { ApplicationName : Bash shell , Environment : [ " SOURCE = input . sh " ] , Imports : [ { File : " / home / rbreu / myscript . sh " , To : input . sh } ] , # If script produces output ( optional ): Exports : [ { File : output . txt , To : output . txt } ] } File import from local machine to job’s working directory File export from job’s working directory to local machine 07.07.2009 Slide 9
- 10. The Bash Application in the IDB Recall: This is how the Bash Application has been defined on server side: < idb:I DB Ap pl ic ation > < id b : Ap p l ic a t ionName > Bash shell </ idb:ApplicationName > < i d b : A p p l i c a t ion Ve rs ion > 3.1.16 </ i db :Ap pl ic at ion Ve rs ion > < js d l : P O S I X A p pli catio n xmlns:jsdl = " http: // schemas . ggf . or < jsdl:Executable >/ bin / bash </ jsdl:Executable > < jsdl:Argument > -- debugger $ DEBUG ? </ jsdl:Argument > < jsdl:Argument > -v $ VERBOSE ? </ jsdl:Argument > < jsdl:Argument >$ ARGUMENTS ? </ jsdl:Argument > < jsdl:Argument >$ SOURCE ? </ jsdl:Argument > </ j s d l : P O S I X A ppli catio n > </ idb :I DB Ap pl ic ation > 07.07.2009 Slide 10
- 11. Specify Resources { ApplicationName : Date , Resources : { Memory : 268435456 , # per CPU in bytes Runtime : 86400 , # per CPU in seconds Nodes : 3 , CPUs : 64 , # per node # Custom resources ( site - dependent !): S t ac k L im i t Pe r T hr e a d : 262144 } } Please be sparing if you try that here at ISSGC ;-) 07.07.2009 Slide 11
- 12. Data management List a remote directory: > ucc ls u6 :// GILDA - CATANIA / home . ssh . bash_history mydata Copy files between local and remote machine: > ucc get - file -s u6 :// GILDA - CATANIA / home / mydata -t mydata > ucc put - file -s mydata2 -t u6 :// GILDA - CATANIA / home / mydata2 07.07.2009 Slide 12
- 13. Batch Processing Use batch command to submit all files within a directory: > ucc batch -i / my / dir / UCC returns when all jobs are finished Output files are fetched automatically With -f option, ucc waits for more files to be copied in that directory: > ucc batch -f -i / my / dir / 07.07.2009 Slide 13
- 14. UCC from within Emacs UCC commands in Emacs Output highlighting Clickable links in output See ucc-1.1.1/extras/ 07.07.2009 Slide 14
- 15. Advanced usage: Groovy scripting Use groovy scripting to automatetasks, implement custom commands, . . . Simple example: Delete all your jobs import de . fzj . unicore . uas . client .*; def lister = new de . fzj . unicore . ucc . helpers . T a r g e t S y s t e m L i s t e r ( registry ) lister . each { it . jobs . each { messageWriter . message " Job at " + it . address . stringValue new JobClient ( it . address . stringValue , it , s e c u r i t y P r o p e r t i e s ). destroy () } } Run with ucc run-groovy -f delete.groovy 07.07.2009 Slide 15
- 16. More information Information for the practical sessions: www.fz-juelich.de/jsc/unicore/ISSGC09/ UNICORE Website: www.unicore.eu/ UCC Documentation: www.unicore.eu/documentation/manuals/unicore6/ucc/ UCC Scripting with Groovy: http://www.unicore.eu/documentation/files/ucc groovy.pdf Mailing list: unicore-support@lists.sf.net More job examples and Groovy examples can be found in the ucc package. 07.07.2009 Slide 16