Conda environment system & how to use it on CSUC machines
Download as PPTX, PDF0 likes241 views
The document provides a comprehensive guide on using the conda environment system, emphasizing the transition from shared to private environments for users on CSUC machines. It details the functionalities of conda, including installation, environment management, package handling, and best practices for efficient usage. Users are encouraged to contact support for assistance during the transition and to streamline their conda environment setup.
![Activating/Deactivating environments
• To see the contents of an env: conda list [-n env_name]
(by default, currently activated environment)
• Note: source activate and source deactivate are obsolete](https://image.slidesharecdn.com/presentacioconda-210317085426/85/Conda-environment-system-how-to-use-it-on-CSUC-machines-14-320.jpg)
![Environments
• Users can create their own private envs (stored at
$HOME/.conda/envs) visible only to them
• To create a new empty environment: conda
create -n <env_name>
• To create a new environment with packages
preinstalled in it: conda create –n <env_name>
[list of packages]](https://image.slidesharecdn.com/presentacioconda-210317085426/85/Conda-environment-system-how-to-use-it-on-CSUC-machines-15-320.jpg)
![Environments
• To install one or more packages in a private
env: conda install [-n env_name] <packages>
• If no env is specified, they are installed in the
currently active environment
• Version and channel can also be
specified: conda install [-n env_name] [-C
channel] <package=version>](https://image.slidesharecdn.com/presentacioconda-210317085426/85/Conda-environment-system-how-to-use-it-on-CSUC-machines-16-320.jpg)
![Environments
• To update packages in an environment: conda
update [-n env_name] <specific packages> or
conda update [-n env_name] --all
• To uninstall packages: conda remove [-n
env_name] <packages>
• To completely delete an environment: conda
remove -n <env_name> --all](https://image.slidesharecdn.com/presentacioconda-210317085426/85/Conda-environment-system-how-to-use-it-on-CSUC-machines-17-320.jpg)
Conda environment system & how to use it on CSUC machines
- 1. Conda environment system & how to use it on CSUC machines Víctor Pérez 17 / 03 / 2021
- 2. Index 1. Notice: model transition 2. What is conda? 3. Scope of the project 4. How to use conda 5. Conda environments 6. Package management 7. Python through conda 8. R through conda
- 3. Notice: model transition • Very important: we are in the process of abandoning shared environments in favour of private environments only • Old shared environments will be available for your old scripts, but won't be modified further • We will clone them as private environments for users and groups who need them
- 4. Notice: model transition • We will progressively contact users of shared environments to assist you in the "privatisation" of the environments you need • If you already use private environments, simply change the module you load and that's all, no transition needed :) • New live and minimal conda installation: module load conda/current
- 5. What is conda? • Originally: Anaconda, a distribution of Python including common scientific packages https://www.anaconda.com/ • Extended to include R and R packages, scientific libraries, other software, etc. • conda: core package manager for the Anaconda project
- 6. What is conda? • conda installs and updates binary versions of Python and R packages from its own (or third party) repositories • It is an alternative to other repository systems, like pip for Python or CRAN for R • It is also a way to manage dependencies for Python and R packages
- 7. But conda isn't... • A repository of system software packages (such as apt or yum) • A repository of source code • A replacement for environment modules • Exhaustive or infallible
- 8. Scope of the project • Python • R • Jupyter, Spyder, Rstudio... • NumPy, SciPy, Pandas, Numba, Dask... • Scikit-learn, TensorFlow, Theano... • Matplotlib, Bokeh, Datashader, Holoviews... • A variety of external libraries and tools
- 9. Conda channels • Channels are thematic collections of packages, useful to avoid version conflicts (equivalent to repositories) • Examples: – pkgs/main: default channel – conda-forge: large collection of third party packages – bioconda: software for bioinformatics – r: tailored to R users
- 10. How to use conda • We need to load an environment module to configure conda. module load conda/current Obsolete versions for backwards compatibility: module load conda/2 & module load conda/3
- 11. How to use conda • Then we use the command conda (+ action) to run it: conda list conda activate conda create conda search conda install conda update conda help
- 12. Conda environments • Inside a given installation of conda, there are a number of environments • Environments are profiles: each will have a different list of packages and versions installed • We are transitioning to working exclusively with private environments owned by users
- 13. Activating/Deactivating environments • To see a list of environments: conda env list • To load an env: conda activate <env_name> • To unload: conda deactivate
- 14. Activating/Deactivating environments • To see the contents of an env: conda list [-n env_name] (by default, currently activated environment) • Note: source activate and source deactivate are obsolete
- 15. Environments • Users can create their own private envs (stored at $HOME/.conda/envs) visible only to them • To create a new empty environment: conda create -n <env_name> • To create a new environment with packages preinstalled in it: conda create –n <env_name> [list of packages]
- 16. Environments • To install one or more packages in a private env: conda install [-n env_name] <packages> • If no env is specified, they are installed in the currently active environment • Version and channel can also be specified: conda install [-n env_name] [-C channel] <package=version>
- 17. Environments • To update packages in an environment: conda update [-n env_name] <specific packages> or conda update [-n env_name] --all • To uninstall packages: conda remove [-n env_name] <packages> • To completely delete an environment: conda remove -n <env_name> --all
- 18. Environments • Private environments can be shared with other users in your group • To be able to use environments created by your group coleague, create a .condarc file in your home with the following lines: • Or just contact us to do it for you!
- 19. Using Python through conda #!/bin/bash #SBATCH –p std #SBATCH –N 1 #SBATCH –n 1 module load conda/current conda activate <your_environment> python example.py
- 20. Using R through conda #!/bin/bash #SBATCH –p std #SBATCH –N 1 #SBATCH –n 1 module load conda/current conda activate <your_environment> Rscript example.R
- 21. Using pip or CRAN through conda • It is possible to install Python packages using pip, or R packages using BioConductor or CRAN, but it requires configuring a proxy • Let us know if you need to do this so we can set up proxy details for you
- 22. Best practices • It is more convenient to designate one person in the group as environment manager and install/configure everything on their account • Avoid clutter; it's better to create multiple single-purpose environments than one large environment with too many packages • Be mindful of version collision when updating environments; if you don't need to update, don't • When in doubt, contact us – we can do it for you