diff options
Diffstat (limited to 'lib')
| -rw-r--r-- | lib/spack/docs/workflows.rst | 242 |
1 files changed, 242 insertions, 0 deletions
diff --git a/lib/spack/docs/workflows.rst b/lib/spack/docs/workflows.rst index 914f84041b..b329a0205c 100644 --- a/lib/spack/docs/workflows.rst +++ b/lib/spack/docs/workflows.rst @@ -1095,6 +1095,248 @@ or filesystem views. However, it has some drawbacks: integrate Spack explicitly in their workflow. Not all users are willing to do this. +------------------------------------- +Using Spack to Replace Homebrew/Conda +------------------------------------- + +Spack is an incredibly powerful package manager, designed for supercomputers +where users have diverse installation needs. But Spack can also be used to +handle simple single-user installations on your laptop. Most macOS users are +already familiar with package managers like Homebrew and Conda, where all +installed packages are symlinked to a single central location like ``/usr/local``. +In this section, we will show you how to emulate the behavior of Homebrew/Conda +using :ref:`environments`! + +^^^^^ +Setup +^^^^^ + +First, let's create a new environment. We'll assume that Spack is already set up +correctly, and that you've already sourced the setup script for your shell. +To create a new environment, simply run: + +.. code-block:: console + + $ spack env create myenv + ==> Updating view at /Users/me/spack/var/spack/environments/myenv/.spack-env/view + ==> Created environment 'myenv' in /Users/me/spack/var/spack/environments/myenv + $ spack env activate myenv + +Here, *myenv* can be anything you want to name your environment. Next, we can add +a list of packages we would like to install into our environment. Let's say we +want a newer version of Bash than the one that comes with macOS, and we want a +few Python libraries. We can run: + +.. code-block:: console + + $ spack add bash + ==> Adding bash to environment myenv + ==> Updating view at /Users/me/spack/var/spack/environments/myenv/.spack-env/view + $ spack add python@3: + ==> Adding python@3: to environment myenv + ==> Updating view at /Users/me/spack/var/spack/environments/myenv/.spack-env/view + $ spack add py-numpy py-scipy py-matplotlib + ==> Adding py-numpy to environment myenv + ==> Adding py-scipy to environment myenv + ==> Adding py-matplotlib to environment myenv + ==> Updating view at /Users/me/spack/var/spack/environments/myenv/.spack-env/view + +Each package can be listed on a separate line, or combined into a single line. +Notice that we're explicitly asking for Python 3 here. You can use any spec +you would normally use on the command line with other Spack commands. + +Next, we want to manually configure a couple of things. In the ``myenv`` +directory, we can find the ``spack.yaml`` that actually defines our environment. + +.. code-block:: console + + $ vim ~/spack/var/spack/environments/myenv/spack.yaml + +.. code-block:: yaml + + # This is a Spack Environment file. + # + # It describes a set of packages to be installed, along with + # configuration settings. + spack: + # add package specs to the `specs` list + specs: [bash, 'python@3:', py-numpy, py-scipy, py-matplotlib] + view: + default: + root: /Users/me/spack/var/spack/environments/myenv/.spack-env/view + projections: {} + config: {} + mirrors: {} + modules: + enable: [] + packages: {} + repos: [] + upstreams: {} + definitions: [] + concretization: separately + +You can see the packages we added earlier in the ``specs:`` section. If you +ever want to add more packages, you can either use ``spack add`` or manually +edit this file. + +We also need to change the ``concretization:`` option. By default, Spack +concretizes each spec *separately*, allowing multiple versions of the same +package to coexist. Since we want a single consistent environment, we want to +concretize all of the specs *together*. + +Here is what your ``spack.yaml`` looks like with these new settings, and with +some of the sections we don't plan on using removed: + +.. code-block:: diff + + spack: + - specs: [bash, 'python@3:', py-numpy, py-scipy, py-matplotlib] + + specs: + + - bash + + - 'python@3:' + + - py-numpy + + - py-scipy + + - py-matplotlib + - view: + - default: + - root: /Users/me/spack/var/spack/environments/myenv/.spack-env/view + - projections: {} + + view: /Users/me/spack/var/spack/environments/myenv/.spack-env/view + - config: {} + - mirrors: {} + - modules: + - enable: [] + - packages: {} + - repos: [] + - upstreams: {} + - definitions: [] + + concretization: together + - concretization: separately + +"""""""""""""""" +Symlink location +"""""""""""""""" + +In the ``spack.yaml`` file above, you'll notice that by default, Spack symlinks +all installations to ``/Users/me/spack/var/spack/environments/myenv/.spack-env/view``. +You can actually change this to any directory you want. For example, Homebrew +uses ``/usr/local``, while Conda uses ``/Users/me/anaconda``. In order to access +files in these locations, you need to update ``PATH`` and other environment variables +to point to them. Activating the Spack environment does this automatically, but +you can also manually set them in your ``.bashrc``. + +.. warning:: + + There are several reasons why you shouldn't use ``/usr/local``: + + 1. If you are on macOS 10.11+ (El Capitan and newer), Apple makes it hard + for you. You may notice permissions issues on ``/usr/local`` due to their + `System Integrity Protection <https://support.apple.com/en-us/HT204899>`_. + By default, users don't have permissions to install anything in ``/usr/local``, + and you can't even change this using ``sudo chown`` or ``sudo chmod``. + 2. Other package managers like Homebrew will try to install things to the + same directory. If you plan on using Homebrew in conjunction with Spack, + don't symlink things to ``/usr/local``. + 3. If you are on a shared workstation, or don't have sudo priveleges, you + can't do this. + + If you still want to do this anyway, there are several ways around SIP. + You could disable SIP by booting into recovery mode and running + ``csrutil disable``, but this is not recommended, as it can open up your OS + to security vulnerabilities. Another technique is to run ``spack concretize`` + and ``spack install`` using ``sudo``. This is also not recommended. + + The safest way I've found is to create your installation directories using + sudo, then change ownership back to the user like so: + + .. code-block:: bash + + for directory in .spack bin contrib include lib man share + do + sudo mkdir -p /usr/local/$directory + sudo chown $(id -un):$(id -gn) /usr/local/$directory + done + + Depending on the packages you install in your environment, the exact list of + directories you need to create may vary. You may also find some packages + like Java libraries that install a single file to the installation prefix + instead of in a subdirectory. In this case, the action is the same, just replace + ``mkdir -p`` with ``touch`` in the for-loop above. + + But again, it's safer just to use the default symlink location. + + +^^^^^^^^^^^^ +Installation +^^^^^^^^^^^^ + +To actually concretize the environment, run: + +.. code-block:: console + + $ spack concretize + +This will tell you which if any packages are already installed, and alert you +to any conflicting specs. + +To actually install these packages and symlink them to your ``view:`` +directory, simply run: + +.. code-block:: console + + $ spack install + +Now, when you type ``which python3``, it should find the one you just installed. + +In order to change the default shell to our newer Bash installation, we first +need to add it to this list of acceptable shells. Run: + +.. code-block:: console + + $ sudo vim /etc/shells + +and add the absolute path to your bash executable. Then run: + +.. code-block:: console + + $ chsh -s /path/to/bash + +Now, when you log out and log back in, ``echo $SHELL`` should point to the +newer version of Bash. + +^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Updating Installed Packages +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Let's say you upgraded to a new version of macOS, or a new version of Python +was released, and you want to rebuild your entire software stack. To do this, +simply run the following commands: + +.. code-block:: console + + $ spack env activate myenv + $ spack concretize --force + $ spack install + +The ``--force`` flag tells Spack to overwrite its previous concretization +decisions, allowing you to choose a new version of Python. If any of the new +packages like Bash are already installed, ``spack install`` won't re-install +them, it will keep the symlinks in place. + +^^^^^^^^^^^^^^ +Uninstallation +^^^^^^^^^^^^^^ + +If you decide that Spack isn't right for you, uninstallation is simple. +Just run: + +.. code-block:: console + + $ spack env activate myenv + $ spack uninstall --all + +This will uninstall all packages in your environment and remove the symlinks. + ------------------------ Using Spack on Travis-CI ------------------------ |