path: root/lib/spack/docs/replace_conda_homebrew.rst

.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
   Spack Project Developers. See the top-level COPYRIGHT file for details.

   SPDX-License-Identifier: (Apache-2.0 OR MIT)

=====================================
Spack for Homebrew/Conda Users
=====================================

Spack is an incredibly powerful package manager, designed for supercomputers
where users have diverse installation needs. But Spack can also be used to
handle simple single-user installations on your laptop. Most macOS users are
already familiar with package managers like Homebrew and Conda, where all
installed packages are symlinked to a single central location like ``/usr/local``.
In this section, we will show you how to emulate the behavior of Homebrew/Conda
using :ref:`environments`!

-----
Setup
-----

First, let's create a new environment. We'll assume that Spack is already set up
correctly, and that you've already sourced the setup script for your shell.
To create a new environment, simply run:

.. code-block:: console

   $ spack env create myenv

Here, *myenv* can be anything you want to name your environment. Next, we can add
a list of packages we would like to install into our environment. Let's say we
want a newer version of Bash than the one that comes with macOS, and we want a
few Python libraries. We can run:

.. code-block:: console

   $ spack -e myenv add bash@5 python py-numpy py-scipy py-matplotlib

Each package can be listed on a separate line, or combined into a single line like we did above.
Notice that we're explicitly asking for Bash 5 here. You can use any spec
you would normally use on the command line with other Spack commands.

Next, we want to manually configure a couple of things:

.. code-block:: console

   $ spack -e myenv config edit

.. code-block:: yaml

   # This is a Spack Environment file.
   #
   # It describes a set of packages to be installed, along with
   # configuration settings.
   spack:
     # add package specs to the `specs` list
     specs: [bash@5, python, py-numpy, py-scipy, py-matplotlib]
     view: true

You can see the packages we added earlier in the ``specs:`` section. If you
ever want to add more packages, you can either use ``spack add`` or manually
edit this file.

We also need to change the ``concretizer:unify`` option. By default, Spack
concretizes each spec *separately*, allowing multiple versions of the same
package to coexist. Since we want a single consistent environment, we want to
concretize all of the specs *together*.

Here is what your ``spack.yaml`` looks like with this new setting:

.. code-block:: yaml

   # This is a Spack Environment file.
   #
   # It describes a set of packages to be installed, along with
   # configuration settings.
   spack:
     # add package specs to the `specs` list
     specs: [bash@5, python, py-numpy, py-scipy, py-matplotlib]
     view: true
     concretizer:
       unify: true

^^^^^^^^^^^^^^^^
Symlink location
^^^^^^^^^^^^^^^^

Spack symlinks all installations to ``/Users/me/spack/var/spack/environments/myenv/.spack-env/view``,
which is the default when ``view: true``.
You can actually change this to any directory you want. For example, Homebrew
uses ``/usr/local``, while Conda uses ``/Users/me/anaconda``. In order to access
files in these locations, you need to update ``PATH`` and other environment variables
to point to them. Activating the Spack environment does this automatically, but
you can also manually set them in your ``.bashrc``.

.. warning::

   There are several reasons why you shouldn't use ``/usr/local``:

   1. If you are on macOS 10.11+ (El Capitan and newer), Apple makes it hard
      for you. You may notice permissions issues on ``/usr/local`` due to their
      `System Integrity Protection <https://support.apple.com/en-us/HT204899>`_.
      By default, users don't have permissions to install anything in ``/usr/local``,
      and you can't even change this using ``sudo chown`` or ``sudo chmod``.
   2. Other package managers like Homebrew will try to install things to the
      same directory. If you plan on using Homebrew in conjunction with Spack,
      don't symlink things to ``/usr/local``.
   3. If you are on a shared workstation, or don't have sudo privileges, you
      can't do this.

   If you still want to do this anyway, there are several ways around SIP.
   You could disable SIP by booting into recovery mode and running
   ``csrutil disable``, but this is not recommended, as it can open up your OS
   to security vulnerabilities. Another technique is to run ``spack concretize``
   and ``spack install`` using ``sudo``. This is also not recommended.

   The safest way I've found is to create your installation directories using
   sudo, then change ownership back to the user like so:

   .. code-block:: bash

      for directory in .spack bin contrib include lib man share
      do
          sudo mkdir -p /usr/local/$directory
          sudo chown $(id -un):$(id -gn) /usr/local/$directory
      done

   Depending on the packages you install in your environment, the exact list of
   directories you need to create may vary. You may also find some packages
   like Java libraries that install a single file to the installation prefix
   instead of in a subdirectory. In this case, the action is the same, just replace
   ``mkdir -p`` with ``touch`` in the for-loop above.

   But again, it's safer just to use the default symlink location.


------------
Installation
------------

To actually concretize the environment, run:

.. code-block:: console

   $ spack -e myenv concretize

This will tell you which if any packages are already installed, and alert you
to any conflicting specs.

To actually install these packages and symlink them to your ``view:``
directory, simply run:

.. code-block:: console

   $ spack -e myenv install
   $ spack env activate myenv

Now, when you type ``which python3``, it should find the one you just installed.

In order to change the default shell to our newer Bash installation, we first
need to add it to this list of acceptable shells. Run:

.. code-block:: console

   $ sudo vim /etc/shells

and add the absolute path to your bash executable. Then run:

.. code-block:: console

   $ chsh -s /path/to/bash

Now, when you log out and log back in, ``echo $SHELL`` should point to the
newer version of Bash.

---------------------------
Updating Installed Packages
---------------------------

Let's say you upgraded to a new version of macOS, or a new version of Python
was released, and you want to rebuild your entire software stack. To do this,
simply run the following commands:

.. code-block:: console

   $ spack env activate myenv
   $ spack concretize --fresh --force
   $ spack install

The ``--fresh`` flag tells Spack to use the latest version of every package
where possible instead of trying to optimize for reuse of existing installed
packages.

The ``--force`` flag in addition tells Spack to overwrite its previous
concretization decisions, allowing you to choose a new version of Python.
If any of the new packages like Bash are already installed, ``spack install``
won't re-install them, it will keep the symlinks in place.

-----------------------------------
Updating & Cleaning Up Old Packages
-----------------------------------

If you're looking to mimic the behavior of Homebrew, you may also want to
clean up out-of-date packages from your environment after an upgrade. To
upgrade your entire software stack within an environment and clean up old
package versions, simply run the following commands:

.. code-block:: console

   $ spack env activate myenv
   $ spack mark -i --all
   $ spack concretize --fresh --force
   $ spack install
   $ spack gc

Running ``spack mark -i --all`` tells Spack to mark all of the existing
packages within an environment as "implicitly" installed. This tells
spack's garbage collection system that these packages should be cleaned up.

Don't worry however, this will not remove your entire environment.
Running ``spack install`` will reexamine your spack environment after
a fresh concretization and will re-mark any packages that should remain
installed as "explicitly" installed.

**Note:** if you use multiple spack environments you should re-run ``spack install``
in each of your environments prior to running ``spack gc`` to prevent spack
from uninstalling any shared packages that are no longer required by the
environment you just upgraded.

--------------
Uninstallation
--------------

If you decide that Spack isn't right for you, uninstallation is simple.
Just run:

.. code-block:: console

   $ spack env activate myenv
   $ spack uninstall --all

This will uninstall all packages in your environment and remove the symlinks.