summaryrefslogtreecommitdiff
path: root/lib
diff options
context:
space:
mode:
authorTodd Gamblin <gamblin2@llnl.gov>2022-12-22 02:47:13 -0800
committerGitHub <noreply@github.com>2022-12-22 11:47:13 +0100
commit09864d00c586e0a3cb069e4b1eeee898dd165cb2 (patch)
tree80385e195b85911b870b15a061cfd87011adc809 /lib
parent0f7fa27327031be0f07bba0abd3fc3f90177792b (diff)
downloadspack-09864d00c586e0a3cb069e4b1eeee898dd165cb2.tar.gz
spack-09864d00c586e0a3cb069e4b1eeee898dd165cb2.tar.bz2
spack-09864d00c586e0a3cb069e4b1eeee898dd165cb2.tar.xz
spack-09864d00c586e0a3cb069e4b1eeee898dd165cb2.zip
docs: remove monitors and analyzers (#34358)
These experimental features were removed in #31130, but the docs were not. - [x] remove the `spack monitor` and `spack analyze` docs
Diffstat (limited to 'lib')
-rw-r--r--lib/spack/docs/analyze.rst162
-rw-r--r--lib/spack/docs/index.rst7
-rw-r--r--lib/spack/docs/monitoring.rst265
3 files changed, 0 insertions, 434 deletions
diff --git a/lib/spack/docs/analyze.rst b/lib/spack/docs/analyze.rst
deleted file mode 100644
index 197c127e35..0000000000
--- a/lib/spack/docs/analyze.rst
+++ /dev/null
@@ -1,162 +0,0 @@
-.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
- Spack Project Developers. See the top-level COPYRIGHT file for details.
-
- SPDX-License-Identifier: (Apache-2.0 OR MIT)
-
-.. _analyze:
-
-=======
-Analyze
-=======
-
-
-The analyze command is a front-end to various tools that let us analyze
-package installations. Each analyzer is a module for a different kind
-of analysis that can be done on a package installation, including (but not
-limited to) binary, log, or text analysis. Thus, the analyze command group
-allows you to take an existing package install, choose an analyzer,
-and extract some output for the package using it.
-
-
------------------
-Analyzer Metadata
------------------
-
-For all analyzers, we write to an ``analyzers`` folder in ``~/.spack``, or the
-value that you specify in your spack config at ``config:analyzers_dir``.
-For example, here we see the results of running an analysis on zlib:
-
-.. code-block:: console
-
- $ tree ~/.spack/analyzers/
- └── linux-ubuntu20.04-skylake
- └── gcc-9.3.0
- └── zlib-1.2.11-sl7m27mzkbejtkrajigj3a3m37ygv4u2
- ├── environment_variables
- │   └── spack-analyzer-environment-variables.json
- ├── install_files
- │   └── spack-analyzer-install-files.json
- └── libabigail
- └── spack-analyzer-libabigail-libz.so.1.2.11.xml
-
-
-This means that you can always find analyzer output in this folder, and it
-is organized with the same logic as the package install it was run for.
-If you want to customize this top level folder, simply provide the ``--path``
-argument to ``spack analyze run``. The nested organization will be maintained
-within your custom root.
-
------------------
-Listing Analyzers
------------------
-
-If you aren't familiar with Spack's analyzers, you can quickly list those that
-are available:
-
-.. code-block:: console
-
- $ spack analyze list-analyzers
- install_files : install file listing read from install_manifest.json
- environment_variables : environment variables parsed from spack-build-env.txt
- config_args : config args loaded from spack-configure-args.txt
- libabigail : Application Binary Interface (ABI) features for objects
-
-
-In the above, the first three are fairly simple - parsing metadata files from
-a package install directory to save
-
--------------------
-Analyzing a Package
--------------------
-
-The analyze command, akin to install, will accept a package spec to perform
-an analysis for. The package must be installed. Let's walk through an example
-with zlib. We first ask to analyze it. However, since we have more than one
-install, we are asked to disambiguate:
-
-.. code-block:: console
-
- $ spack analyze run zlib
- ==> Error: zlib matches multiple packages.
- Matching packages:
- fz2bs56 zlib@1.2.11%gcc@7.5.0 arch=linux-ubuntu18.04-skylake
- sl7m27m zlib@1.2.11%gcc@9.3.0 arch=linux-ubuntu20.04-skylake
- Use a more specific spec.
-
-
-We can then specify the spec version that we want to analyze:
-
-.. code-block:: console
-
- $ spack analyze run zlib/fz2bs56
-
-If you don't provide any specific analyzer names, by default all analyzers
-(shown in the ``list-analyzers`` subcommand list) will be run. If an analyzer does not
-have any result, it will be skipped. For example, here is a result running for
-zlib:
-
-.. code-block:: console
-
- $ ls ~/.spack/analyzers/linux-ubuntu20.04-skylake/gcc-9.3.0/zlib-1.2.11-sl7m27mzkbejtkrajigj3a3m37ygv4u2/
- spack-analyzer-environment-variables.json
- spack-analyzer-install-files.json
- spack-analyzer-libabigail-libz.so.1.2.11.xml
-
-If you want to run a specific analyzer, ask for it with `--analyzer`. Here we run
-spack analyze on libabigail (already installed) _using_ libabigail1
-
-.. code-block:: console
-
- $ spack analyze run --analyzer abigail libabigail
-
-
-.. _analyze_monitoring:
-
-----------------------
-Monitoring An Analysis
-----------------------
-
-For any kind of analysis, you can
-use a `spack monitor <https://github.com/spack/spack-monitor>`_ "Spackmon"
-as a server to upload the same run metadata to. You can
-follow the instructions in the `spack monitor documentation <https://spack-monitor.readthedocs.org>`_
-to first create a server along with a username and token for yourself.
-You can then use this guide to interact with the server.
-
-You should first export our spack monitor token and username to the environment:
-
-.. code-block:: console
-
- $ export SPACKMON_TOKEN=50445263afd8f67e59bd79bff597836ee6c05438
- $ export SPACKMON_USER=spacky
-
-
-By default, the host for your server is expected to be at ``http://127.0.0.1``
-with a prefix of ``ms1``, and if this is the case, you can simply add the
-``--monitor`` flag to the install command:
-
-.. code-block:: console
-
- $ spack analyze run --monitor wget
-
-If you need to customize the host or the prefix, you can do that as well:
-
-.. code-block:: console
-
- $ spack analyze run --monitor --monitor-prefix monitor --monitor-host https://monitor-service.io wget
-
-If your server doesn't have authentication, you can skip it:
-
-.. code-block:: console
-
- $ spack analyze run --monitor --monitor-disable-auth wget
-
-Regardless of your choice, when you run analyze on an installed package (whether
-it was installed with ``--monitor`` or not, you'll see the results generating as they did
-before, and a message that the monitor server was pinged:
-
-.. code-block:: console
-
- $ spack analyze --monitor wget
- ...
- ==> Sending result for wget bin/wget to monitor.
diff --git a/lib/spack/docs/index.rst b/lib/spack/docs/index.rst
index b755f2f376..e2e1d9c8e8 100644
--- a/lib/spack/docs/index.rst
+++ b/lib/spack/docs/index.rst
@@ -67,7 +67,6 @@ or refer to the full manual below.
build_settings
environments
containers
- monitoring
mirrors
module_file_support
repositories
@@ -80,12 +79,6 @@ or refer to the full manual below.
.. toctree::
:maxdepth: 2
- :caption: Research
-
- analyze
-
-.. toctree::
- :maxdepth: 2
:caption: Contributing
contribution_guide
diff --git a/lib/spack/docs/monitoring.rst b/lib/spack/docs/monitoring.rst
deleted file mode 100644
index eaf57a8ad7..0000000000
--- a/lib/spack/docs/monitoring.rst
+++ /dev/null
@@ -1,265 +0,0 @@
-.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
- Spack Project Developers. See the top-level COPYRIGHT file for details.
-
- SPDX-License-Identifier: (Apache-2.0 OR MIT)
-
-.. _monitoring:
-
-==========
-Monitoring
-==========
-
-You can use a `spack monitor <https://github.com/spack/spack-monitor>`_ "Spackmon"
-server to store a database of your packages, builds, and associated metadata
-for provenance, research, or some other kind of development. You should
-follow the instructions in the `spack monitor documentation <https://spack-monitor.readthedocs.org>`_
-to first create a server along with a username and token for yourself.
-You can then use this guide to interact with the server.
-
--------------------
-Analysis Monitoring
--------------------
-
-To read about how to monitor an analysis (meaning you want to send analysis results
-to a server) see :ref:`analyze_monitoring`.
-
----------------------
-Monitoring An Install
----------------------
-
-Since an install is typically when you build packages, we logically want
-to tell spack to monitor during this step. Let's start with an example
-where we want to monitor the install of hdf5. Unless you have disabled authentication
-for the server, we first want to export our spack monitor token and username to the environment:
-
-.. code-block:: console
-
- $ export SPACKMON_TOKEN=50445263afd8f67e59bd79bff597836ee6c05438
- $ export SPACKMON_USER=spacky
-
-
-By default, the host for your server is expected to be at ``http://127.0.0.1``
-with a prefix of ``ms1``, and if this is the case, you can simply add the
-``--monitor`` flag to the install command:
-
-.. code-block:: console
-
- $ spack install --monitor hdf5
-
-
-If you need to customize the host or the prefix, you can do that as well:
-
-.. code-block:: console
-
- $ spack install --monitor --monitor-prefix monitor --monitor-host https://monitor-service.io hdf5
-
-
-As a precaution, we cut out early in the spack client if you have not provided
-authentication credentials. For example, if you run the command above without
-exporting your username or token, you'll see:
-
-.. code-block:: console
-
- ==> Error: You are required to export SPACKMON_TOKEN and SPACKMON_USER
-
-This extra check is to ensure that we don't start any builds,
-and then discover that you forgot to export your token. However, if
-your monitoring server has authentication disabled, you can tell this to
-the client to skip this step:
-
-.. code-block:: console
-
- $ spack install --monitor --monitor-disable-auth hdf5
-
-If the service is not running, you'll cleanly exit early - the install will
-not continue if you've asked it to monitor and there is no service.
-For example, here is what you'll see if the monitoring service is not running:
-
-.. code-block:: console
-
- [Errno 111] Connection refused
-
-
-If you want to continue builds (and stop monitoring) you can set the ``--monitor-keep-going``
-flag.
-
-.. code-block:: console
-
- $ spack install --monitor --monitor-keep-going hdf5
-
-This could mean that if a request fails, you only have partial or no data
-added to your monitoring database. This setting will not be applied to the
-first request to check if the server is running, but to subsequent requests.
-If you don't have a monitor server running and you want to build, simply
-don't provide the ``--monitor`` flag! Finally, if you want to provide one or
-more tags to your build, you can do:
-
-.. code-block:: console
-
- # Add one tag, "pizza"
- $ spack install --monitor --monitor-tags pizza hdf5
-
- # Add two tags, "pizza" and "pasta"
- $ spack install --monitor --monitor-tags pizza,pasta hdf5
-
-
-----------------------------
-Monitoring with Containerize
-----------------------------
-
-The same argument group is available to add to a containerize command.
-
-^^^^^^
-Docker
-^^^^^^
-
-To add monitoring to a Docker container recipe generation using the defaults,
-and assuming a monitor server running on localhost, you would
-start with a spack.yaml in your present working directory:
-
-.. code-block:: yaml
-
- spack:
- specs:
- - samtools
-
-And then do:
-
-.. code-block:: console
-
- # preview first
- spack containerize --monitor
-
- # and then write to a Dockerfile
- spack containerize --monitor > Dockerfile
-
-
-The install command will be edited to include commands for enabling monitoring.
-However, getting secrets into the container for your monitor server is something
-that should be done carefully. Specifically you should:
-
- - Never try to define secrets as ENV, ARG, or using ``--build-arg``
- - Do not try to get the secret into the container via a "temporary" file that you remove (it in fact will still exist in a layer)
-
-Instead, it's recommended to use buildkit `as explained here <https://pythonspeed.com/articles/docker-build-secrets/>`_.
-You'll need to again export environment variables for your spack monitor server:
-
-.. code-block:: console
-
- $ export SPACKMON_TOKEN=50445263afd8f67e59bd79bff597836ee6c05438
- $ export SPACKMON_USER=spacky
-
-And then use buildkit along with your build and identifying the name of the secret:
-
-.. code-block:: console
-
- $ DOCKER_BUILDKIT=1 docker build --secret id=st,env=SPACKMON_TOKEN --secret id=su,env=SPACKMON_USER -t spack/container .
-
-The secrets are expected to come from your environment, and then will be temporarily mounted and available
-at ``/run/secrets/<name>``. If you forget to supply them (and authentication is required) the build
-will fail. If you need to build on your host (and interact with a spack monitor at localhost) you'll
-need to tell Docker to use the host network:
-
-.. code-block:: console
-
- $ DOCKER_BUILDKIT=1 docker build --network="host" --secret id=st,env=SPACKMON_TOKEN --secret id=su,env=SPACKMON_USER -t spack/container .
-
-
-^^^^^^^^^^^
-Singularity
-^^^^^^^^^^^
-
-To add monitoring to a Singularity container build, the spack.yaml needs to
-be modified slightly to specify wanting a different format:
-
-
-.. code-block:: yaml
-
- spack:
- specs:
- - samtools
- container:
- format: singularity
-
-
-Again, generate the recipe:
-
-
-.. code-block:: console
-
- # preview first
- $ spack containerize --monitor
-
- # then write to a Singularity recipe
- $ spack containerize --monitor > Singularity
-
-
-Singularity doesn't have a direct way to define secrets at build time, so we have
-to do a bit of a manual command to add a file, source secrets in it, and remove it.
-Since Singularity doesn't have layers like Docker, deleting a file will truly
-remove it from the container and history. So let's say we have this file,
-``secrets.sh``:
-
-.. code-block:: console
-
- # secrets.sh
- export SPACKMON_USER=spack
- export SPACKMON_TOKEN=50445263afd8f67e59bd79bff597836ee6c05438
-
-
-We would then generate the Singularity recipe, and add a files section,
-a source of that file at the start of ``%post``, and **importantly**
-a removal of the final at the end of that same section.
-
-.. code-block::
-
- Bootstrap: docker
- From: spack/ubuntu-bionic:latest
- Stage: build
-
- %files
- secrets.sh /opt/secrets.sh
-
- %post
- . /opt/secrets.sh
-
- # spack install commands are here
- ...
-
- # Don't forget to remove here!
- rm /opt/secrets.sh
-
-
-You can then build the container as your normally would.
-
-.. code-block:: console
-
- $ sudo singularity build container.sif Singularity
-
-
-------------------
-Monitoring Offline
-------------------
-
-In the case that you want to save monitor results to your filesystem
-and then upload them later (perhaps you are in an environment where you don't
-have credentials or it isn't safe to use them) you can use the ``--monitor-save-local``
-flag.
-
-.. code-block:: console
-
- $ spack install --monitor --monitor-save-local hdf5
-
-This will save results in a subfolder, "monitor" in your designated spack
-reports folder, which defaults to ``$HOME/.spack/reports/monitor``. When
-you are ready to upload them to a spack monitor server:
-
-
-.. code-block:: console
-
- $ spack monitor upload ~/.spack/reports/monitor
-
-
-You can choose the root directory of results as shown above, or a specific
-subdirectory. The command accepts other arguments to specify configuration
-for the monitor.