Use spack/user-specific stage root by default; stage cleaning (#12516)

* When cleaning the stage root, only remove directories that appear
  to be used for staging Spack packages. Previously Spack was clearing
  all directories in the stage root, which could remove content not
  related to Spack if the user chose a staging root which contains
  files/directories not managed by Spack.
* The documentation is updated with warnings about choosing a stage
  directory that is only managed by Spack (although generally the
  check added in this PR for "spack clean" should avoid removing
  content that was not created by Spack)
* The default stage directory (in config.yaml) is now
  $tempdir/$user/spack-stage and the logic is updated to omit the
  $user portion of this path if $tempdir already contains a $user
  directory.
* When creating stage root assign user read/write permissions to all
  directories in the path under $user. Previously Spack was assigning
  the permissions of the first existing parent directory

1 files changed, 12 insertions, 7 deletions

diff --git a/etc/spack/defaults/config.yaml b/etc/spack/defaults/config.yaml
index 33a37bbdfa..1b6d83f359 100644
--- a/etc/spack/defaults/config.yaml
+++ b/etc/spack/defaults/config.yaml
@@ -40,11 +40,8 @@ config:
   # Recommended options are given below.
   #
   # Builds can be faster in temporary directories on some (e.g., HPC) systems.
-  # Specifying `$tempdir` will ensure use of the system default temporary
-  # directory (as returned by `tempfile.gettempdir()`).  Spack will append
-  # `spack-stage` and, if the username is not already in the path, the value
-  # of `$user` to the path.  The latter is used to avoid conflicts between
-  # users in shared temporary spaces.
+  # Specifying `$tempdir` will ensure use of the default temporary directory
+  # (i.e., ``$TMP` or ``$TMPDIR``).
   #
   # Another option that prevents conflicts and potential permission issues is
   # to specify `~/.spack/stage`, which ensures each user builds in their home
@@ -52,13 +49,21 @@ config:
   #
   # A more traditional path uses the value of `$spack/var/spack/stage`, which
   # builds directly inside Spack's instance without staging them in a
-  # temporary space.
+  # temporary space.  Problems with specifying a path inside a Spack instance
+  # are that it precludes its use as a system package and its ability to be
+  # pip installable.
+  #
+  # In any case, if the username is not already in the path, Spack will append
+  # the value of `$user` in an attempt to avoid potential conflicts between
+  # users in shared temporary spaces.
   #
   # The build stage can be purged with `spack clean --stage` and
   # `spack clean -a`, so it is important that the specified directory uniquely
   # identifies Spack staging to avoid accidentally wiping out non-Spack work.
   build_stage:
-    - $tempdir/spack-stage
+    - $tempdir/$user/spack-stage
+    - ~/.spack/stage
+  # - $spack/var/spack/stage
 
 
   # Cache directory for already downloaded source tarballs and archived