.. _install:

Installation
============


Required packages
-----------------

*XCLASS* requires the following packages. (To make sure python version and the related
packages and libraries are as required please see Sect. :ref:`virtual_environment`).

-  gcc (version 7.4.0 or newer),

-  gfortran (version 7.4.0 or newer) with OpenMP,

-  OpenMPI (version 1.8.6),

-  python 3.x (recommended >= 3.12),

-  numpy (version 1.19.5 or newer),

-  scipy (version 1.5.4 or newer),

-  matplotlib (version 3.3.4 or newer),

-  astropy (version 4.1 or newer),

-  spectral_cube (version 0.6.3 or newer),

-  regions (version 0.5 or newer),

-  PyQt5 (version 5.14.1 or newer),

-  h5py (version 3.1.0 or newer),

-  lxml (version 4.9.2 or newer),

-  emcee (version 3.1.4 or newer),

-  ultranest (version 3.6.4 or newer),

-  libraries: libz, libm, libdl, libcurl, libpthread, and libgomp.



.. _virtual_environment:

Virtual Environment:
--------------------

It's recommended to create and activate a virtual environment before installing *XCLASS* package.
This isolates the *XCLASS* package and its dependencies from other projects and you can use the
latest Python version and the latest Python packages, regardless of the version used system-wide.

Python 3.12 is not available everywhere on the servers. It is therefore necessary to install
Miniconda locally in the home directory.

To install Miniconda, you can follow these steps:

*   First download the Miniconda installer by opening a terminal and executing the following
    command

.. code:: shell

    wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh


*   Make the installer executable and run it:

.. code:: shell

    chmod +x Miniconda3-latest-Linux-x86_64.sh
    ./Miniconda3-latest-Linux-x86_64.sh


*   During the installation, you will be prompted to review the license agreement and choose
    the installation location. You can accept the default location or specify a different
    one.


*   After the installation is complete, you may need to initialize Miniconda. The following
    command modifies your shell configuration file to enable conda in your terminal:

.. code:: shell

    ~/miniconda3/bin/conda init


*   Close and reopen your terminal or run the following command to apply the changes:

.. code:: shell

    source ~/.bashrc


*   You can verify that Miniconda is installed correctly by checking the conda version:

.. code:: shell

    conda --version


To create a new virtual environment open a terminal window and use the ``conda create``
command to create a new environment. You can specify the name of the environment and the
Python version you want to use. For example, to create an environment named ``myenv``
with Python 3.12, you would run:

.. code:: shell

    conda create --name myenv python=3.12


You can replace ``myenv`` with your desired environment.

After the environment ``myenv`` is created, you need to activate it. You can do this
with the following command:

.. code:: shell

    conda activate myenv


Once activated, your terminal prompt will change to indicate that you are now working
within the ``myenv`` environment.

When you are done working in the environment, you can deactivate it by running:

.. code:: shell

    conda deactivate


To simplify the activation and deactivation of the virtual environment, it is
advisable to include the following two lines in the .bashrc (or .bash_profile) file

.. code:: shell

    alias venv_on='conda activate myenv'
    alias venv_off='conda deactivate'


Please note that if you use a name other than ``myenv``, you must adapt the definition
of the first alias accordingly.

To fix a bug in PyQt5 please add the following lines to your .bashrc (or .bash_profile)
file as well

.. code:: shell

    ## extend LD_LIBRARY_PATH to use PyQt5 within virtual environment
    export LD_LIBRARY_PATH=/home/USERNAME/miniconda3/envs/myenv/lib/python3.12/site-packages/PyQt5/Qt5/lib/:$LD_LIBRARY_PATH


Please replace ``“USERNAME”`` and ``"myenv"`` here accordingly.

The following command must then be executed so that the changes are immediately available

.. code:: shell

    source ~/.bashrc


The two commands ``“venv_on”`` and ``“venv_off”`` are then available at the command
line to enter and exit the virtual environment.

Within the virtual environment you can now install packages using the  command
``conda install`` or ``pip``. To install the required packages
use

.. code:: shell

    pip install --upgrade pip install numpy scipy matplotlib astropy spectral_cube \
                          regions aplpy lxml lmfit emcee h5py PyQt5 meson


After that you have to install the ultranest package, but this only supports numpy 1.2.6

.. code:: shell

    pip install --upgrade ultranest


You now have a virtual environment with Python 3.12 and all the necessary packages. You
can use this virtual environment from all servers.

To install XCLASS, we can now go to the XCLASS zip file, decompress the zip file, and enter
the virtual environment using ``“venv_on”``. We can now install XCLASS, see Sect.
:ref:`install_sec`.



.. _install_sec:

PyPI
----

In order to install the *XCLASS* package just execute

.. code:: shell

    python3 -m pip install --upgrade -vvv xclasspip/


During compilation, XCLASS creates temporary files with the extension “.out” to simplify
screen output. If an error occurs, you should find such a temporary file in the “xclasspip/”
directory, in which the problems are described in detail.


Mac
---

(Thanks to David Friedlander)

On a Apple Silicon (M1 Pro) hardware architectures the following additional environment variables have to be defined

.. code:: shell

    export FC=gfortran-mp-12
    export CC=gcc-mp-12
    export LDFLAGS="-L/opt/local/lib -lcurl -lz"


For some other env variables we need to determine where things are first, as this will change as MacPorts packages get upgraded
over the years. For each of these items we can search the list of contents of a given package, and then run that output
through "dirname" which gives us the enclosing directory. In each case, use the name of the actual package installed
on *your* system.

"mpif90" compiler

.. code:: shell

    dirname `port contents openmpi-gcc12 | grep mpif90$`

leads to:

.. code:: shell

   export PATH="/opt/local/libexec/openmpi-gcc12/:${PATH}"


"f951" binary (needed by gfortran)

.. code:: shell

    dirname `port contents gcc12 | grep f951`

leads to (don't just copy these-- the CPU architecture matters!) :

.. code:: shell

    export PATH="/opt/local/libexec/gcc/x86_64-apple-darwin22/12.2.0:${PATH}"

or

.. code:: shell

    export PATH="/opt/local/libexec/gcc/arm64-apple-darwin22/12.2.0:${PATH}"

And the code needs to be able to find the omp_lib.mod module:

.. code:: shell

    dirname `port contents gcc12 | grep omp_lib.mod`

leads to:

.. code:: shell

    export FINCLUDE="-I/opt/local/lib/gcc12/gcc/x86_64-apple-darwin22/12.2.0/finclude"

or

.. code:: shell

    export FINCLUDE="-I/opt/local/lib/gcc12/gcc/arm64-apple-darwin22/12.2.0/finclude"


Init file
---------

During the installation process, *XCLASS* creates a new subdirectory, called ``.xclass`` located in the user's home directory, i.e.

::

    /path-to-user-home-directory/.xclass/

This subdirectory contains a small text file called ``init.dat``, which describes some internal parameters used by *XCLASS*.
Additionally, *XCLASS* creates a subdirectory called ``db/``, i.e.

::

    /path-to-user-home-directory/.xclass/db/

which contains the *XCLASS* database file ``cdms_sqlite.db``.


Parallelization
---------------


In order to use the parallelization option of the interface, the user might increase the stack size for OpenMP by adding the following lines to the .bashrc (or .bash_profile) file:

.. code:: shell

    ulimit -s unlimited
    export KMP_STACKSIZE='3999M'
    export OMP_STACKSIZE='3999M'
    export GOMP_STACKSIZE='3999M'

Please note, if more or less RAM is available, please increase/decrease the value ``"3999"`` to a value useful for your machine.


Job directories
---------------

The *XCLASS* interface creates so-called job directories for many *XCLASS* function, where all files created by a function call are stored in.
By default, all these job-directories are stored in a so-called run directory which is created within the .xclass subdirectory with name ``“run”``,
i.e.

::

    /path-to-user-home-directory/.xclass/run/

Sometimes it is useful to create the run directory, not within the *XCLASS* subdirectory. By defining the
environment variable **XCLASSJobDirectory**

.. code:: shell

    export XCLASSJobDirectory="run_somewhere_else"

the user can define another location for the run directory.


Temporary files
---------------

During the fit process, the MAGIX optimization package included in *XCLASS* creates many temporary files, which are written to the temporary directory (``temp``).
This directory is by default also located within the .xclass subdirectory with name ``“temp”``, i.e.

::

    /path-to-user-home-directory/.xclass/temp/

By defining the environment variable **MAGIXTempDirectory**

.. code:: shell

    export MAGIXTempDirectory="temp_somewhere_else"

in the bashrc (or .bash_profile) file, the user can define another location for this temporary directory. It is strongly recommended to use a so-called RAM drive,
i.e. set the environment variable to (Linux users)

.. code:: shell

    export MAGIXTempDirectory="/dev/shm/user-name/"

whenever possible. (The RAM drive is a common name for a temporary file storage facility on many Unix-like operating systems. The usage of a RAM
drive improve the performance of *XCLASS* because the temporary files are not written to the hard drive but to the RAM, which is orders of
magnitude faster.)

For Mac users, add the following lines to your .bash_profile file to create a RAM drive (no guarantee)

.. code:: shell

    ## create RAM drive on Mac
    if [ -d /Volumes/RAMDisk/ ]; then echo ' '; else diskutil erasevolume HFS+ 'RAMDisk' `hdiutil attach -nomount ram://16777216`; fi

    ## create a subdirectory there for user 'user-name'
    if [ -d /Volumes/RAMDisk/user-name/ ]; then echo ' '; else mkdir -p /Volumes/RAMDisk/user-name/; fi

    ## set temp directory environment variable for MAGIX
    export MAGIXTempDirectory=/Volumes/RAMDisk/user-name/


Here, replace the phrase ``“user-name”`` with the corresponding user name.


Troubleshooting
---------------

*   The following error message may appear on Macs

    ::

        --------------------------------------------------------------------------
        A system call failed during shared memory initialization that should
        not have.  It is likely that your MPI job will now either abort or
        experience performance degradation.

        Local host:  gs66-draco
        System call: unlink(2) /var/folders/v9/d5smf9694zjd9q3jtnykh1m46xv24k/T//ompi.gs66-draco.232622226/pid.27703/1/vader_segment.gs66-draco.232622226.caa90001.4
        Error:       No such file or directory (errno 2)
        --------------------------------------------------------------------------

    ->  The error can be fixed by adding

        .. code:: shell

            export TMPDIR=/tmp

        to **.bash_profile** `(see https://github.com/open-mpi/ompi/issues/7393) <https://github.com/open-mpi/ompi/issues/7393>`_

*   ...