Installation requirements.txt

[benglvr]

Suggestion: Create requirements.txt file (pip freeze > requirements.txt) which can be installed with pip install -r requirements.txt for easy installation.

For debian 8 / python 2.7.14 in a fresh pyenv, the following is installed (using release_candidate branch)

backports.functools-lru-cache==1.5
cycler==0.10.0
h5py==2.7.1
kiwisolver==1.0.1
matplotlib==2.2.2
mpi4py==3.0.0
numpy==1.14.2
Pillow==5.1.0
pyFFTW==0.10.4
pyparsing==2.2.0
python-dateutil==2.7.2
pytz==2018.4
pyzmq==17.0.0
scipy==1.0.1
six==1.11.0
subprocess32==3.2.7

Tests: 3 failed, 133 passed (?)
https://gist.github.com/benglvr/4e8fbebc778f8697292487542cf019e0
minimal_prep_and_run.py runs and generates an image as expected

Additionally the following system packages were required for some of these pip extensions to compile (and to run ptypy under mpiexec) - should be added to documentation
sudo apt-get install openmpi-bin
sudo apt-get install mpich2
sudo apt-get install libopenmpi-dev
sudo apt-get install libfftw3-dev
There may be other requirements that I already have installed.

[aaron-parsons]

Hi Ben,

Sorry it took me so long to get back to you! I was involved in some projects.

Currently the way we manage our dependencies is via conda. You'll notice an environment.yml file in the repo which you can activate using conda env create --file environment.yml. The rest of the install commands are inside .travis.yml file which is tested by travis every time someone pushes to their remote.

This means that our versions are always up to date with the latest versions available in conda and we spot any bugs ahead of time.

We are currently revising the dependencies to make sure only core dependencies are required, with additional optional dependencies supplying additional functionality.

We will change this for 0.4 to rely on conda recipes instead which are much nicer.

Have you tried the environment.yml? Did you find any bugs?

[benglvr]

Hi Aaron,

No worries about the delay. I'll have a go on Monday installing this under conda in a fresh virtual machine and get back to you.

Regards

[Nordicus]

Trying to get started as a new user, I think this is possibly the correct open issue to continue.
The documentation is currently somewhat unclear on how to perform a clean installation.

Might I recommend removing the setup.py (which seems to have been left to code-rot long enough that I couldn't get a valid setup using it), and changing the documentation to suggest the use of conda? For example, the installation quicklinks in the docs still suggest using setup.py, and python 2.7.

Less obviously fixable though is that even after a clean install, the tests all fail, commenting lines 133-136 of plot_utils.py allows some tests to pass (as all of them fail for a PIL reason) but that still leaves 52 tests failing in a clean install.

If we're cleaning up clean installations, I would also recommend removing the .idea directory. Happy to make these changes myself. Thoughts?

[jcesardasilva]

Hi,
I just wanted to follow up the remarks by @Nordicus . Indeed, we could reorganise things there, but if I may, I would like to avoid the installation only via Conda (as for example, Tomopy does and it is a mess for me and it not so easy to create containers with Conda - at least I don't like). I don't see a problem in keeping the setup.py file, but one could make it better. If Ptypy will be installable via Conda, could it be also installable via Pip (via the twine package) for people not using Conda at all as myself?

[Nordicus]

@jcesardasilva Unfortunately, the setup.py file in its current implementation is broken.
It does not install the dependencies, and therefore the package is not installable through its use. I'd recommend either fixing the file (which may be as simple as uncommenting the dependencies for an equivalent of the core-dependencies.yml) or may be more challenging to have both core- and full- dependency options. I'm not super familiar with setuptools. However, I believe it either needs to be fixed or removed.

[bjoernenders]

@Nordicus @jcesardasilva We are preparing a new release that is essentially ready (and also fixes a lot of things in the docs) We moved to conda as the primary installation medium. I'm happy to take a PR though (after the release) if you want to contribute a requirements.txt for a non-conda installation. The new release will be happening this week. If you don't want to wait @Nordicus the pycuda branch is the basis for the new release.

[daurer]

@Nordicus Welcome to our community of ptypy users :) sorry to hear that you were having troubles getting started with a clean installation and apologies for the lack of documentation! We have finally managed to get the 0.5 relase over the line, with new updated documentation here: https://ptycho.github.io/ptypy/rst/getting_started.html#installation

I agree that our installation process might be a little "old-style" but we are trying to keep things flexible and don't want to make too many radical changes to accommodate our existing users at the different facilities, see for example the comment from @jcesardasilva. Our recommendation (as now stated in the docs) is to install all the necessary dependencies inside a fresh conda environment (e.g. using the dependencies_full.yml provided) and then locallly pip install ptypy into that environment. This approach has been extensively tested (on linux) with Python 3.7,3.8 and 3.9 and seems to work fine.

That being said, we are always happy to improve our code base so feel free to create a PR with your suggestions for improvement.

Further to your point regarding the PIL errors. There has been a recent change in PIL 9.1.0 which is being shipped with the Python 3.10 conda environment. This caused some errors on our end, which have been fixed with this PR and is already merged into master, and this happened before we made the 0.5 release.

For running our tests (be warned that they are not complete and still work in progress) we recommend installing pytest into the conda environment and then executing

pytest 

at the root level of ptypy. On my machine (linux) and with the GitHub Actions CI runner those tests run through without any errors.

I hope this answers your questions and allows you to get started with ptypy!
Benedikt