feature: IMDReader Integration #4923
Conversation
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hello there first time contributor! Welcome to the MDAnalysis community! We ask that all contributors abide by our Code of Conduct and that first time contributors introduce themselves on GitHub Discussions so we can get to know you. You can learn more about participating here. Please also add yourself to package/AUTHORS as part of this PR.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #4923 +/- ##
===========================================
- Coverage 93.86% 85.89% -7.98%
===========================================
Files 179 180 +1
Lines 22249 22422 +173
Branches 3161 3186 +25
===========================================
- Hits 20885 19260 -1625
- Misses 902 2704 +1802
+ Partials 462 458 -4 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Your thoughts on this are appreciated - @orbeckst |
There was a problem hiding this comment.
Thanks for the initial PR.
- The first big step is to get the tests running properly so that the CI uses an imdclient without IMDReader. Otherwise we are not sure we're testing the code here.
- Minor initial comments while I skimmed.
- Simple thing: run
blackover all files to get the formatting and ordering of imports right
|
I set the PR to Work in progress for the time being, just to indicate that we're not yet at the stage where the CI is working. Once the tests run properly, we can update the status. Obviously, this shouldn't discourage anyone from contributing and commenting. |
There was a problem hiding this comment.
I read through and nothing jumped out at me that wasn't already mentioned, except that I didn't see these changes in the documentation that was linked. The following needs to be added:
Add doc/sphinx/source/documentation_pages/coordinates/IMD.rst
.. automodule:: MDAnalysis.coordinates.IMD
doc/sphinx/source/documentation_pages/coordinate_modules.rst
coordinates/IMD
doc/sphinx/source/documentation_pages/references.rst
If you use IMD capability...
|
@amruthesht do we have a plan for pushing ahead with suggested changes? |
Working on the imdclient repository right now. I had a few small issues there with using the |
1. Moved `parse_host_port` to `IMD.py` and deleted `util.py` 2. Cleaned up `test_imd.py` - changes to `assert_*` functions and simplified non-applicable test to pass automatically
|
@amruthesht kicking CI here. |
|
@yuxuanzhuang would you be able to have a run through? |
|
I will have another look at the PR this week. One immediate thing is to add documentation. Please refer to the comments below.
|
|
Need to figure out tests also @amruthesht |
| ) | ||
|
|
||
|
|
||
| class IMDModuleStateManager: |
There was a problem hiding this comment.
There may well be a neater way to do this using mock/monkeypatch without a custom class.
However, this one is now working locally where the other approach failed.
There was a problem hiding this comment.
Yeah, my friend ChatGPT proposed to use monkeypatch, but I didn't manage to make it work (seems like you have a better friend :)
There was a problem hiding this comment.
I used cursor where I can choose my friends. Also, I first had to convince it that using subprocess was too ugly...
There was a problem hiding this comment.
I implemented a simpler approach using monkeypatch without using IMDModuleStateManager custom class, with some help from a freind :)
@orbeckst, @yuxuanzhuang - Please have a look when you can, thank you. If it doesn't seem good, please feel free to revert the commit
- testing imports of IMD/imdclient module is now isolated from other tests - testing different versions of imdclient and IMD.HAS_IMDCLIENT is now done in separate tests
There was a problem hiding this comment.
Coverage is now good and all new functionality seems to be covered.
All tests are passing.
Great work @amruthesht , @ljwoods2 and everyone else!
🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉
|
@yuxuanzhuang @IAlibay could you please check if your comments have been addressed and if that's the case, approve the PR? |
There was a problem hiding this comment.
LGTM! Well done everyone (particularly @amruthesht and @ljwoods2)
| else: | ||
| return False | ||
|
|
||
| def close(self): |
There was a problem hiding this comment.
Above it says: \
- No independent copies: Cannot create separate reader instances for the same stream
Does that mean that if the IMDReader is not closed gracefully with this function that it will stay active and prevent another connection to the stream? I ask because this can be an issue with Jupyter notebooks where people are in the habit of restarting the kernel which will not close the IMDReader gracefully.
This is an active issue I have with python interfacing with postgres
There was a problem hiding this comment.
Yes, the socket connection would saty active unless terminated properly.
The IMDReader does have automatic cleanup - it inherits a del method from ReaderBase that calls close(), which in turn calls self._imdclient.stop() to properly shut down the connection.
Jupyter kernel restarts are the exception. it is safer to use a try/excpet block when using a Jupyter notebook.
Side Note: But technically it is possible to form multiple conenctions to the same port address. This is however limited by the host. Currently only NAMD supports this. But each connection has its seperate socket and stream of data which would need to be processed sperately by each client object seperately. The distinction is that these data streams may not always necessarily have the same data, depending on how the host has been configured. So, one can't make an independent copy on the same stream of data but can open a new connection while an existing connection is present (at least in NAMD).
There was a problem hiding this comment.
Add a note to the docs on best practices for using IMDReader in notebooks.
Regarding connecting multiple streams to one MD engine port, mention that the behavior is MD engine implementation dependent (some may allow it, others may fail).
1. API doc on multiple connections 2. Jupyter usage of `close()` for secure closure of connections
Replace `IMDModuleStateManager` with standard pytest patterns
Fixes #4827
This draft PR addresses the feature request discussed in #4827.
Note:
The
IMDReaderfeature which was previously a part of theimdclientpackage has been moved intoMDAnalysisbelow. Any other modules have been in retained inimdclient, which has been added as an optional dependency here. We are currently in the process of splitting theimdclientpackage as mentioned above. (Issue, PR)Major changes made in this Pull Request:
IMDReader, other associated base classes and a utility function were added to coordinates in the main package.test-imd.py*.yamlfilesimdclientwas added as an optional dependencyPR Checklist
package/CHANGELOGfile updated?package/AUTHORS? (If it is not, add it!)Developers Certificate of Origin
I certify that I can submit this code contribution as described in the Developer Certificate of Origin, under the MDAnalysis LICENSE.
📚 Documentation preview 📚: https://mdanalysis--4923.org.readthedocs.build/en/4923/