Skip to content

Conversation

s3alfisc
Copy link
Member

@s3alfisc s3alfisc commented Mar 8, 2025

Hi @elchorro , here is a first attempt at creating a minimal environment.

It allows users to access all estimation function and methods (feols, fepois, feglm etc) and a range of post-processing methods (tidy, summary, plotting via plt).

It drops joblib, great_tables, tqdm and tabulate from the main env and moves it into a secondary environment full which will be installable by typing pip install pyfixest[full].

In the new "core" env

  • tqdm is replaced by a dummy function
  • joblib only executed when installed
  • calling GT produces an ImportError
  • calling tabulate produces an ImportError

What still would have to be done:

  • We should add logic to the CI workflow to run tests in the "core environment" vs the base environment, to make sure things work well without the "optional" dependencies.
  • We have to update the readme.
  • We should decide if the "minimal" environment should be the "default" environment that gets installed by running pip install pyfixest. I assume that it would be more user friendly to install etable etc by default and require users with other specs to specifically install the "core" env? I would be curious about @dsliwka's and @juanitorduz's thoughts on this?
  • Should we make seaborn an optional dependency? If no, we could replace the seaborn dependency by a matplotlib dependecy (the only piece of code that relies on seaborn is the plotting method for randomization inference which should be an easy fix.
  • We could even make narwhals optional, in which case we'd have to thrown an error wherever it converts non pd.DataFrame's for us (happens in maybe 3 scripts).
  • Clean up some code duplicates (i.e. the try from tqdm import tqdm parts).

If you have some time to polish this PR, it would be much appreciated @elchorro ! =)

@s3alfisc s3alfisc linked an issue Mar 8, 2025 that may be closed by this pull request
@s3alfisc s3alfisc added the dependencies Pull requests that update a dependency file label Mar 8, 2025
Copy link

codecov bot commented Mar 8, 2025

Codecov Report

Attention: Patch coverage is 34.78261% with 30 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
pyfixest/estimation/ritest.py 25.00% 15 Missing ⚠️
pyfixest/report/summarize.py 0.00% 8 Missing ⚠️
pyfixest/estimation/decomposition.py 53.84% 6 Missing ⚠️
pyfixest/estimation/feols_compressed_.py 80.00% 1 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (8476d50) and HEAD (c086b45). Click for more details.

HEAD has 4 uploads less than BASE
Flag BASE (8476d50) HEAD (c086b45)
core-tests 2 0
tests-vs-r 2 1
tests-extended 1 0
Flag Coverage Δ
core-tests ?
tests-extended ?
tests-vs-r 48.00% <34.78%> (-0.10%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
pyfixest/estimation/feols_compressed_.py 76.57% <80.00%> (-7.06%) ⬇️
pyfixest/estimation/decomposition.py 24.11% <53.84%> (-74.01%) ⬇️
pyfixest/report/summarize.py 4.41% <0.00%> (-83.33%) ⬇️
pyfixest/estimation/ritest.py 20.66% <25.00%> (-45.30%) ⬇️

... and 29 files with indirect coverage changes

🚀 New features to boost your workflow:
  • Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@s3alfisc
Copy link
Member Author

s3alfisc commented Mar 9, 2025

I adjusted the pyproject toml so that the default env has a richer dependency than minimal. Users who run pip install pyfixest install gt, tabulate, etc by default; users who run pip install pyfixest[minimal] will get an env with only the core dependency set.

(pyfixest) PS C:\Users\alexa\Documents\pyfixest> pixi list -x                                               
Package       Version  Build               Size        Kind   Source                                        
formulaic     1.1.1                        371.3 KiB   pypi   formulaic-1.1.1-py3-none-any.whl              
great_tables  0.16.1                       6.4 MiB     pypi   great_tables-0.16.1-py3-none-any.whl
joblib        1.4.2                        886.2 KiB   pypi   joblib-1.4.2-py3-none-any.whl
narwhals      1.29.1                       1.5 MiB     pypi   narwhals-1.29.1-py3-none-any.whl
numba         0.61.0                       11.7 MiB    pypi   numba-0.61.0-cp312-cp312-win_amd64.whl        
numpy         2.1.3                        40.2 MiB    pypi   numpy-2.1.3-cp312-cp312-win_amd64.whl
pandas        2.2.3                        37.7 MiB    pypi   pandas-2.2.3-cp312-cp312-win_amd64.whl        
pyfixest      0.28.0                                   pypi    (editable)
python        3.12.8   h3f84c4b_1_cpython  15.1 MiB    conda  python
scipy         1.15.2                       115.1 MiB   pypi   scipy-1.15.2-cp312-cp312-win_amd64.whl        
seaborn       0.13.2                       1018.2 KiB  pypi   seaborn-0.13.2-py3-none-any.whl
tabulate      0.9.0                        128.4 KiB   pypi   tabulate-0.9.0-py3-none-any.whl
tqdm          4.67.1                       220.5 KiB   pypi   tqdm-4.67.1-py3-none-any.whl
(pyfixest) PS C:\Users\alexa\Documents\pyfixest> pixi shell --environment minimal
(pyfixest:minimal) PS C:\Users\alexa\Documents\pyfixest> pixi list -x
Environment: minimal
Package    Version  Build               Size        Kind   Source
formulaic  1.1.1                        371.3 KiB   pypi   formulaic-1.1.1-py3-none-any.whl
narwhals   1.29.1                       1.5 MiB     pypi   narwhals-1.29.1-py3-none-any.whl
numba      0.61.0                       11.7 MiB    pypi   numba-0.61.0-cp312-cp312-win_amd64.whl
numpy      2.1.3                        40.2 MiB    pypi   numpy-2.1.3-cp312-cp312-win_amd64.whl
pandas     2.2.3                        37.7 MiB    pypi   pandas-2.2.3-cp312-cp312-win_amd64.whl
pyfixest   0.28.0                                   pypi    (editable)
python     3.12.8   h3f84c4b_1_cpython  15.1 MiB    conda  python
scipy      1.15.2                       115.1 MiB   pypi   scipy-1.15.2-cp312-cp312-win_amd64.whl
seaborn    0.13.2                       1018.2 KiB  pypi   seaborn-0.13.2-py3-none-any.whl

I guess we should still check with the pixi team if this is "supported" behavior or a non-supported hack?

Comment on lines 9 to 16

try:
from tqdm import tqdm
except ImportError:

def tqdm(iterable, *args, **kwargs):
"Define a dummy tqdm function."
return iterable
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is being repeated a lot ... why don't we let tqdm be a core dep? It seems very lightweight

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same with joblib, actually - it has no other dependencies except for python . More or less the motivation of this PR is to make the package so lightweight that it can easily be installed in closed environments (with no access to pypi / conda). My prior would be that both joblib and tqdm should actually be installed on every system that runs Python, though I'm not 100% sure.

I am more or less gravitating towards having a default / core env that includes joblib, tqdm, GT etc that users install with pip install pyfixest and then a minimal but still functional env for those who can really only work with numpy, scipy, etc. Would be your suggestion to keep tqdm in the "minimal" dependency set @juanitorduz?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add minimal environment option
2 participants