AttributeError: 'NoneType' object has no attribute '__version__'

[ojc97]

SUMMARY

When using the community.postgresql.postgresql_user or community.postgresql.postgresql_schema ansible modules, I am facing a python stacktrace which has the error "AttributeError: 'NoneType' object has no attribute 'version'".

I have looked online and seen that this could be due to 'psycopg2' being missing but I have added an ansible step to ensure this is present.

Any help greatly appreciated.

ISSUE TYPE

• Bug Report

COMPONENT NAME

community.postgresql.postgresql_user
community.postgresql.postgresql_schema

ANSIBLE VERSION

ansible [core 2.12.4]
  config file = None
  configured module search path = ['/Users/olliecole/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /opt/homebrew/lib/python3.9/site-packages/ansible
  ansible collection location = /Users/olliecole/.ansible/collections:/usr/share/ansible/collections
  executable location = /opt/homebrew/bin/ansible
  python version = 3.9.12 (main, Mar 26 2022, 15:44:31) [Clang 13.1.6 (clang-1316.0.21.2)]
  jinja version = 3.1.1
  libyaml = True

COLLECTION VERSION

# /Users/olliecole/.ansible/collections/ansible_collections
Collection           Version
-------------------- -------
community.postgresql 2.1.3  

# /opt/homebrew/lib/python3.9/site-packages/ansible_collections
Collection           Version
-------------------- -------
community.postgresql 1.7.1  

CONFIGURATION

~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
(END)

OS / ENVIRONMENT

MacOS 12.3.1
Database is on AWS RDS Postgres.

STEPS TO REPRODUCE

---
- name: Create ansible_tooling user, set password, grant privs
  community.postgresql.postgresql_user:
    login_host: "{{ cloud_database_postgresql_fqdn }}"
    port: "{{ tooling_ansible_database_postgresql_port }}"
    login_user: "{{ cloud_database_postgresql_master_username }}"
    login_password: "{{ cloud_database_postgresql_master_password }}"
    name: "{{ tooling_ansible_database_postgresql_username }}"
    password: "{{ tf_tooling_ansible.outputs.tooling_ansible_database_postgresql_password }}"
    role_attr_flags: "{{ tooling_ansible_database_postgresql_role_attr_flags }}"

EXPECTED RESULTS

Playbook should run and configure users.

ACTUAL RESULTS

The full traceback is:
Traceback (most recent call last):
  File "/Users/olliecole/.ansible/tmp/ansible-tmp-1651226309.226697-22461-187613661793656/AnsiballZ_postgresql_user.py", line 107, in <module>
    _ansiballz_main()
  File "/Users/olliecole/.ansible/tmp/ansible-tmp-1651226309.226697-22461-187613661793656/AnsiballZ_postgresql_user.py", line 99, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/Users/olliecole/.ansible/tmp/ansible-tmp-1651226309.226697-22461-187613661793656/AnsiballZ_postgresql_user.py", line 47, in invoke_module
    runpy.run_module(mod_name='ansible_collections.community.postgresql.plugins.modules.postgresql_user', init_globals=dict(_module_fqn='ansible_collections.community.postgresql.plugins.modules.postgresql_user', _modlib_path=modlib_path),
  File "/opt/homebrew/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 210, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/opt/homebrew/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/opt/homebrew/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/var/folders/dr/2fw5_ncd7hn_x18fyfcnvq700000gn/T/ansible_community.postgresql.postgresql_user_payload_hsbp_1tn/ansible_community.postgresql.postgresql_user_payload.zip/ansible_collections/community/postgresql/plugins/modules/postgresql_user.py", line 1038, in <module>
  File "/var/folders/dr/2fw5_ncd7hn_x18fyfcnvq700000gn/T/ansible_community.postgresql.postgresql_user_payload_hsbp_1tn/ansible_community.postgresql.postgresql_user_payload.zip/ansible_collections/community/postgresql/plugins/modules/postgresql_user.py", line 954, in main
  File "/var/folders/dr/2fw5_ncd7hn_x18fyfcnvq700000gn/T/ansible_community.postgresql.postgresql_user_payload_hsbp_1tn/ansible_community.postgresql.postgresql_user_payload.zip/ansible_collections/community/postgresql/plugins/module_utils/postgres.py", line 192, in get_conn_params
AttributeError: 'NoneType' object has no attribute '__version__'
fatal: [localhost]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/Users/olliecole/.ansible/tmp/ansible-tmp-1651226309.226697-22461-187613661793656/AnsiballZ_postgresql_user.py\", line 107, in <module>\n    _ansiballz_main()\n  File \"/Users/olliecole/.ansible/tmp/ansible-tmp-1651226309.226697-22461-187613661793656/AnsiballZ_postgresql_user.py\", line 99, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/Users/olliecole/.ansible/tmp/ansible-tmp-1651226309.226697-22461-187613661793656/AnsiballZ_postgresql_user.py\", line 47, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.community.postgresql.plugins.modules.postgresql_user', init_globals=dict(_module_fqn='ansible_collections.community.postgresql.plugins.modules.postgresql_user', _modlib_path=modlib_path),\n  File \"/opt/homebrew/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py\", line 210, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/opt/homebrew/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py\", line 97, in _run_module_code\n    _run_code(code, mod_globals, init_globals,\n  File \"/opt/homebrew/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py\", line 87, in _run_code\n    exec(code, run_globals)\n  File \"/var/folders/dr/2fw5_ncd7hn_x18fyfcnvq700000gn/T/ansible_community.postgresql.postgresql_user_payload_hsbp_1tn/ansible_community.postgresql.postgresql_user_payload.zip/ansible_collections/community/postgresql/plugins/modules/postgresql_user.py\", line 1038, in <module>\n  File \"/var/folders/dr/2fw5_ncd7hn_x18fyfcnvq700000gn/T/ansible_community.postgresql.postgresql_user_payload_hsbp_1tn/ansible_community.postgresql.postgresql_user_payload.zip/ansible_collections/community/postgresql/plugins/modules/postgresql_user.py\", line 954, in main\n  File \"/var/folders/dr/2fw5_ncd7hn_x18fyfcnvq700000gn/T/ansible_community.postgresql.postgresql_user_payload_hsbp_1tn/ansible_community.postgresql.postgresql_user_payload.zip/ansible_collections/community/postgresql/plugins/module_utils/postgres.py\", line 192, in get_conn_params\nAttributeError: 'NoneType' object has no attribute '__version__'\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}

[Andersson007]

@ojc97 hello, welcome to the project and thank you for reporting the issue!

I think it relates to psycopg2 (missed or i don't know why the interpreter doesn't see it), could you please recheck? Open the interpreter Ansible use on the target machine and try to run import psycopg2.

The issue helps spot another logical bug @hunleyd @jchancojr:

• In plugins/module_utils/postgres.py, we have the ensure_required_libs function which is called in the connect_to_db function.
• In the reported modules, psycopg2 is invoked in the get_ conn_params function which also lives in plugins/module_utils/postgres.py
• The problem is that get_conn_params is invoked in the modules earlier than connect_to_db and ensure_required_libs in it resp. In sort, now we check required libs after we call it the first time...

That's why it reports that object is None @ojc97. It should report that the lib is missed.

So the solution can be, for example:

1. Import ensure_required_libs in modules
2. Invoke it there before get_conn_params in all of the modules
3. Remove ensure_required_libs from connect_to_db
   Feel free to suggest your solution.

Does anyone want to pick it up? Should be an easy fix. I'd be happy to help if there are any issues.
What do you folks think?

[ojc97]

@Andersson007 First of all thanks for your swift reply!!

I have already installed psycopg2. I will try and run import psycopg2 now and report back shortly.

[ojc97]

@Andersson007 I assume this is what you meant.

Doesn't work as expected though.

➜  ~ python3                    
Python 3.9.12 (main, Mar 26 2022, 15:44:31) 
[Clang 13.1.6 (clang-1316.0.21.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import psycopg2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/homebrew/lib/python3.9/site-packages/psycopg2/__init__.py", line 51, in <module>
    from psycopg2._psycopg import (                     # noqa
ImportError: dlopen(/opt/homebrew/lib/python3.9/site-packages/psycopg2/_psycopg.cpython-39-darwin.so, 0x0002): symbol not found in flat namespace '_PQbackendPID'
>>> 

[Andersson007]

@Andersson007 I assume this is what you meant.

@ojc97 yep
If i understand correctly your controller is on MacOS and you want your playbook to run on your controller and connect to the remote postgres via psycopg2, correct?
If yes, i don't see in your example the task was delegated to your controller, e.g. via delegate_to: localhost.
Also in this case you need to install psycopg2 on your controller as it would be like an Ansible target host.
Did you do that?

[Andersson007]

in case described above, that task that installs psycopg2 should be also with delegate_to: localhost so it will install in on your controller.
If you did that, there's homebrew mentioned, i have no experience in working with it. So there's probably issues in the local configuration. Maybe you could try to install psycopg2 manually.

[Andersson007]

https://stackoverflow.com/questions/67013694/django-unable-to-load-psycopg2-module-python-is-also-unable-to-load-it-even-th
people recommend to run pip install psycopg2-binary

[jchancojr]

Does anyone want to pick it up? Should be an easy fix. I'd be happy to help if there are any issues. What do you folks think?

I can pick this up @Andersson007! Should be able to get to it ~today or this weekend.

[Andersson007]

@jchancojr thanks! This week or next one would be imo fine:) No rush with this

[Andersson007]

@ojc97 when you investigate what's the reason and how to fix it, please let us know.
It can help other users with a similar case find a solution.
If you have any questions, feel free to ask.

[ojc97]

@Andersson007 I have been looking into the reasons regarding this today and i am yet to have an answer...

It appears that lots of people have problems with psycopg2 on mac os. Even more with M1 ARM Macs.

If you have any recommendations, id love to hear them.

So far i have reinstalled, used psycopg2-binary instead, added exports as suggested here and some others...

[Andersson007]

@ojc97 as a workaround you could try to:

1. take a look at pyenv or
2. install a virtual machine with a linux distro, say, fedora or ubuntu, and run your controller in there

[mschwrdtnr]

@Andersson007 @ojc97 I am facing the same issue on a current CentOS machine. I think I have everything necessary installed such as:

• python3-devel
• libpq-devel
• psycopg2

python3 -c "import psycopg2; print(psycopg2.__version__)"
2.9.3 (dt dec pq3 ext lo64)

pg_config --version
PostgreSQL 13.5

but it still don't work...