Add mysql clear password auth

[silverbullettruck2001]

Adding upon @rathboma's changes to implement mysql_clear_password auth plugin

[sidorares]

There is a fix for failing tests in b96364e - either cherry-pick into your branch or wait for me to merge fix into master and rebase on top

[sidorares]

hey @silverbullettruck2001 do you know what's the easiest way to test this?
I'm trying to script initial setup and making CREATE USER testuser IDENTIFIED WITH mysql_clear_password BY 'test'

results in sqlMessage: "Plugin 'mysql_clear_password' is not loaded" error. My server is docker run -it -p 3306:3306 -e MYSQL_ROOT_PASSWORD=test -e LIBMYSQL_ENABLE_CLEARTEXT_PLUGIN=Y mysql:8 ( same result with and without LIBMYSQL_ENABLE_CLEARTEXT_PLUGIN=Y )

[silverbullettruck2001]

@sidorares can you try this instead CREATE USER testuser IDENTIFIED BY 'mysql_clear_password' or try

docker run -it -p 3306:3306 -e MYSQL_ROOT_PASSWORD=test -e LIBMYSQL_ENABLE_CLEARTEXT_PLUGIN=Y mysql:8 mysqld --default-authentication-plugin=mysql_clear_password

[silverbullettruck2001]

@sidorares did you make any progress with getting the tests for this working?

[sidorares]

@silverbullettruck2001 your docker command stoped with the error Can't start server: Invalid value for --default-authentication-plugin when I tied it, unfortunately didn't have time to progress past that. I really want to try at least simple successful login locally using AuthSwitch to mysql_clear_password before merging the PR. The code looks OK to me but want to test it manually first

[silverbullettruck2001]

@sidorares I did some digging and some testing. What I found out was this. Based on the mysql documentation there is no server side plugin that needs to be loaded for this to work. It's simply a way to invoke the client side connection so that it sends the password in clear text. It's up to the server side plugin (whichever one you pick) to be able to consume the password in clear text. The environment variable that you are setting only specifies that all client connections will use clear text. So I used the below command to start my container:

docker run -it -p 3306:3306 -e MYSQL_ROOT_PASSWORD=test -e LIBMYSQL_ENABLE_CLEARTEXT_PLUGIN=Y mysql:8

Then you just need to specify the --enable-cleartext-plugin in your connection parameters as such:

mysql --host=localhost --port=3306 --user=root --password=test --enable-cleartext-plugin

Afterwards the connection is established (by sending the password in clear text) you can create a user like you would any other user. You do not have to specify the mysql_clear_password in the IDENTIFIED BY as it is not used that way, it simply is just a way to tell the client to send the specified password in clear text, instead of hashing it.

Here's the documentation I found on this (in paragraph 3):
mysql Doc

[rathboma]

@sidorares have you considered using test containers for these types of tests? - https://github.com/testcontainers/testcontainers-node

I use them in Beekeeper Studio to do integration tests - defining the whole thing in code, runs great on github actions.

Here's the MySQL example from my codebase:
https://github.com/beekeeper-studio/beekeeper-studio/blob/master/apps/studio/tests/integration/lib/db/clients/mysql.spec.js

I know right now you use Docker and have an action per-database type, not sure if this would simplify that work, but it does make it easy to do stuff like looping through several variations of a database with different flags and running the same tests. Like for PSQL I run several versions with a simple loop

[sidorares]

interesting @rathboma, I never tried that. How does it work under the hood and what is the difference with "docker run mysql" like we have in ci workflow?

node-mysql2/.github/workflows/ci-linux.yml

Line 69 in dbb344e

run: docker run -d -e MYSQL_ALLOW_EMPTY_PASSWORD=1 -e MYSQL_ROOT_PASSWORD=${{ env.MYSQL_PASSWORD }} -e MYSQL_DATABASE=${{ env.MYSQL_DATABASE }} -v $PWD/mysqldata:/var/lib/mysql/ -v $PWD/examples/custom-conf:/etc/mysql/conf.d -v $PWD/examples/ssl/certs:/certs -p ${{ env.MYSQL_PORT }}:3306 ${{ matrix.mysql-version }}

[sidorares]

@silverbullettruck2001 I'll need to investigate what mysql cli is doing when --enable-cleartext-plugin, I suspect when this switch is set it might use mysql_clear_password as a plugin in the initial client hello packet. Currently mysql2 always sends hello with mysql_native_password plugin name and only switches to another plugin if requested by server AuthSwitchRequest command

In order to test above flow I need to add this functionality ( default plugin to be used initially )

[rathboma]

@sidorares mostly it allows for easier scripting of containers as part of the javascript, and provides an easy api for doing stuff like checking if the container is alive before running tests. In development it can also keep containers around for a little while automatically so repeated runs of tests are faster.

You already have it set-up with CI, so certainly not urgent, but worth considering if you ever do a big rewrite!

[silverbullettruck2001]

@sidorares is there anything I can do to help with this? I am limited but would be willing to discuss on Slack or discord or whatever to keep this moving forward so I can make sure this gets incorporated so I can continue using this tool.

[silverbullettruck2001]

@sidorares Just following up on this. Is there any way to finalize this?

[sidorares]

Notes for me on how to configure mysql server to use cleartext auth plugin:

1. install mysql-community-test-debug package

> docker exec -it mysql_container /bin/bash
# apt-get update
# apt-get install mysql-community-test-debug

2. enable plugin via sql query:

INSTALL PLUGIN cleartext_plugin_server SONAME 'auth_test_plugin.so'

3. add user

CREATE USER uplain@localhost IDENTIFIED WITH 'cleartext_plugin_server' AS 'cleartext_test'

@silverbullettruck2001 @rathboma I still struggle to connect after that point. Did you have a chance to connect to a real ( or mock ) mysql server using cleartext plugin? What are your steps?

For some reason mysql2 client ( mysql.createConnection({ user: 'uplain', password: 'cleartext_test' }) ) still gets caching_sha2_password plugin in the first AUTH_SWITCH_REQUEST packet

[silverbullettruck2001]

@sidorares In my specific use case, I am connecting to an AWS RDS instance that uses MySQL and IAM authentication. I provided links below that show how to set this up on AWS, if that helps.

Also, the way I have interpreted the mysql_clear_password plugin is that it client side only. So all it is doing is instructing the client connection to not encrypt/hash the password. If that is the case, then really, you could just set it up with your above steps and find a way to capture the traffic when you make a connection request and inspect it to ensure that the packets are being sent in clear text instead of being encrypted.

AWS IAM Authentication

[silverbullettruck2001]

Also, I believe @tkchk said he had this working as well and had a test script that he provided so maybe he has some steps to use to setup the database? @tkchk can you assist?

[tkchk]

Sure. have a look at my last comment in #1403

[silverbullettruck2001]

@tkchk could you provide some steps on how you configured your mysql database and user to verify that it authenticated correctly?

[tkchk]

@silverbullettruck2001 So, the setup is a bit tricky.
Usually, mysql requires mysql_native_password plugin, since it's using hashes and is secure. This is default
In order to create user with mysql_clear_password you have to use plugin with external authentication, like auth_pam_compat that will authenticate user against linux system (not the mysql internal user table mysql.user)
I did all these tests with percona 5.7 cluster on Debian 10 system.

root@server /etc/pam.d # dpkg -l | grep percona-xtradb-cluster-full-57
ii  percona-xtradb-cluster-full-57          5.7.38-31.59-1.buster         amd64        Percona XtraDB Cluster with Galera
root@server /etc/pam.d # cat /etc/os-release | grep PRETTY_NAME
PRETTY_NAME="Debian GNU/Linux 10 (buster)"

1. Create some test user on linux(!) system. adduser test and provide some simple password.
2. Install plugin inside mysql. mysql> INSTALL PLUGIN auth_pam_compat SONAME 'auth_pam_compat.so';
3. Create several pam files with this content:
4. /etc/pam.d/mysql-unix

#%PAM-1.0
auth            include         password-auth
account		include		password-auth

5. /etc/pam.d/password-auth

@include common-auth
@include common-account
@include common-session-noninteractive

6. Create user inside mysql without password and give some privileges to it.

mysql> CREATE USER 'test'@'localhost' IDENTIFIED WITH 'auth_pam_compat' AS 'mysql-unix';
mysql> GRANT ALL PRIVILEGES ON *.* TO 'test'@'localhost';
mysql> FLUSH PRIVILEGES;

7. Add mysql to shadow group. Database has to read /etc/shadow somehow.

usermod -a -g shadow mysql

8. Restart the database. systemctl restart mysql
9. Try logging in

mysql -utest -ppassword -hlocalhost --enable-cleartext-plugin

This instruction may be incomplete though.
If you have logged in, you can test the same credentials in your nodejs script.

[sidorares]

I'm very tempted to merge this PR without any integration or unit tests but might give it another try over the weekend
Plan is:

• integration: compile simple auth plugin ( https://dev.mysql.com/doc/extending-mysql/8.0/en/compiling-plugin-libraries.html , https://github.com/mysql/mysql-server/blob/8.0/plugin/auth/test_plugin.cc ), add compiled x86_64 test_plugin.so to the repo, copy it to a mysql server container at the beginning of a test workflow, configure a user with this plugin as a default auth
• semi-integration - mock server side of auth switch and test against it

[silverbullettruck2001]

@sidorares I think the risk of doing this is relatively low. I support you merging this.

[sidorares]

relatively low. I support you merging this.

its more about accidentally breaking this feature by some other future change if its not covered by tests
To me it looks safe re not breaking other existing functionality right now

[silverbullettruck2001]

@sidorares any further thoughts on how to get this merged?

[sidorares]

@silverbullettruck2001 merged. I'll track e2e test via real docker server or mock server separately

I'll try to setup https://github.com/googleapis/release-please here some time later today and hopefully will have releases more often