Support loading Python agent through sw-python CLI

[Superskyyy]

This PR adds functionality of non-intrusive integration CLI for the Python agent.

Though the code is completed, as a dangerous feature that modifies Python bootstrap I will need to test it a bit more before it's ready for review.

closes apache/skywalking#7381

[kezhenxu94]

Looking at the docs just makes me feel exciting about this new feature that I've been planning for so long, and I believe this feature will be the last piece before our 1.0 GA. Look forward to this feature and our 1.0 GA version, soon!

[kezhenxu94]

Let me review and try this new CLI tomorrow 🤩

[Superskyyy]

Looking at the docs just makes me feel exciting about this new feature that I've been planning for so long, and I believe this feature will be the last piece before our 1.0 GA. Look forward to this feature and our 1.0 GA version, soon!

That's what I thought! This feature will ease a lot of work in deployment.

Let me review and try this new CLI tomorrow 🤩

I also need to run it more in a Linux env tmr, I'm struggling with my WSL and Windows doesn't communicate to each other properly over localhost... So my local test coverage is quite limited for now (not able to deploy gunicorn on windows etc.).

After I verify most cases, I could use some help testing this CLI on different OS and shells and see if it breaks for some corner cases.

[wu-sheng]

Use GHA to verify running in various env.

[kezhenxu94]

Hi @Superskyyy , I just test (on MacOS) and review the codes, and left some comments.

Since you did a lot of test cases, it would be perfect if you can add those cases into our test matrix in GitHub Actions, if you find any trouble in adding that, ping me at anytime, I can offer my help!

[kezhenxu94]

One thing we should note when implementing this is that we've been trying to minimize our dependencies as an agent, so we'd better choose a reasonable file format that Python built-in libs can parse, instead of bringing another library, to avoid version conflicts with user applications, yaml is a common case where users may already use and we should not require dependency upgrade or downgrade.

[Superskyyy]

Hi @Superskyyy , I just test (on MacOS) and review the codes, and left some comments.

Since you did a lot of test cases, it would be perfect if you can add those cases into our test matrix in GitHub Actions, if you find any trouble in adding that, ping me at anytime, I can offer my help!

I'm actually thinking what if I change/ duplicate our existing plugin unit tests to use the sw-python command in docker-compose command, instead of calling agent in those consumer providers, that should work.

Idk if we need to keep the old tests that directly call agent.start(), just that having two versions doing essentially the same thing will double the GHA running time, which seems awfully long.

[kezhenxu94]

I'm actually thinking what if I change/ duplicate our existing plugin unit tests to use the sw-python command in docker-compose command, instead of calling agent in those consumer providers, that should work.

Idk if we need to keep the old tests that directly call agent.start(), just that having two versions doing essentially the same thing will double the GHA running time, which seems awfully long.

What I think is to replace all existing tests with the sw-python as we will recommend users to adopt the later method after it gets merged, we don't need to duplicate the tests.

[kezhenxu94]

I'm actually thinking what if I change/ duplicate our existing plugin unit tests to use the sw-python command in docker-compose command, instead of calling agent in those consumer providers, that should work.
Idk if we need to keep the old tests that directly call agent.start(), just that having two versions doing essentially the same thing will double the GHA running time, which seems awfully long.

What I think is to replace all existing tests with the sw-python as we will recommend users to adapt the later method after it gets merged, we don't need to duplicate the tests.

That could be done in following pull requests though.

[kezhenxu94]

Thank you @Superskyyy very much for doing this!! 🎉