Refactor LA into native and PETSc part and add assigners

[jhale]

Supersedes #3652 which made it apparent that dolfinx.la should have a dedicated dolfinx.la.petsc module.

This PR introduces API changes:

dolfinx.la.create_petsc_vector -> dolfinx.la.petsc.create_vector
dolfinx.la.create_petsc_vector_wrap -> dolfinx.la.petsc.create_vector_wrap

Note that dolfinx.fem.create_vector is now a very light wrapper over dolfinx.la.create_vector, perhaps bringing into question the need for the former.

[jorgensd]

Couldn’t we place all PETSC related structures in dolfinx.petsc instead of having spread namespaces as nls,fem,la that has all to be explicitly imported in DOLFINx?

[jhale]

Couldn’t we place all PETSC related structures in dolfinx.petsc instead of having spread namespaces as nls,fem,la that has all to be explicitly imported in DOLFINx?

Well, yes, possibly, but not in this PR. But it's a huge API change. Let's discuss it on Slack.

[michalhabera]

The separation into fem.petsc and la.petsc reads very nice! And also follows the C++ structure.

Why are all these helpers (la.petsc.create_vector_wrap, la.petsc.create_vector, la.petsc.assign, fem.petsc.assign, ...) written in Python only? Is the argument that C++ users would not need/want such helpers?

[jhale]

Why are all these helpers (la.petsc.create_vector_wrap, la.petsc.create_vector, la.petsc.assign, fem.petsc.assign, ...) written in Python only? Is the argument that C++ users would not need/want such helpers?

I discussed this with @garth-wells - providing similar functionality for C++ users would be beneficial as interfacing SNES in particular is apparently "very hard". However, it's not clear we would want to write helpers in C++ using PETSc and then expose to Python, or just provide two good designs that suits C++ and Python via PETSc C and petsc4py, respectively.

[garth-wells]

Why are all these helpers (la.petsc.create_vector_wrap, la.petsc.create_vector, la.petsc.assign, fem.petsc.assign, ...) written in Python only? Is the argument that C++ users would not need/want such helpers?

I discussed this with @garth-wells - providing similar functionality for C++ users would be beneficial as interfacing SNES in particular is apparently "very hard". However, it's not clear we would want to write helpers in C++ using PETSc and then expose to Python, or just provide two good designs that suits C++ and Python via PETSc C and petsc4py, respectively.

There is a helper at

dolfinx/cpp/dolfinx/la/petsc.h

Line 107 in 43dafba

std::vector<std::vector<PetscScalar>> get_local_vectors(

.

More C++ 'helpers' could be helpful, but we should do it with a purpose, e.g. when adding SNES helpers, to get it right rather than adding it for hypothetical reasons.