Skip to content

Commit 2485983

Browse files
jorgensdmichalhaberajhalegarth-wells
authored
dolfinx.fem.petsc.LinearProblem for nest and block systems (#3684)
* Add snes solver * Replace monolitic snes from test * Add nate to copyright * Add snes solvers for blocked problems * Add nest snes solver * Ruff formatting * Shorten description * Add preconditioner property * Apply suggestions from code review * Improve documentation to address reviewer comments * Fix documentation * Typo * Apply suggestions from code review * Unify docs * Ruff * Improve documentation of blocked problem. * Remove communicator from snesproblem * Move petsc imports into tests to avoid failure when petsc is not installed * Update naming scheme * Apply Michals suggestions Co-authored-by: Michal Habera <michal.habera@gmail.com> * Use typing protocol to remove inheritance as well as adding a helper function for extracting Jacobians * Add francesco's comments * Unify snes solver into one * Ruff formatting * Ruff + mypy * Remove error if not converged * Do not update the variable `u` used in the variational forms as part of `solve`. Let the user do this with the Problem-class function replace solution. Also expose the petsc objects in the solver through properties * Start prototyping functional interface * Make solvers work * Remove nonexistent classes * Ruff formatting * mypy + ruff * Fix import * Use public members * Remove snesproblem def * Add note * Fully functional * Remove protocol * Move import * Add docstrings * Use functools partial * Remove update solution * Move snes options to constructor * Rename data structures * Rename again * Rename * Move dfx.fem.Function<->PETSc.Vec methods to dolfinx.fem.petsc * Add example of direct usage with snes * Stricter tolerance * Set tolerance depending on eps * Ruff format * Insert initial guess for hand written solver * Move assembly type into dolfinx.fem.petsc * Rename "default"->"standard". Switch order in Vec->Function * Add prefix for snes options * Apply suggestions from code review * Add full type hints * default->standard for residual and jacobian helper function * Make _create_snes_matrices_and_vectors private * Update python/dolfinx/fem/petsc.py Co-authored-by: Garth N. Wells <gnw20@cam.ac.uk> * Apply suggestions from code review * More minor updates to adhere to reviewer comments * Merge main into snes solver. Add required ghost updates within F and J computations * Fix typehint * Update typesetting of singleton assemble_jacobian * Use try-except instead of if-else * Proposal for supporting blocked problems (not nest) in `LinearProblem`. * Add asserts for typehinting to demos * Add kind as input argument and allow for nest * Use same logic for both blocked and non-blocked problems * Test all use-cases * Centralize ghostupdate function * Simplify vector creation * add element dtypes * Ruff formatting * Use realtype * Use inner * Add kind which is nested list of types * Make ghostupdate private * Ruff format comment and add Preconditioner input to `LinearProblem` * Ruff format * Add entity maps as optional input to snessolver * Fix check * Various import cleanups * Remove unused code. * Unify assemble_residual (#3682) * Unify assemble_residual * Ruff format * Fix imports * Add back comma * Move assignment of block data out of assembler * Ignore type-hints * Switch order of operations * Ruff format * Add entity maps * Introduce major API changes and deprecations. * Improve documentation messages on deprecation. * Refactor in progress, tests not passing yet. * Update python/dolfinx/nls/petsc.py Co-authored-by: Jørgen Schartum Dokken <dokken92@gmail.com> * Update python/dolfinx/fem/petsc.py Co-authored-by: Jørgen Schartum Dokken <dokken92@gmail.com> * Fix tests. * Fix test * Fix Cahn-Hilliard - move to new NonlinearProblem class? * Fix ruff. * Refactor compute_jacobian * Ruff format. * Update petsc.py * Illustrate usage of LinearProblem with blocked matrices * Fix import * Consistency with LinearProblem class. * A few extra comments in test. * Consistency petsc_options with LinearProblem * Disable test * Use sequence consistently. * More specific. * Tidy up. * Loosen typing. * Leave that unknown * Fix typing. * More typing fixes. * Fixup comments and typing. * Remove duplicate classes --------- Co-authored-by: Michal Habera <michal.habera@gmail.com> Co-authored-by: Jack S. Hale <mail@jackhale.co.uk> Co-authored-by: Garth N. Wells <gnw20@cam.ac.uk>
1 parent 3be58ec commit 2485983

9 files changed

Lines changed: 307 additions & 102 deletions

File tree

python/demo/demo_axis.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -676,6 +676,7 @@ def create_eps_mu(pml, rho, eps_bkg, mu_bkg):
676676
},
677677
)
678678
Esh_m = problem.solve()
679+
assert isinstance(Esh_m, fem.Function)
679680
assert problem.solver.getConvergedReason() > 0, "Solver did not converge!"
680681

681682
# Scattered magnetic field

python/demo/demo_biharmonic.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -228,6 +228,7 @@
228228

229229
problem = LinearProblem(a, L, bcs=[bc], petsc_options={"ksp_type": "preonly", "pc_type": "lu"})
230230
uh = problem.solve()
231+
assert isinstance(uh, fem.Function)
231232

232233
# The solution can be written to a {py:class}`XDMFFile
233234
# <dolfinx.io.XDMFFile>` file visualization with ParaView or VisIt

python/demo/demo_navier-stokes.py

Lines changed: 34 additions & 59 deletions
Original file line numberDiff line numberDiff line change
@@ -181,7 +181,6 @@
181181

182182
import ufl
183183
from dolfinx import default_real_type, fem, io, mesh
184-
from dolfinx.fem.petsc import apply_lifting, assemble_matrix, assemble_vector, set_bc
185184

186185
try:
187186
from petsc4py import PETSc
@@ -195,6 +194,7 @@
195194
print("This demo requires petsc4py.")
196195
exit(0)
197196

197+
from dolfinx.fem.petsc import LinearProblem
198198

199199
if np.issubdtype(PETSc.ScalarType, np.complexfloating):
200200
print("Demo should only be executed with DOLFINx real mode")
@@ -307,7 +307,6 @@ def jump(phi, n):
307307
a -= ufl.inner(p, ufl.div(v)) * ufl.dx
308308
a -= ufl.inner(ufl.div(u), q) * ufl.dx
309309

310-
a_blocked = fem.form(ufl.extract_blocks(a))
311310

312311
f = fem.Function(W)
313312
u_D = fem.Function(V)
@@ -317,43 +316,39 @@ def jump(phi, n):
317316
+ (alpha / h) * ufl.inner(ufl.outer(u_D, n), ufl.outer(v, n)) * ufl.ds
318317
)
319318
L += ufl.inner(fem.Constant(msh, default_real_type(0.0)), q) * ufl.dx
320-
L_blocked = fem.form(ufl.extract_blocks(L))
321319

322320
# Boundary conditions
321+
msh.topology.create_connectivity(msh.topology.dim - 1, msh.topology.dim)
323322
boundary_facets = mesh.exterior_facet_indices(msh.topology)
324323
boundary_vel_dofs = fem.locate_dofs_topological(V, msh.topology.dim - 1, boundary_facets)
325324
bc_u = fem.dirichletbc(u_D, boundary_vel_dofs)
326325
bcs = [bc_u]
327326

328327

329328
# Assemble Stokes problem
330-
A = assemble_matrix(a_blocked, bcs=bcs)
331-
A.assemble()
332-
333-
b = assemble_vector(L_blocked, kind=PETSc.Vec.Type.MPI)
334-
bcs1 = fem.bcs_by_block(fem.extract_function_spaces(a_blocked, 1), bcs)
335-
apply_lifting(b, a_blocked, bcs=bcs1)
336-
b.ghostUpdate(addv=PETSc.InsertMode.ADD, mode=PETSc.ScatterMode.REVERSE)
337-
bcs0 = fem.bcs_by_block(fem.extract_function_spaces(L_blocked), bcs)
338-
set_bc(b, bcs0)
339-
340-
# Create and configure solver
341-
ksp = PETSc.KSP().create(msh.comm)
342-
ksp.setOperators(A)
343-
ksp.setType("preonly")
344-
ksp.getPC().setType("lu")
345-
ksp.getPC().setFactorSolverType("mumps")
346-
opts = PETSc.Options() # type: ignore
347-
opts["mat_mumps_icntl_14"] = 80 # Increase MUMPS working memory
348-
opts["mat_mumps_icntl_24"] = 1 # Option to support solving a singular matrix (pressure nullspace)
349-
opts["mat_mumps_icntl_25"] = 0 # Option to support solving a singular matrix (pressure nullspace)
350-
opts["ksp_error_if_not_converged"] = 1
351-
ksp.setFromOptions()
352-
353-
# Solve Stokes for initial condition
354-
x = A.createVecRight()
329+
solver_options = {
330+
"ksp_type": "preonly",
331+
"pc_type": "lu",
332+
"pc_factor_mat_solver_type": "mumps",
333+
"mat_mumps_icntl_14": 80, # Increase MUMPS working memory
334+
"mat_mumps_icntl_24": 1, # Option to support solving a singular matrix (pressure nullspace)
335+
"mat_mumps_icntl_25": 0, # Option to support solving a singular matrix (pressure nullspace)
336+
"ksp_error_if_not_converged": 1,
337+
}
338+
u_h = fem.Function(V)
339+
p_h = fem.Function(Q)
340+
p_h.name = "p"
341+
stokes_problem = LinearProblem(
342+
ufl.extract_blocks(a),
343+
ufl.extract_blocks(L),
344+
u=[u_h, p_h],
345+
bcs=bcs,
346+
kind="mpi",
347+
petsc_options=solver_options,
348+
)
349+
355350
try:
356-
ksp.solve(b, x)
351+
stokes_problem.solve()
357352
except PETSc.Error as e: # type: ignore
358353
if e.ierr == 92:
359354
print("The required PETSc solver/preconditioner is not available. Exiting.")
@@ -362,15 +357,6 @@ def jump(phi, n):
362357
else:
363358
raise e
364359

365-
# Split the solution
366-
u_h = fem.Function(V)
367-
p_h = fem.Function(Q)
368-
p_h.name = "p"
369-
offset = V.dofmap.index_map.size_local * V.dofmap.index_map_bs
370-
u_h.x.array[:offset] = x.array_r[:offset]
371-
u_h.x.scatter_forward()
372-
p_h.x.array[: (len(x.array_r) - offset)] = x.array_r[offset:]
373-
p_h.x.scatter_forward()
374360
# Subtract the average of the pressure since it is only determined up to
375361
# a constant
376362
p_h.x.array[:] -= domain_average(msh, p_h)
@@ -406,37 +392,26 @@ def jump(phi, n):
406392
+ ufl.inner((ufl.dot(u_n, n))("-") * u_uw, v("-")) * ufl.dS
407393
+ ufl.inner(ufl.dot(u_n, n) * lmbda * u, v) * ufl.ds
408394
)
409-
a_blocked = fem.form(ufl.extract_blocks(a))
410395

411396
L += (
412397
ufl.inner(u_n / delta_t, v) * ufl.dx
413398
- ufl.inner(ufl.dot(u_n, n) * (1 - lmbda) * u_D, v) * ufl.ds
414399
)
415-
L_blocked = fem.form(ufl.extract_blocks(L))
400+
401+
navier_stokes_problem = LinearProblem(
402+
ufl.extract_blocks(a),
403+
ufl.extract_blocks(L),
404+
u=[u_h, p_h],
405+
bcs=bcs,
406+
kind="mpi",
407+
petsc_options=solver_options,
408+
)
416409

417410
# Time stepping loop
418-
bcs1 = fem.bcs_by_block(fem.extract_function_spaces(a_blocked, 1), bcs)
419411
for n in range(num_time_steps):
420412
t += delta_t.value
421413

422-
A.zeroEntries()
423-
fem.petsc.assemble_matrix(A, a_blocked, bcs=bcs) # type: ignore
424-
A.assemble()
425-
426-
with b.localForm() as b_loc:
427-
b_loc.set(0)
428-
assemble_vector(b, L_blocked)
429-
apply_lifting(b, a_blocked, bcs=bcs1)
430-
b.ghostUpdate(addv=PETSc.InsertMode.ADD, mode=PETSc.ScatterMode.REVERSE)
431-
set_bc(b, bcs0)
432-
433-
# Compute solution
434-
ksp.solve(b, x)
435-
436-
u_h.x.array[:offset] = x.array_r[:offset]
437-
u_h.x.scatter_forward()
438-
p_h.x.array[: (len(x.array_r) - offset)] = x.array_r[offset:]
439-
p_h.x.scatter_forward()
414+
navier_stokes_problem.solve()
440415
p_h.x.array[:] -= domain_average(msh, p_h)
441416

442417
u_vis.interpolate(u_h)

python/demo/demo_pml.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -721,6 +721,7 @@ def create_eps_mu(
721721
},
722722
)
723723
Esh = problem.solve()
724+
assert isinstance(Esh, fem.Function)
724725
assert problem.solver.getConvergedReason() > 0, "Solver did not converge!"
725726
# -
726727

python/demo/demo_poisson.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -158,6 +158,7 @@
158158
# +
159159
problem = LinearProblem(a, L, bcs=[bc], petsc_options={"ksp_type": "preonly", "pc_type": "lu"})
160160
uh = problem.solve()
161+
assert isinstance(uh, fem.Function)
161162
# -
162163

163164
# The solution can be written to a {py:class}`XDMFFile

python/demo/demo_scattering_boundary_conditions.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -640,6 +640,7 @@ def curl_2d(f: fem.Function):
640640
gdim = mesh_data.mesh.geometry.dim
641641
V_dg = fem.functionspace(mesh_data.mesh, ("Discontinuous Lagrange", degree, (gdim,)))
642642
Esh_dg = fem.Function(V_dg)
643+
assert isinstance(Esh, fem.Function)
643644
Esh_dg.interpolate(Esh)
644645

645646
with io.VTXWriter(mesh_data.mesh.comm, "Esh.bp", Esh_dg) as vtx:

0 commit comments

Comments
 (0)