Missing support for variable interpolation in .env files

[hashkool]

podman-compose deviates from docker-compose in

• that it does not interpolate strings in .env files, and
• that it does not know about host env vars in .env (like $HOME, $PWD etc)

To Reproduce

FILE: .env

A=bar
B=foo$A
C=$HOME/foo
D="${XDG_DATA_HOME:-$HOME/.local/share}/blabla"

FILE: docker-compose.yml

version: "4"
services:
    variables:
        image: busybox
        command: ["/bin/busybox", "sh", "-c", "export | grep EXAMPLE"]
        environment:
            EXAMPLE_SIMPLE: "$B"
            EXAMPLE_HOSTVAR_SIMPLE: $C
            EXAMPLE_HOSTVAR_COMPLEX: $D

Commands

$ podman-compose config

Expected behavior
All the variables should be substituted as you would expect. Note that docker-compose config just works as expected.
Also no need to explicitly pass around variables with -e from when invoking docker-compose.

Actual behavior
Actual behaviour is that $B evaluates to literally foo$A, meaning it doesn't interpolate its value.
$C and $D results in errors, as podman-compose doesn't know about $HOME at all.

Output

$ podman-compose version
podman-compose version: 1.0.6
['podman', '--version', '']
using podman version: 4.5.0
podman-compose version 1.0.6
podman --version 
podman version 4.5.0

Environment:
Linux 6.1.31-2

Additional context

Current tests in repo do not test equality of output between docker-compose and podman-compose. That would possibly make your life easier to keep feature parity.

[herzenschein]

I can confirm this. It's easy to reproduce with the docker.io/postgres image and a sample env file like this:

DB_NAME=databasename
DB_USER=databaseuser
DB_PASS=databasepassword

POSTGRES_DB=${DB_NAME}
POSTGRES_USER=${DB_USER}
POSTGRES_PASSWORD=${DB_PASS}

Something like this causes postgres to fail with the following error in podman-compose logs:

FATAL:  invalid character in extension owner: must not contain any of ""$'\"

This issue also seems very similar to #721, they might be duplicates.

Meanwhile, using the following in the compose.yaml with --env-file works just fine.

    environment:
      POSTGRES_DB: ${DB_NAME}
      POSTGRES_USER: ${DB_USER}
      POSTGRES_PASSWORD: ${DB_PASS}

[Franbo]

I'm also seeing that complex variables from the .env file like ${SAMPLE:-$PWD} also do not get the $PWD portion resolved in the yaml compose file, such as for volume mounting.

[meanderix]

AFAICT the .env-file is processed in two ways:

1. Using the python-dotenv-library (which supports expansions within the .env-file; i.e. any expansion can be reused within compose.yaml).
2. As an --env-file argument passed to podman when starting the pods (this will not resolve any inline expansions within the .env-file, so a work-around is to manually do this in compose.yaml as stated above).

[cyqsimon]

I'm also seeing that complex variables from the .env file like ${SAMPLE:-$PWD} also do not get the $PWD portion resolved in the yaml compose file, such as for volume mounting.

Ran into this a moment ago. Didn't run into exactly this, but the equivalent thing in compose.yml where environment variable replacement is already supported. Wanted to write something like this - ${OVERRIDE_DIR:-${HOME}/some/dir}:/other/dir:ro,z but it was only interpolated once into - ${OVERRIDE_DIR:-/home/me/some/dir}:/other/dir:ro,z.

I took a look at the source code, and it seems like there is no recursion to handle nested interpolation (which is part of the spec):

podman-compose/podman_compose.py

Lines 267 to 282 in d704622

	elif is_str(value):
	def convert(m):
	if m.group("escaped") is not None:
	return "$"
	name = m.group("named") or m.group("braced")
	value = subs_dict.get(name)
	if value == "" and m.group("empty"):
	value = None
	if value is not None:
	return str(value)
	if m.group("err") is not None:
	raise RuntimeError(m.group("err"))
	return m.group("default") or ""
	value = var_re.sub(convert, value)

Should be a very easy fix though; will submit a PR in these few days.

Edit 1: After some initial experimentation, it seems like I was overly optimistic in assessing the difficulty of this task. It's pretty evident to me now that it would be practically impossible to implement a reliable nested interpolation parser without bringing in a proper lexer dependency (or worse, raw dogging one ourselves). Regex simply isn't cutting it. I'll continue to work on it but don't expect anything quick.

Edit 2: After looking away for a few days and coming back in a less sleep-deprived state, I am finding the problem slightly less intimidating but still quite tricky. I think I will be able to make a regex-based POC but there are some unresolved questions, mainly regarding ambiguities in the interpolation spec. We can discuss them in the PR.

Edit 3: While writing the PR comments, the more I wrote the more problems I found, and it is quickly spiralling out of control. Recursive string replacement is just not going to be nearly good enough, because fundamentally we cannot distinguish between a lexical token (e.g. $$) and a concatenated string (e.g. $$${FOO} where ${FOO} is $ -> $$). And if we start doing structured replacements, we might as well be writing an AST (which I'm certainly not doing in Python). So I guess I won't be submitting a PR. Sorry. ¯\_(ツ)_/¯

[bcspragu]

I hit this today with something like:

environment:
  - DATABASE_USER=${DATABASE_USER:-postgres}
  - DATABASE_URL=${DATABASE_URL:-postgresql://$DATABASE_URL:somepassword@host/database}

No variation of the last DATABASE_URL (eg. $DATABASE_URL or ${DATABASE_URL}) gets resolved, it always stays as a literal postgresql://$DATABASE_URL:somepassword@host/database

[luckylinux]

I also got hit by this.

Not specifically to podman-compose but in Quadlets, but since the .env File is directly fed to the Container, the end Result is the same.

Issue struck on PostgreSQL Container for me as well.

A way to get around this is to rename the .env File to .env.template, then let a small Script do the Evaluation.

Make sure that you remember to ONLY modify the Template File and NOT the .env File directly (hence the Warning that I print to the Head of the File)

This seems to work for me:

#!/usr/bin/env python

# Import Core Libraries
from dotenv import dotenv_values
from pathlib import Path

# Load Configuration from Template File
config = dotenv_values(".env.template")

# Define Rendered Filename
destination_filename = ".env"
destination_filepath = Path().cwd().joinpath(destination_filename)

# Remove existing Rendered File
if destination_filepath.exists():
    destination_filepath.unlink()

# Create new Empty File
destination_filepath.touch()

# Open File in Write Mode
with open(destination_filepath.as_posix(), "w") as destination_filehandle:
    # Write Warning Header
    destination_filehandle.write("# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!")
    destination_filehandle.write("\n")
    destination_filehandle.write("# !! WARNING: THIS FILE HAS AUTOMATICALLY BEEN GENERATED FROM TEMPLATE !!")
    destination_filehandle.write("\n")
    destination_filehandle.write("# !! WARNING: ANY MODIFICATION MADE TO THIS FILE WILL BE OVERWRITTEN   !!")
    destination_filehandle.write("\n")
    destination_filehandle.write("# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ")
    destination_filehandle.write("\n")
    destination_filehandle.write("\n")

    # Print each Key to File
    for key in config:
        # Get Value (will be evaluated)
        value = config.get(key)

        # Debug
        # print(f"{key}={value}")

        # Write to File
        destination_filehandle.write(f"{key}={value}")
        destination_filehandle.write("\n")

[Kaurin]

Same boat. Workaround:

podman compose --env-file myenv.env up -d

Version details

Client:       Podman Engine
Version:      5.4.2
API Version:  5.4.2
Go Version:   go1.24.1
Git Commit:   be85287fcf4590961614ee37be65eeb315e5d9ff
Built:        Wed Apr  2 22:40:07 2025
OS/Arch:      linux/amd64

[BolajiOlajide]

podman mostly gets confuse when the key of the environment variable contains an underscore.

And it's specific to the environment key in a compose.yaml file.