Unspecified environment variable ends up as an empty string

[jfly]

We're trying to use schematic like this:

use schematic::ConfigLoader;

#[derive(Debug, schematic::Config)]
struct Config {
    #[setting(env = "EXAMPLE_VAR")]
    EXAMPLE_VAR: String,
}

fn main() {
    let config: Config = ConfigLoader::new()
        .load()
        .context("loading configuration from environment").unwrap()
        .config;
    dbg!(config);
}

And if we run this, we get something like:

[.../src/main.rs:33] config = Config {
    EXAMPLE_VAR: "",
}

when we'd really expect to get an error about EXAMPLE_VAR not being specified.

[milesj]

@jfly Env vars are always optional, but you do have a few options here:

• Add a custom parse_env that errors if the env var is missing/empty: https://moonrepo.github.io/schematic/config/struct/env.html#parse-function
• Try using the not_empty validator: https://docs.rs/schematic/latest/schematic/validate/fn.not_empty.html

[jfly]

Thanks, @milesj. A couple of questions:

Env vars are always optional

Sorry, I'm having trouble understanding this. I know I mentioned environment variables in the title, but in my mind, this isn't a question of whether an env var is required or optional, it's about a config setting that the user is required to specify. In the above example, it's the this rust struct attribute Config.EXAMPLE_VAR that is required (it's a String, not an Option<String>). I know schematic can handle loading config from multiple places, but regardless of how many places we're loading the config from, this feels to me like a situation where the user really should be expected give us a value somehow.

Add a custom parse_env ...[or]... Try using the not_empty validator

We can give that a shot, but do you happen to know if we'll be able to distinguish between missing and truly present, but empty string? For missing, we'd like the user to receive an error, but if they set it to empty string, that's ok.

[milesj]

@jfly Yeah so the struct that derives Config is the "final state", after all partials have been loaded. If a field is missing in the partial, it gets filled with Default. So in this final config, a non-option setting doesn't mean that it's required, just that it would always have a value, even if empty/default.

You would need to use validation to handle this. We have a #[setting(required)] attribute, but that's only for Options.

[mightyiam]

This design, where env vars are always optional... Why?

[milesj]

This design is primarily for config files, and ease of use on the Rust side. Environment variables are secondary, as they are used to populate a value and override those found in the config file.

[mightyiam]

Could we consider promoting env to a first class source of config, instead?

That would support the use case of an app that takes config only via env.

[milesj]

You could do that by implementing your own parse_env or validator functions.

[mightyiam]

I guess you have your priorities. Thanks anyway.

[milesj]

If it was built into core, it would be one of those approaches anyways.

[milesj]

Closing as there's nothing actionable.

[JeanMertz]

@jfly Yeah so the struct that derives Config is the "final state", after all partials have been loaded. If a field is missing in the partial, it gets filled with Default. So in this final config, a non-option setting doesn't mean that it's required, just that it would always have a value, even if empty/default.

You would need to use validation to handle this. We have a #[setting(required)] attribute, but that's only for Options.

@milesj would you be open to allowing required on non-Option fields?

In my mind, if I have a struct:

#[derive(Config)]
struct Foo {
    field: String
}

This would generate:

struct FooPartial {
    field: Option<String>
}

And then, when doing the partial loading, if, after all partials are merged, field is still None, I would expect an error to be returned indicating that field is a required field, and cannot be unset when generating the final Foo configuration.

If I wanted that field not to be required, I would have made the Foo struct have field: Option<String>.

That seems like a fairly heavy breaking change, and potentially doesn't work with other design goals of this project. If that's the case, then allowing required on non-Option fields would be the second-best solution to defining fields that must be provided to create a valid configuration.

The idea (for me at least) is that I want to be explicit to the user about any required fields that we provide default values for. So in my app, I would generate a default config.toml file with all required fields populated with default values. The user can then inspect that file and understand what fields get what default value, without there being any "magic defaults" that are invisible to the user. However, if they then remove a default value without supplying a new one, I want the before-mentioned "required field missing" error to pop up.

Even if this were possible currently with the required attribute on Option fields, that makes working with the library in the code itself less ergonomic, as that means I have to constantly unwrap that field, expecting/knowing that it can never be None, but only if generated through the schematic builder.

Does that make sense? Thoughts? I know you are redesigning this library (not sure if/when that redesign is expected to land), so perhaps you don't want to deal with this question right now, if so, let me know.

[milesj]

Refer to #171 (comment)