Create a new KeyInfoManager based on SQLite

[hug-dev]

See #394 for history and the reason behind this. See also these community meeting minutes.

This new KIM could enable:

• more long term robustness around key mappings, possibly easier scaling
• multiple authenticators at the same time
• multiple providers of the same type
• allowing non-breaking updates of psa-crypto

However, there would need to be a way to transfer key mappings from the old KIM to the new one.

Also we need to think if multiple KIMs are allowed in parallel.

[MattDavis00]

---

Proposed Design Overview

This is a living proposal for the SqliteKeyInfoManager design & implementation.

---

Contents

• Motivation
• Config Changes
• Additional Dependencies
• Database Schema (Structure Overview)
• Execution Flow
• Testing
• Discussion Points

---

Motivation

• Allow multiple authenticators at the same type.
• Allow multiple providers of the same type.
• Possibly easier scaling & better performance.
• More long term robustness around key mappings.
• Allow dynamic Provider ID

---

Config Changes

[[key_manager]]
name = "sqlite-manager"
manager_type = "Sqlite"
sqlite_db_path = "/var/lib/parsec/sqlite-key-info-manager.sql"

Add a sqlite_db_path config option so that users can configure where the key mappings are stored. Default to "/var/lib/parsec/sqlite-key-info-manager.sql". Reuse the name & manager_type config options, adding Sqlite as a manager_type.

[[provider]]
name = "trusted-services-provider"

Add a optional provider_name config option so that this can be used within the KIM to separate keys.
Should default to the name of the provider e.g. "trusted-services-provider", "crypto-auth-lib-provider" etc.

---

Additional Dependencies

• rusqlite crate

It should only be the rusqlite crate that should be required for the SqliteKeyInfoManager. This is as the following config change can be used to bundle libsqlite3-sys dependencies including a SQLite library:

[dependencies.rusqlite]
version = "0.25.1"
features = ["bundled"]

---

Database Schema (Structure Overview)

This section aims to give an overview of what current storage schema is with the OnDiskKeyInfoManager, and what the proposed SqliteKeyInfoManager database schema looks like.

Current OnDiskKeyInfoManager Schema:

mappings_dir_path/
   |---app1/
   |   |---provider1/
   |   |   |---key1
   |   |   |---key2
   |   |   |   ...
   |   |   |---keyP
   |   |---provider2/
   |   |   ...
   |   |---providerM/
   |---app2/
   |   ...
   |---appN/

Where:

• app1 is a directory containing all of the providers in use by that app.
• provider1 is a directory containing all of the keys used for that app & provider combination.
• key1 is a file containing KeyInfo.

Proposed SqliteKeyInfoManager Schema:

key_mapping table:

Field Name	Field Type	Primary Key	Description
authenticator_id	INT	✔	Id of the authenticator used.
application_name	TEXT	✔	Name of the application.
key_name	TEXT	✔	The unique unicode key name.
provider_uuid	TEXT	❌	UUID of the provider
provider_name	TEXT	❌	Name of the provider, used to distinguish two providers of the same type running in parallel.
key_id	BLOB	❌	A key_id representing a reference to a key in the provider, encoded into bytes using bincode::serialize(key_id).
key_id_version	INT	❌	An integer version number tracking the serialization used to encode the key_id
key_attributes	BLOB	❌	A psa_key_attributes::Attributes struct encoded into bytes using bincode::serialize(key_attributes).
key_attributes_version	INT	❌	An integer version number tracking the serialization used to encode the key_attributes

The primary key of the table will consist of a authenticator_id-application_name-key_name composite key. Therefore, unless a key/request matches all 5 of these properties it will be treated as a separate key.

manager_metadata table:

Field Name	Field Type	Primary Key	Description
key	TEXT	✔	The metadata item key name.
int_value	INT	❌	The value of the metadata item.

For example:

Key	Value
"schema_version"	1

---

Execution Flow

Initialization:

1. If the db does not exist, create it (sqlite-key-info-manager.sql).
2. Check the parsec_version value from the manager_metadata table.
3. If the KIM parsec_version > Parsec parsec_version (KIM: 0.9.0, Parsec: 0.8.0 for example):
   1. Throw an error as we don't support backwards-migration of Parsec.
   2. If they want to do this at their own risk, they can make a backup of the key mappings and then edit the parsec_version manually but compatibility is not guaranteed as the KIM structure may have changed.
4. If the KIM parsec_version == Parsec parsec_version (KIM: 0.8.0, Parsec: 0.8.0 for example):
   1. Do nothing as it is correct pairing.
5. If the KIM parsec_version < Parsec parsec_version (KIM: 0.7.0, Parsec: 0.9.0 for example):
   1. Make a copy of sqlite-key-info-manager.sql as sqlite-key-info-manager-0.7.0-backup.sql in the same directory:
   2. While (KIM parsec_version < the Parsec parsec_version):
      1. Iteratively read migration files:
         1. Read sqlite-kim-migration-0.7.0-to-0.8.0.sql from disk.
         2. Execute migration SQL on the database.
         3. Read sqlite-kim-migration-0.8.0-to-0.9.0.sql from disk.
         4. Execute migration SQL on the database.
6. sqlite-key-info-manager.sql key_mapping table queried using SELECT statements to retrieve persistent data.
7. Persistent data saved to a key_store<KeyTriple, KeyInfo> HashMap, acts as a read cache.
8. New SqliteKeyInfoManager is returned.

Inserting KeyInfo:

1. SqliteKeyInfoManager.insert(KeyTriple, KeyInfo) action is called.
2. First perform INSERT on key_mapping table with the data supplied.
3. Then add the mapping to the key_store HashMap read cache.

---

Testing

• Currently the OnDiskKeyInfoManager has some unit tests that would be applicable to the SqliteKeyInfoManager, such as testing large key names with emoticons. This logic should be split out, so that the non-manager-specific tests are run against all current and future Key Info Managers.
• SqliteKeyInfoManager-specific unit tests.

---

Discussion Points

• Should we consider moving from a KeyTriple? To a KeyQuadruple or KeyQuintuple? Relates to Enable multiple authenticators to work simultaneously #271
  • For Example:

    KeyIdentity {
      application: ApplicationIdentity {
        name: String,
        authenticator_id: u8
      },
      provider: ProviderIdentity {
        uuid: String,
        name: String,
      },
      key_name: String,
    }
    

• Are we going to allowing migration of data stored using OnDiskKeyInfoManager to SqliteKeyInfoManager?
• Do we think the database schema is appropriate?

---

Any feedback is appreciated.

Thanks,
Matt

[hug-dev]

Thanks for the design. It generally looks good to me. I have some comments/discussion points!

Some comments on the proposal above:

• in the motivation section, I would also add "Allow dynamic Provider ID", as they won't be present anymore in the mappings
• I think instead of having directly key_info in the schema, it would be better to split with key_id and key_attributes (both BLOB). That would allow us in the future to use the protobuf version of the key attributes instead of the bincode one (that is more resistant to breaking changes).
• I think adding non-primary key fields, like parsec_version is fine. Could be useful for debugging in case things go really wrong. We really need to make sure that reading/writing key mappings is always stable, without having to use this field though I think.
• worth to say somewhere that provider_name would be a new optional field in the configuration of providers.

About the discussion points:

Should we consider moving from a KeyTriple?

I think we could also keep the KeyTriple but modify its contents:

• ApplicationName -> Application which would be a tuple of an authenticator and a name
• ProviderID -> Provider which would be a tuple of a provider uuid and a name

Note that those changes would also impact the Provide trait which would now need the Application type instead of just the ApplicationName. The Provider type could later be useful to be replaced throughout the service where we use ProviderID. For dynamic provider ID (in the future), we would build a mapping somewhere between provider ID and Provider.

Are we going to allowing migration of data stored using OnDiskKeyInfoManager to SqliteKeyInfoManager?

I think we could have a script that reads the OnDiskKeyInfoManager info and insert it into the database? We will have to think how to populate the information that is not implicit like authenticator_id and provider_name (maybe the default value for that one?). I don't think we should systematically convert all mappings from one KIM to the other but maybe offer the script to who wants it, because they need it to unlock other Parsec features.

Do we think the database schema is appropriate?

I personally think so. A related question that I have is how this schema handles stability? What happens if in future Parsec version we add new fields? Can they be primary keys? What if Parsec is newer than the database and tries to read fields that do not yet exist?

Other

• We should have a look at the threat model and see if it needs any updates with this change. Particularly attackers A6/A7.
• What are the new dependencies needed because of the new KIM?

[MattDavis00]

• I think instead of having directly key_info in the schema, it would be better to split with key_id and key_attributes (both BLOB). That would allow us in the future to use the protobuf version of the key attributes instead of the bincode one (that is more resistant to breaking changes).

Seems good to me.

• worth to say somewhere that provider_name would be a new optional field in the configuration of providers.

Yes very true, I thought it had already been implemented but I was mistaken.

Should we consider moving from a KeyTriple?

I think we could also keep the KeyTriple but modify its contents:

• ApplicationName -> Application which would be a tuple of an authenticator and a name
• ProviderID -> Provider which would be a tuple of a provider uuid and a name

Sounds sensible to me.

Are we going to allowing migration of data stored using OnDiskKeyInfoManager to SqliteKeyInfoManager?

I think we could have a script that reads the OnDiskKeyInfoManager info and insert it into the database? We will have to think how to populate the information that is not implicit like authenticator_id and provider_name (maybe the default value for that one?). I don't think we should systematically convert all mappings from one KIM to the other but maybe offer the script to who wants it, because they need it to unlock other Parsec features.

I think an optional migration script is definitely the way to go.
provider_name could be set to the default value for each key's current provider type.
Then possibly they get to pass an authenticator_id as an argument to the script?

Do we think the database schema is appropriate?

I personally think so. A related question that I have is how this schema handles stability? What happens if in future Parsec version we add new fields? Can they be primary keys? What if Parsec is newer than the database and tries to read fields that do not yet exist?

I would suggest on startup reading the current parsec version of the SqliteKeyInfoManager from the database. As mentioned, this could also be stored in a separate table, instead of including it on a per-key basis, such as:

manager_metadata table:

Key	Value
"parsec_version"	"0.7.0"

A typical SqliteKeyInfoManager initialization could look like so:

Initialization Flow:

1. If the db does not exist, create it.
2. Check the parsec_version value from the manager_metadata table.
3. If the KIM parsec_version > Parsec parsec_version (KIM: 0.9.0, Parsec: 0.8.0 for example):
   1. Throw an error as we don't support backwards-migration of Parsec.
   2. If they want to do this at their own risk, they can make a backup of the key mappings and then edit the parsec_version manually but compatibility is not guaranteed as the KIM structure may have changed.
4. If the KIM parsec_version == Parsec parsec_version (KIM: 0.8.0, Parsec: 0.8.0 for example):
   1. Do nothing as it is correct pairing.
   2. Create SqliteKeyInfoManager instance as normal.
5. If the KIM parsec_version < Parsec parsec_version (KIM: 0.7.0, Parsec: 0.9.0 for example):
   1. Make a copy of sqlite-key-info-manager.sql as sqlite-key-info-manager-0.7.0-backup.sql in the same directory:
   2. While (KIM parsec_version < the Parsec parsec_version):
      1. Iteratively read migration files:
         1. Read sqlite-kim-migration-0.7.0-to-0.8.0.sql from disk.
         2. Execute migration SQL on the database.
         3. Read sqlite-kim-migration-0.8.0-to-0.9.0.sql from disk.
         4. Execute migration SQL on the database.
   3. Create SqliteKeyInfoManager instance.

In terms of can we add new primary keys, yes. If we have an appropriate default value to set the new primary key as upon creation this shouldn't be an issue.

• We should have a look at the threat model and see if it needs any updates with this change. Particularly attackers A6/A7.

As Sqlite stores the database as a file on the filesystem, I think the same model as the previous KIM would also apply here.
The database file should be stored in a directory with strict access control, managed by the OS, only available to Parsec etc.

• What are the new dependencies needed because of the new KIM?

Should just be the rusqlite crate.

---

Thanks for the feedback! 👌

[ionut-arm]

A few thoughts/questions in no apparent order.

ProviderID -> Provider which would be a tuple of a provider uuid and a name

Maybe not a tuple and use a normal struct instead? Tuples are a bit annoying in terms of readability.

I think an optional migration script is definitely the way to go.
provider_name could be set to the default value for each key's current provider type.
Then possibly they get to pass an authenticator_id as an argument to the script?

Hmm, I'd argue we should accept some sort of configuration file for the script so we can tell it exactly what to convert and to what. For example, we could even consider expanding it to cover app name migration. Maybe it would accept something like

[authenticator]
id = 123

[[providers]]
id = 456
name = "some-name"

And the script would then use these mappings to shift things around. I'm leaning towards something like this because it's nice to have some persistent representation of what got converted to what, maybe you want to refer to it in the future or compare with your old/new service config file.

We should probably worry about this migration after the new KIM is available, though.

Should just be the rusqlite crate.

I thought I read that rusqlite doesn't come with its own implementation of SQLite, so we need that as a library as well.

[MattDavis00]

I'd argue we should accept some sort of configuration file

Yeah that's a good idea.

I thought I read that rusqlite doesn't come with its own implementation of SQLite, so we need that as a library as well.

Added some details about dependencies earlier, don't know if it was in the revision you saw. You can have rusqlite in bundled mode where it's sub-dependency libsqlite3-sys will include a SQLite library at build. Specific part in docs for reference.

[paulhowardarm]

Are we intending to encapsulate all of this with a feature flag? Should we? (All providers and authenticators have feature flags, and it possibly should follow that KIMs have them as well, although the existing on-disk KIM doesn't).

[paulhowardarm]

I like the proposal, although it has started me thinking about the slightly awkward relationship that currently exists between the KIM and the providers. Even the on-disk KIM today is effectively using both the application identity and the provider identity as part of the namespace. But this isn't how we normally describe the semantics of Parsec namespaces, which are supposed to be based only on the client identity.

If I was starting from a blank slate, I think I would recommend the following:

• A KIM should be associated with an authenticator, not with a provider, and this association should be strictly 1-1.
• A KIM should not treat the provider as part of the namespace for lookups. The provider uuid/name should be part of KeyInfo rather than KeyTriple (or whatever that evolves into). I think it should be on the "right-hand side" of the mapping, rather than the "left-hand side" as it currently is. It then tells us which provider holds the key, but without being part of the key's identity in terms of lookup.

This would mean that a given KIM would not be able to store two keys with the same name in two different providers. But I would argue that this is actually the correct data model, and probably should have been to begin with. It allows a use case where a client can choose the "best" provider at key-creation time, but any subsequent code that uses the key no longer needs to care which provider it is in, because the combination of application-identity/key-name implies the provider from that point.

We are, of course, not starting from a blank slate! We necessarily have to preserve all aspects of how Parsec is working today. But if we're introducing a new KIM, then perhaps it's a good opportunity to make sure that we have the semantics exactly right, on the assumption that everyone will eventually move to it. And of course, with single-provider deployments, it doesn't matter anyway.

If the KIM were 1-1 bound to an authenticator, then there would be no need for authenticator_id in the schema, because the entire DB would be scoped to the authenticator already.

[paulhowardarm]

BTW, maybe KeySelector or KeyIdentifier would be better names than KeyTriple, especially if we're changing its fields.

[MattDavis00]

Are we intending to encapsulate all of this with a feature flag? Should we? (All providers and authenticators have feature flags, and it possibly should follow that KIMs have them as well, although the existing on-disk KIM doesn't).

I'm not familiar with how we are currently using feature flags, what would be the advantage of this? Improved compilation time?

This would mean that a given KIM would not be able to store two keys with the same name in two different providers. But I would argue that this is actually the correct data model, and probably should have been to begin with. It allows a use case where a client can choose the "best" provider at key-creation time, but any subsequent code that uses the key no longer needs to care which provider it is in, because the combination of application-identity/key-name implies the provider from that point.

Yes, maybe something like this would be more appropriate?

Field Name	Field Type	Index	Description
authenticator_id	INT	🔑 PRIMARY	Id of the authenticator used.
application_name	TEXT	🔑 PRIMARY	Name of the application.
key_name	TEXT	🔑 PRIMARY	The unique unicode key name.
provider_uuid	TEXT	🔷 INDEX	UUID of the provider
provider_name	TEXT	🔷 INDEX	Name of the provider, used to distinguish two providers of the same type running in parallel.
key_id	BLOB	❌	A key_id representing a reference to a key in the provider, encoded into bytes using bincode::serialize(key_id).
key_attributes	BLOB	❌	A psa_key_attributes::Attributes struct encoded into bytes using bincode::serialize(key_attributes).

But yes, as you pointed out this would cause stability problems; creating 2 keys with the same name in this structure would throw an error whereas in previous versions of parsec it could have been a valid call.

If the KIM were 1-1 bound to an authenticator, then there would be no need for authenticator_id in the schema, because the entire DB would be scoped to the authenticator already.

Would you suggest a new KIM instance for each authenticator? In this case, on-disk, we would end up with sqlite-key-info-manager-direct-authenticator.sql, sqlite-key-info-manager-unix-peer.sql etc correct?

BTW, maybe KeySelector or KeyIdentifier would be better names than KeyTriple, especially if we're changing its fields.

Yeah, I feel like a name change is appropriate. Maybe KeyIdentity?

[ionut-arm]

I'm not familiar with how we are currently using feature flags, what would be the advantage of this? Improved compilation time?

Yes, but also flexibility, binary size etc. For example, some providers require various dependencies locally when you try to compile them, so it might be desirable to only compile, say, the TPM provider on your system if that's all you need. Similarly for authenticators, the JWT SVID authenticator adds a lot of dependencies. Adding a new feature, at least until the KIM work is done sounds reasonable.

• A KIM should be associated with an authenticator, not with a provider, and this association should be strictly 1-1.
• A KIM should not treat the provider as part of the namespace for lookups. The provider uuid/name should be part of KeyInfo rather than KeyTriple (or whatever that evolves into). I think it should be on the "right-hand side" of the mapping, rather than the "left-hand side" as it currently is. It then tells us which provider holds the key, but without being part of the key's identity in terms of lookup.

This would mean that a given KIM would not be able to store two keys with the same name in two different providers. But I would argue that this is actually the correct data model, and probably should have been to begin with. It allows a use case where a client can choose the "best" provider at key-creation time, but any subsequent code that uses the key no longer needs to care which provider it is in, because the combination of application-identity/key-name implies the provider from that point.

This seems a bit odd to me, we'd be introducing artificial constraints even though they're not needed. I think we all agree that you shouldn't "cross" these boundaries - e.g. a key created in provider A shouldn't be accessible in provider B - but I can't grasp the (overwhelming) benefit of not allowing the name to be reused across the boundaries in a case like this. Being able to use the same name for keys that have identical "profiles" in different providers seems just as useful as being able to identify an exact provider to use given a key name.

If the KIM were 1-1 bound to an authenticator, then there would be no need for authenticator_id in the schema, because the entire DB would be scoped to the authenticator already.

Well yes, but if you want to support multiple authenticators then you need to maintain two KIMs, potentially of the same kind, in parallel (e.g. two mappings folders on disk, or two databases). Maybe that's an acceptable tradeoff, again, not sure.