Specify the `source_catalog.meta` in the schemas

[WilliamJamieson]

Currently romancal populates the meta attribute on the source_catalog tables, with some general information which is the same for all rows in the table. This metadata is not documented in the schemas, but is easily specified.

Should we update the schemas with these specifications so they are documented in RAD?

[WilliamJamieson]

Tagging @schlafly.

[schlafly]

Sorry I dropped this.

The bigger question here is how to handle the parquet metadata vs. the astropy Table metadata. It's awkward to have two metadata locations, but they each have advantages & disadvantages:
Table metadata

• understood by tab = astropy.Table.read(...) tab.meta and things like columns names, descriptions, and units. If we don't use this parquet is "worse" than ecsv / fits / ... in that there isn't an easy standard way to get this information.
• stored in a long yaml string in the parquet metadata
  parquet metadata
• more natural and easier to read from a pure parquet perspective

Right now we use both types of metadata for different products, which seems unfortunate. We should probably use one or the other or duplicate everything. Eventually CSB will start ingesting our data release catalogs and will want to know which of these columns should go into their databases, and we probably don’t want to tell them that they need to use astropy to ingest a parquet file or that they need to implement a yaml parser for that ingest process, which would argue for the parquet approach. I kind of think astropy should try to make better use of the ”native” parquet metadata.

Tyler is talking about L4 metadata with the SSC ~now and maybe this should be part of that discussion.

For now I'm happy having sort of core bits of metadata be in the parquet metadata (that we expect CSB to parse) and sticking other bits of metadata in the table.meta, though it's not a good long-term solution.