Skip to content

Conversation

@ByteMaDo
Copy link
Contributor

We use KDI for one of our customers, who sends Kafka messages via the Avro single-object encoding specification.
This pull-request introduces support for this encoding type.

Implementation details:

  • Our client does not (nor want to) utilize a schema registry. As a result, this implementation supports one or more provided schema files instead.
  • The implementation requires apache-avro version 0.17.0 or later due to a bug in the fingerprint calculation in earlier versions.

We’re happy to address any feedback or suggestions to improve this pull-request further.

@ByteMaDo ByteMaDo marked this pull request as draft November 25, 2024 19:39
@ByteMaDo ByteMaDo marked this pull request as ready for review November 25, 2024 19:40
@rtyler
Copy link
Member

rtyler commented Mar 9, 2025

@ByteMaDo at a high level this pull request looks good to me, can you rebase off of main and fix the conflict so it can merge cleanly?

@rtyler rtyler self-assigned this Mar 9, 2025
@rtyler rtyler self-requested a review March 9, 2025 17:23
@ByteMaDo
Copy link
Contributor Author

I'll do that in the next few days :-)

@ByteMaDo
Copy link
Contributor Author

ByteMaDo commented Mar 12, 2025

Okay - I am not sure why synching my fork of delta-io/kafka-delta-ingest with main automatically closed this pull request.
However, I will either reopen or create a new one once i've finished the rebase. :-)

@rtyler I'm having the same build issues with the current main branch as described in #212. i.e. the described version issues with strum.
I'll go ahead with the pull request once the issue is fixed. In the mean time the single-object encoding lives on in this branch: https://github.com/ByteMaDo/kafka-delta-ingest/tree/soe-avro

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants