cmd,server: Send stream health events to a metadata queue#1946
Merged
cmd,server: Send stream health events to a metadata queue#1946
Conversation
victorges
force-pushed
the
vg/fix/webhook-logs
branch
from
e1f2d01 to
bcf9acc
Compare
victorges
force-pushed
the
vg/feat/metadata-queue
branch
from
5f9136c to
2b6f269
Compare
victorges
force-pushed
the
vg/feat/metadata-queue
branch
2 times, most recently
from
bdf9c65 to
b104da0
Compare
victorges
force-pushed
the
vg/feat/metadata-queue
branch
from
b104da0 to
617fe10
Compare
victorges
force-pushed
the
vg/feat/metadata-queue
branch
13 times, most recently
from
05c14bf to
cafd5d4
Compare
victorges
changed the title
iameli
reviewed
Jul 12, 2021
Contributor
iameli
left a comment
iameli
left a comment
There was a problem hiding this comment.
The more I think about this the more I think that we should probably be pushing an event for every transcode attempt and failure. It'll be good for the Stream Health service to be able to evaluate how things are performing in real-time, not only after multiple retries are happening. (Though there oughta be some sort of correlation ID so it can figure out which attempts go with which successes, I suppose.)
I'll leave review of the actual code structure here to @yondonfu and @jailuthra
Contributor
|
It occurs to me that we'll probably want to develop this in conjunction with the service that consumes it — inevitably we'll want to make tweaks as everything comes together. So maybe we'll keep this branch open for a little bit if folks don't object. |
victorges
force-pushed
the
vg/feat/metadata-queue
branch
3 times, most recently
from
d03fe63 to
503ca09
Compare
- Include both orch eth address and transcoder URI - transcodeAttemptInfo as an out parameter of transcodeSegment - named returns+defer trick to avoid repetitions of time&error setting
- Remove defer from retry loop - Fix error field type on transcode info - Make transcode attempt info a ret-val instead of ptr Still keep the named-rets+defer logic though which I guess was the real deal
Put it all together ✨
The library closes that channel when the channel is closed as well, so we need to handle it in case that msg comes before the actual on closed channel.
Contributor
Author
|
Yeah weird, let me try rebasing to see if it works. Edit: Apparently it did 🤔 |
Final timestamp so it doesn`t need to be derived and byte size of segments.
Also add first character of manifest ID as separate topic segment
It's not really a queue per se, but it's just the easier way to refer and reason about it.
Co-authored-by: Yondon Fu <yondon.fu@gmail.com>
Co-authored-by: Yondon Fu <yondon.fu@gmail.com>
- Create the helper stubTestTranscoder to avoid repeating logic - Repeat the pattern on other tests to create a single rtmpConnection and just keep changing the sessManager on it instead. - Some additional guarantees in metadata queue tests
Co-authored-by: Yondon Fu <yondon.fu@gmail.com>
victorges
force-pushed
the
vg/feat/metadata-queue
branch
from
bb3bce3 to
c262c3c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this pull request do? Explain your changes. (required)
This is the first step for implementing a stream health dashboard. The idea is
to send events about stream transcoding to an AMQP queue (exchange). This
queue will later be processed by another (go) service which will aggregate the
information so it can be displayed to the final user.
Specific updates (required)
In the order of the commits:
broadcaster.goto gather transcode attempts information[1]underserver/eventpackage[2][1]In the ideal world, we would just use an existing library that handledall the protocol complexity, bur I couldn't find any other library with enough
developer activity to be safe enough to use.
[2]Is it ok to create sub-packages like that, or should it all be underserver? Ormaybe somewhere else and have
serverdepend on it?How did you test each of these updates (required)
./test.shhealth code, as well as actually test the whole thing running.
Does this pull request close any open issues?
Implements #1939
Checklist:
makeruns successfully./test.shpass