Skip to content

core: reuse remote transcoder for stream#1849

Merged
kyriediculous merged 1 commit intolivepeer:masterfrom
reubenr0d:reuse-transcoder-for-stream
May 12, 2021
Merged

core: reuse remote transcoder for stream#1849
kyriediculous merged 1 commit intolivepeer:masterfrom
reubenr0d:reuse-transcoder-for-stream

Conversation

@reubenr0d
Copy link
Copy Markdown
Contributor

@reubenr0d reubenr0d commented Apr 21, 2021

What does this pull request do? Explain your changes. (required)

Specific updates (required)

  • Adds map to keep track of transcoders assigned to sessions
  • Updated selectTranscoder to return and set transcoders to sessions
  • Removed transcoder from map in case of session end/transcoder errors

How did you test each of these updates (required)

On local offchain setup:

  • Setup 1 O, 2 T, 1 B
  • Start stream and observed that only one transcoder is processing stream
  • Killed that transcoder and stream moves to other transcoder
  • Created Unit Tests

Does this pull request close any open issues?

Fixes #1842
Fixes #1712
Fixes #1273

Checklist:

@reubenr0d reubenr0d force-pushed the reuse-transcoder-for-stream branch from f3744b9 to bd7b60f Compare April 22, 2021 08:34
@kyriediculous
Copy link
Copy Markdown
Contributor

We also want to remove the currentTranscoder from the streamSessions map when it returns an error here

res, err := currentTranscoder.Transcode(md)

@kyriediculous
Copy link
Copy Markdown
Contributor

We also want to remove the currentTranscoder from the streamSessions map when the connection closes like we do here

delete(rtm.liveTranscoders, transcoder.stream)

@reubenr0d reubenr0d requested a review from kyriediculous April 23, 2021 15:14
@reubenr0d reubenr0d force-pushed the reuse-transcoder-for-stream branch 4 times, most recently from 5b8003e to cb99f0c Compare April 25, 2021 16:03
@reubenr0d reubenr0d force-pushed the reuse-transcoder-for-stream branch 4 times, most recently from 358d391 to b248bef Compare April 25, 2021 17:18
@reubenr0d reubenr0d marked this pull request as ready for review April 25, 2021 17:24
@reubenr0d reubenr0d requested a review from kyriediculous April 25, 2021 17:49
@reubenr0d reubenr0d force-pushed the reuse-transcoder-for-stream branch 2 times, most recently from 021aa53 to f659900 Compare April 28, 2021 07:56
Copy link
Copy Markdown
Contributor

@kyriediculous kyriediculous left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good ! Just two very small comments.

Feel free to rebase and flatten the commit history once you've addressed those.

Let's use a commit message that indicates which packages were altered and also use that in the title of the PR

e.g. "core: re-use remote transcoder for stream"

@reubenr0d reubenr0d force-pushed the reuse-transcoder-for-stream branch from f659900 to 0ec6d9d Compare April 28, 2021 13:19
@kyriediculous kyriediculous changed the title Reuse transcoder for stream (#1842) core: reuse remote transcoder for stream Apr 28, 2021
@reubenr0d reubenr0d force-pushed the reuse-transcoder-for-stream branch from 0ec6d9d to fb76f12 Compare April 28, 2021 14:13
@reubenr0d reubenr0d force-pushed the reuse-transcoder-for-stream branch 3 times, most recently from 3e8dcea to 59b545b Compare May 1, 2021 06:53
@reubenr0d
Copy link
Copy Markdown
Contributor Author

reubenr0d commented May 1, 2021

I've pushed the changes to the same commit(will use !fixup henceforth), have a look. The Ts are getting removed from the list properly now, but don't seem to adding back, if there is no error. This is the assertion in question. Shouldn't it be 2 after the T succeeds? I'm not quite sure how would it get selected, if the T was not in the list in the first place. Any advice?

@kyriediculous
Copy link
Copy Markdown
Contributor

I've pushed the changes to the same commit(will use !fixup henceforth), have a look. The Ts are getting removed from the list properly now, but don't seem to adding back, if there is no error. This is the assertion in question. Shouldn't it be 2 after the T succeeds? I'm not quite sure how would it get selected, if the T was not in the list in the first place. Any advice?

@reubenr0d Transcoder 2 gets removed here: https://github.com/reubenr0d/go-livepeer/blob/59b545b57a94f350879307da304a838e99d2c390/core/orch_test.go#L343

@reubenr0d
Copy link
Copy Markdown
Contributor Author

Transcoder 2 gets removed here

Right, the comment "assert transcoder gets added back to remoteTranscoders if no transcoding error" threw me off, if it's not supposed to get added back, shall I just remove the last set of assertions after that comment?

@reubenr0d reubenr0d force-pushed the reuse-transcoder-for-stream branch from f45fb1b to e5654d8 Compare May 4, 2021 13:36
@kyriediculous
Copy link
Copy Markdown
Contributor

@yondonfu Did you have any other remarks regarding the changes you requested ?

@reubenr0d reubenr0d requested a review from kyriediculous May 4, 2021 15:42
@yondonfu
Copy link
Copy Markdown
Member

yondonfu commented May 4, 2021

See #1864 (comment)

@kyriediculous kyriediculous force-pushed the reuse-transcoder-for-stream branch from e5654d8 to bf5397d Compare May 12, 2021 15:29
Copy link
Copy Markdown
Member

@yondonfu yondonfu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@kyriediculous kyriediculous merged commit 62b4830 into livepeer:master May 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants