Repository navigation
Conversation
With EFFECTS_ATLAS, libass renders the subtitles inside media3's video effects. When rendering is slower than the frame rate (heavy typesetting, animated karaoke), media3's two ways of dropping late frames work badly: - After the effects, media3 drops the frames that turned out late. By then the subtitles are already rendered, so dropping the frame saves no time: the work is thrown away and the next frame is just as late. - Before the effects, media3 drops frames it *predicts* will be late, based on how late the frames it showed were. That information is a couple of frames old, and dropped frames don't count toward it, so while libass stays slower than the frame rate the prediction keeps saying "late". It ends up dropping up to 2 frames out of 3, when dropping about 1 out of 10 would be enough to keep up. The result is very choppy video where a few dropped frames would do. Add AssMediaCodecVideoRenderer, right before a frame enters the video effects, drop it if its display time is already over (pts + duration < now). Never drop if no frame was shown for 100ms, so playback slows to about 10fps instead of freezing. Only frames that can no longer be shown on time are dropped, and they are dropped before libass spends any time on them. AssRenderersFactory takes an optional Context to create the renderer and buildWithAssSupport() passes it. withAssSupport() gets the same optional parameter. Without a Context, or when the video renderer is a subclass of MediaCodecVideoRenderer (which may have behavior of its own), the renderer is kept and media3's dropping is used, with a warning in the log.
|
I tested this pull request using the sample. To the human eye, playback looked similar to my previous test mentioned in damontecres/Wholphin#2227. However the dropped frames did improve from 124 to 97. I would love to test this build integrated with Wholphin and see if the ~30 seconds video and audio desycn is improved, but that is beyond my current skill to setup and build. If that apk was provided, I would be happy to test it though. |
I am surprised that you only get this kind of amelioration. For me, mpv drops around 20 frames. Before this PR, I was getting around 60 frame dropped and now I get 7 to 10, so a pretty big improvement.
Probably not. This PR doesn't do anything related to that. |
For the original sample ioxho8.mkv and pure mpv-android, i had 89 dropped frames, pretty comparable to this pull requests 97 dropped frames. 8u4w9r.mkv had 227 dropped frames in pure mpv-android and 235 dropped frames for this pull requests This all suggests this pull request brings |
With this sample (https://files.catbox.moe/8u4w9r.mkv), I was getting around 60 frames dropped. Now, with these changes, I get around 7 to 10 frames.
With EFFECTS_ATLAS, libass renders the subtitles inside media3's video effects. When rendering is slower than the frame rate (heavy typesetting, animated karaoke), media3's two ways of dropping late frames work badly:
The result is very choppy video where a few dropped frames would do.
Add AssMediaCodecVideoRenderer, right before a frame enters the video effects, drop it if its display time is already over (pts + duration < now). Never drop if no frame was shown for 100ms, so playback slows to about 10fps instead of freezing. Only frames that can no longer be shown on time are dropped, and they are dropped before libass spends any time on them.
AssRenderersFactory takes an optional Context to create the renderer and buildWithAssSupport() passes it. withAssSupport() gets the same optional parameter. Without a Context, or when the video renderer is a subclass of MediaCodecVideoRenderer (which may have behavior of its own), the renderer is kept and media3's dropping is used, with a warning in the log.
@ethaldeman @damontecres