An app that plays the viewer's own video files, from a folder, a USB drive, a WebDAV or SMB share, meets many containers and codecs: MKV, AVI, WMV, disc images, 10-bit H.264, sound formats a TV box has no decoder for. Yonto Player is that kind of app, so it carries two players, and a small rule decides which one a given file starts in. This post is about the Android apps (TV, phone and tablet); Apple versions are on their way. I think the interesting part of "two players" is not the second player, it is the seam between them.
Why the platform player alone is not enough
On Android the platform player is ExoPlayer (media3). In my view it is the right default. It uses the device's hardware decoders, shows HDR where the screen can, and works with picture-in-picture and casting. For H.264 and H.265 in MP4, WebM, TS and, on Android, MKV, it is, in my view, what should play.
It reads a fixed set of containers and decodes what the device's decoders decode. A library of arbitrary files goes past both limits: an AVI or a WMV is a container it does not read, and a Hi10P H.264 file or a film whose sound no decoder on the device handles is a file it reads but cannot decode. A second player built on libVLC, a software engine with its own demuxers and decoders, covers a lot of that ground. The public statement of which player plays what is on the video formats page.
The rule is one function
Both apps (TV and phone) share the code that decides, in a module with no UI in it. The answer is one of three values:
- the platform player alone,
- the software player alone,
- the platform player first, with the software player ready to take over.
A simplified sketch of the decision (generic names, the real function has a few more inputs):
// Simplified sketch, not the real file. enum class Players { Platform, Software, PlatformThenSoftware } fun playersFor(target: Target, hasSoftware: Boolean, prefersSoftware: Boolean): Players = when { !hasSoftware || target.isLive -> Players.Platform target.isPlaylist -> Players.Platform containerOf(target) in softwareOnly -> Players.Software prefersSoftware && target.namesAFile -> Players.Software else -> Players.PlatformThenSoftware } Reading it top to bottom is the design.
A live channel stays in the platform player. A channel is a playlist the platform player reads, and the software engine is for files. The same goes for a stream in pieces (HLS or DASH) on Android: libVLC reads through a small local server that serves one file, so a playlist's segments would not reach it, and sending it there would only remove the chance to play.
Next comes a short list of containers the platform player does not read: AVI, WMV, ASF, RM, RMVB, VOB and disc images (iso). For those, starting in the platform player would only be a detour before the same hand-over, so the software player starts at once. When the address says nothing (a media server's stream?static=true), the stream's declared type is used, and a generic video/mp4 is ignored because sources send it for files stored in any container.
Then the viewer's own setting, Always use the second player in Settings, which starts in the software player whatever the file is, as long as the file can be named as a file.
Everything else, which is most videos, starts in the platform player and keeps the other one in reserve. I like that the list of "never reads" containers is small on purpose: the platform player keeps hardware decoding, HDR and the rest for every file that might play, and a container joins the list only when it does not play there at all. The fall-back catches everything else.
The seam: handing over mid-title
PlatformThenSoftware is where the work is. The session above the players, the screen with its seek bar, the History record and the resume position, should see one player that happened to take a moment longer to start. So the pair is wrapped in one object that implements the same controls as a single player.
It hands over in two situations.
- The first player fails on a format it does not read. The failure carries a cause, and the rule only hands over on causes the second player might do better on: a format it does not read, or a damaged stream. A network failure or a refused request would be met again by the second player, so those are shown as they are.
- The first player says it has no decoder for a track. That is a separate event from failure. A film may play, but its sound or its profile of video cannot be decoded here, and the first player reports it instead of playing black or silent. The second player takes over for that title.
The hand-over itself is a few lines of state moved from one player to the other:
// Simplified sketch, not the real file. private fun fallBack(firstFailed: Boolean): Boolean { val position = first.currentPosition val speed = first.speed val shouldPlay = firstFailed || first.playWhenReady val second = try { owned(newSecond) } catch (_: Throwable) { return false } current = second first.release() second.speed = speed if (position > 0) second.seekTo(position) second.playWhenReady = shouldPlay return true } The second player is built only when it is needed. Most titles never create it, so for a title that plays in the first player no decoder instance is created for the other one.
Each player gets its own event sink that forwards to the session only while that player is the current one. A callback arriving late from a released player cannot undo what the new one is doing. The hand-over is guarded: each callback checks that its player is still the current one before it reaches the session.
If the second player cannot be built, nothing changes: a first player that still plays goes on, and one that had failed shows its own failure. The viewer gets the message from the player that tried first, because the second one reports a cause only when its source failed, and otherwise says only that it could not read the stream.
What it costs
Two players is not free, and I would not pretend it is.
There are two code paths for the same controls: seeking, tracks, speed, picture-in-picture, casting. Each feature has to work behind both players, and the video formats page is where the differences are written down for viewers.
The libVLC build is our own. The prebuilt "all" distribution contains GPL code, and I wanted the app to carry an LGPL-only engine, so there is a build script that makes libVLC from VideoLAN's sources without GPL components and without a video encoder, and Gradle reads the built libraries from a repository pinned to one exact commit, at a version number of its own. That is a piece of infrastructure to keep alive, and a second engine's native libraries add weight to the app. The extra sound decoders (AC3, DTS, TrueHD and a few more) are FFmpeg built the same way, as LGPL shared libraries.
The rule also needs tests that are not about video at all. The decision is a pure function and the hand-over takes its players as constructor arguments, so both are tested with fake players: which container goes where, which failure causes hand over, that a late event from the first player is ignored, and that a player paused when it is handed over stays paused. That is the benefit of keeping the seam in a module without UI.
What I would take from it
If you build a player that has to cope with files you did not choose, make the choice of engine a pure function of the file and a few settings, and put the hand-over behind the same interface as a single player. Keep the "always start in the other one" list as short as you can justify, and let a typed failure cause, not a message, decide whether a second try could help. The viewer then sees one player, and you get to test the decision without a device.
