1. Help us improve the editor usability and artist workflows. Join our discussion to provide your feedback.
    Dismiss Notice
  2. We're looking for feedback on Unity Starter Kits! Let us know what you’d like.
    Dismiss Notice
  3. We’re giving 2017.1 beta testers a chance to win t-shirts and a Nintendo Switch. Read more on the blog.
    Dismiss Notice
  4. Unity 2017.1 beta is now available for download.
    Dismiss Notice
  5. Unity 5.6 is now released.
    Dismiss Notice
  6. Get further faster with the Unity Plus Accelerator Pack, free for new Unity Plus subscribers for a limited time. Click here for more details.
    Dismiss Notice
  7. Check out all the fixes for 5.6 on the patch releases page.
    Dismiss Notice
  8. Learn how you'll soon be able to publish your games to China in four simple steps with Xiaomi. Sign up now for early access.
    Dismiss Notice

Settings for importing a video with an alpha channel

Discussion in '5.6 Beta' started by iflo, Feb 22, 2017.

  1. iflo

    iflo

    Joined:
    Jan 7, 2016
    Posts:
    2
    Hi guys/gals,

    I'm struggling to import a video into 5.6 that contains an alpha channel. I'm getting crashes with most of the formats I've tried and was wondering if someone knows the best format/codec to use.

    So far I've tried exporting as:
    Quicktime MOV - this errors with a WinfowsVideoMedia error 0xc00d5212 and prevents the transcode button from enabling.
    AVI animation codec - quits to desktop on click of the asset.
    MKV encoded to VP8 in Handbrake - quits to desktop on click of the asset.

    I've also tried a couple of different sizes and movie lengths just in case it's related to that.

    If you've managed to import an alpha channeled video I'd love to hear the settings you used!

    Thanks
     
  2. DominiqueLrx

    DominiqueLrx

    Unity Technologies

    Joined:
    Dec 14, 2016
    Posts:
    89
    Hi iflo!

    There are surprisingly few movie file formats that can contain alpha and are supported by the native video decoding libraries on the various Unity editor/runtime platforms.

    Furthermore, to get a full understanding on what to do in order to get clips with transparency in various contexts, you must realize the implication of choosing whether you ask Unity to transcode your clips or not:
    • If you choose to transcode the clip, Unity uses an internal layout that makes transparency work across the board, for both VP8 and H.264, which are the two video codecs the VideoClip importer knows how to transcode into.
    • If you choose not to transcode, it means you know the target platform(s) will natively support the clip's original format. You save time but it is up to you to verify the clip works as expected.
    Supported codecs with native capatility for transparency

    Apple ProRes 4444
    This is a proprietary codec made by Apple. It is natively supported on OSX and as such the Unity editor is able to decode these .mov clips on this editor platform as well as in the OSX standalone player. This is more of a work format however, and may be on the heavy side for distribution with your game. It has the advantage of being less compressed, meaning it preserves a good amount of quality when used as a transport format to transcode into another format later on. On Windows, since Apple discontinued the support of Quicktime for Windows, there is no builtin option to Apple ProRes 4444 on this platform.

    You can author movie files that use the Apple ProRes 4444 codec using applications found here: https://support.apple.com/en-us/HT200321

    Webm VP8 with alpha
    Webm is a royalty-free movie container format sponsored by Google. It supports the VP8 and VP9 video codecs, also royalty-free and owned by Google. Unity currently only supports VP8. For this codec, the webm container definition has an extension to allow embedding an alpha channel with the color stream. We have implemented this extension in Unity so your webm+vp8+alpha will be understood on all the editor platforms as well as most runtime platforms. A notable exception to this is Android, which has a native - possibly hardware-assisted - implementation for VP8 decoding but does not support the alpha extension, so here will most likely have to enable Unity's transcoding for its transparency to be available.

    There are not many applications that produce webm with vp8 and alpha, but thanks to tools such as ffmpeg, you can easily convert your source material when needed. See the section about ffmpeg later on.

    Unity's layout-based workaround to support transparency in VP8 and H.264

    In order to work around the difficulties of using standard clips with alpha in your Unity built game, Unity offers the "Keep Alpha" option in its VideoClip importer. When Unity detects the video has native alpha in it, the Keep Alpha option shows up in the VideoClip importer and if you enable it along with the transcode option, the resulting VideoClip asset will use an internal layout where the alpha is encoded in hidden color channels. During decoding at runtime, we recombine these hidden color channels with the actual color content on the GPU to yield a RGBA movie without needing any special decoding capabilities from the platform.

    An example

    So you can go ahead right now and download this clip in Unity on any editor platform: https://simpl.info/videoalpha/video/dancer1.webm

    Just drag-and-drop the movie in your Assets folder and you can use it without transcoding in your scene. Drag-and-droping the VideoClip asset onto the Main camera will cause the video to play on the Camera's far plane, leaving the skybox visible in the transparent portions of the movie.

    You can even just put the URL in the VideoPlayer Component's URL field (after you have set its Source popup to URL) and you'll be playing it from the web without needing to create an asset.

    Now, if you decide you target a platform that cannot read webm with transparency (e.g.: Android's hardware decoder does not support vp8 with alpha), what you can do instead is transcode. Go to the VideoClip importer for this movie file, enable "Keep Alpha", enable "Transcode", hit "Apply" and the resulting clip can then be used, with transparency fully working, in the VideoPlayer on any target platform. The downside to this method is that transcoding in Unity takes time and is currently not interruptible. Both of these aspects will be addressed later on.

    Converting with ffmpeg

    If for some reason you have movie files that have transparency but are in a format that is not supported by Unity, you can use an external tool such as ffmpeg (https://ffmpeg.org) to convert into any of the Unity-supported movie file formats that can include transparency. ffmpeg supports reading and writing a broad range of file types so it is a good fit for this task. It is not an easy tool to figure out, however, but there is plenty of examples available on the web when you search for "how do you do xyz with ffmpeg". Be ready for some command-line action, though.

    To convert a movie that you know has transparency into a webm with vp8 and alpha:

    ffmpeg -i my_clip_with_alpha.xyz -vcodec vp8 -pix_fmt yuva420p -metadata:s:v:0 alpha_mode="1" my_vp8_clip_with_alpha.webm

    To convert a movie that you know has transparency into a ProRes 4444:

    ffmpeg -i my_clip_with_alpha.xyz -vcodec prores_ks -pix_fmt yuva444p10le -alpha_bits 16 -profile:v 4444 -f mov my_prores_4444_clip_with_alpha.mov

    This is just a start

    So, as you can see, the topic of movie files with alpha is not a simple one but hopefully with this background and instructions, you'll be able to get going. Hopefully, as we gather feedback, we'll be able to streamline this process so it's more automatic and leaves less chance to mistakes and uncertainties.

    Hope this helps!

    Dominique
    A/V developer at Unity
     
    Last edited: Mar 9, 2017
  3. iflo

    iflo

    Joined:
    Jan 7, 2016
    Posts:
    2
    Hi Dominique, thank you so much for the comprehensive answer. This is perfect!

    The dancer clip worked perfectly as you suggested (great video btw)

    Using FFMPEG with the settings you supplied has also worked a treat on our own alpha channeled video, all playing beautifully in Android now. I just needed to add an extra -auto-alt-ref 0 parameter to allow ffmpeg to run.

    So once again, thank you for taking the time to respond. :)

    Flo
     
    cloclo and airenkuang like this.