Alternate Audio Tracks Tutorial

A common requirement in professional transcoding workflows is to replace the audio of a particular video with an alternate audio track (often in another language). This tutorial will show you how to perform that operation, as well as more complex substitutions. If you have not yet read the Stitching Tutorial, please do so now. This tutorial assumes that you already know the content of that tutorial.

In order to do complex operations like replacing or remapping audio tracks, we need to use the asset_complex object in our Hybrik JSON file. As a reminder, the complex_asset has a sub-element called an asset_versions array. This array contains all the media objects that will be assembled. Each media object is further broken down into an asset_components array. This array contains all of the sub-elements for a particular asset (video, audio, subtitles, etc.).

Swapping Audio Tracks

When you are assembling a source asset from multiple components, you need to tell Hybrik precisely how to interpret the various components. In the following JSON example, we are specifying a source asset that will be composed of the video from my_video_english.mov and the audio from a file called my_video_spanish.mp4:

{
  "uid": "source_file",
  "kind": "source",
  "payload": {
    "kind": "asset_complex",
    "payload": {
      "asset_versions": [
        {
          "location": {
            "storage_provider": "s3",
            "path": "s3://my_bucket/my_folder"
          },
          "asset_components": [
            {
              "kind": "name",
              "name": "my_video_english.mov",
              "contents": [
                {
                  "kind": "video"
                }
              ]
            },
            {
              "kind": "name",
              "name": "my_video_spanish.mp4",
              "contents": [
                {
                  "kind": "audio"
                }
              ]
            }
          ]
        }
      ]
    }
  }
}

In this example, you can see that there is a contents array for each of our asset_components. This array tells Hybrik which audio and video components to use. If a track is not specified in the contents array, then it is not used.

Audio Channel Mapping

The first example above takes all of the audio tracks from the my_video_spanish.mp4 and merges them with the video from my_video_english.mov. But what if you only wanted some of the audio tracks moved? Or if you wanted them moved and re-arranged? You can do that as well with Hybrik’s map object. The map object allows you to specify how tracks and channels in the input relate to tracks and channels in the output. See the Audio Mapping Tutorial for a more in-depth discussion of this topic.

Remember that each track of audio can have multiple channels. Often there are scenarios where source audio is delivered as separate mono tracks, but the output needs to be merged as channels of an output track. Let’s say that your source has two tracks of audio that you need to map to a single stereo track. That map object would look like this:

"map": [
  {
    "input": {
      "track": 0,
      "channel": 0
    },
    "output": {
      "track": 0,
      "channel": 0
    }
  },
  {
    "input": {
      "track": 1,
      "channel": 0
    },
    "output": {
      "track": 0,
      "channel": 1
    }
  }
]

We can use that map object in our source element as follows:

{
  "uid": "source_file",
  "kind": "source",
  "payload": {
    "kind": "asset_url",
    "payload": {
      "storage_provider": "s3",
      "url": "s3://my_bucket/my_folder/my_video.mp4",
      "contents": [
        {
          "kind": "video"
        },
        {
          "kind": "audio",
          "map": [
            {
              "input": {
                "track": 0,
                "channel": 0
              },
              "output": {
                "track": 0,
                "channel": 0
              }
            },
            {
              "input": {
                "track": 1,
                "channel": 0
              },
              "output": {
                "track": 0,
                "channel": 1
              }
            }
          ]
        }
      ]
    }
  }
}

Adding Multiple Audio Tracks

In the first example, we replaced the original English audio with an alternative Spanish audio. What if instead we had wanted to ADD the Spanish audio as an additional track. So now we would want our output to have 2 tracks of stereo audio – the first track English and second track Spanish. In the example above, we only included the video component of the English version. We will now want to include the audio component as well. Further, we want to tell Hybrik that the English should be in track 0 and the Spanish should be in track 1. We can do this using the map function we used above. The asset_components therefore looks like this:

"asset_components": [
  {
    "kind": "name",
    "name": "my_video_english.mov",
    "contents": [
      {
        "kind": "video"
      },
      {
        "kind": "audio",
        "map": [
          {
            "input": {
              "track": 0
            },
            "output": {
              "track": 0
            }
          }
        ]
      }
    ]
  },
  {
    "kind": "name",
    "name": "my_video_spanish.mp4",
    "contents": [
      {
        "kind": "audio",
        "map": [
          {
            "input": {
              "track": 0
            },
            "output": {
              "track": 1
            }
          }
        ]
      }
    ]
  }
]
...

Alternate Audio Tracks Examples

• Audio Swap Tracks Remap example
• More advanced audio mapping in our Audio Mapping Tutorial