Segmented Rendering

segmented_rendering gives us the ability to temporally segment our source and encode it in chunks. This allows us to transcode the chunks across multiple computing instances and gain the speed benefits of distributed encoding. The transcoded chunks are then automatically combined into a single output file as the final step of the transcode task.

This is especially helpful on long-running encodes for two reasons. Not only can you spread your encode across tens or hundreds of machines for a speed benefit, but in the event of a spot instance being taken away during encoding, only that segment is required to be re-encoded for the entire job to be completed.

We configure our job to use segmented rendering via a “Job Modifier”

Example Job Modifier Task

Here is an example snippet for how to use segmented rendering. You set the kind of the modifier to transcode_segmented_rendering and tell the task which transcode task to segment by target_element_uid.

This “Job Modifier Task” is connected in the connections array like any other task. Usually the job flow will flow from source, to job_modifier, to transcode. The duration of each segment is specified in seconds. See the examples at the bottom for a full working example.

modifiers is an array, you can have multiple modifiers in a single job_modifier task. For example if you have an 8-bit avc transcode task and 10-bit hevc transcode task in the same job, you would want to segment both tasks in the job modifier. A single job cannot have multipe job_modifier tasks. Instead, you want to have multiple modifiers in a single job_modifier task.

{ 
    "uid": "segment_script_task",
    "kind": "script",
    "payload": {
        "kind": "job_modifier",
        "payload": {
            "modifiers": [
                {
                    "kind": "transcode_segmented_rendering",
                    "target_element_uid": "transcode_task",
                    "duration_sec": "{{segmented_rendering_duration_sec}}"
                }
            ]
        }
    }
}, 

Segment Size

Increasing the number of segments by using a shorter segment duration will distribute the transcoding to more machines, and can decrease total transcode time as they run in parallel. However, the overall quality of your video can suffer if the segment duration is too short. This is because some video codecs - such as long-gop h264 and h265 - perform best when longer sections of video are encoded. The goal is to set the segment duration long enough for the encoder to make good predictions and encoding decisions. Therefore, it is best to run some tests using your typical source files to find a sweet spot with the segment duration. We recommend starting with a segment duration between 60 and 180 seconds as a starting point.

Limitations

• There is limited format support (h264, h265, ProRes)
• Audio is not chunked and is transcoded as a single thread. Typically this is acceptable as audio transcodes perform much faster than video
• Source frame_rate and target frame_rate must match
• Using stitched sources is supported only if all sources have the same frame rate

Example Job

• segmented_rendering example