Analyze and Quality Control Tutorials
Hybrik supports both analysis and quality control of media content. Automated analysis and QC can be executed prior to transcoding to ensure that the incoming content meets the required standards, as well as after transcoding to ensure that the results meet the expected output criteria. You can also execute jobs that only do analysis without doing any subsequent transcoding.
Hybrik separates analysis and quality control into separate Analyze and QC Tasks. The analyze
task will report the requested data about the file, while the qc
task makes a pass/fail decision for your content by comparing the metadata reported by the Analyze Task with criteria you supply. For example, the Analyze Task could report that there are five seconds of black at the beginning of a video. This might be correct for your requirements, or it might be incorrect. Using the QC Task, you would specify what are the allowable values for each type of analysis.
There are a variety of different types of analysis that Hybrik can perform on your media, and some of them may be inappropriate (or unnecessary) for your media type. Some analysis tasks can take significant time to run, so Hybrik allows you to specify what types of analysis you would like to perform on your content.
In this tutorial, we’ll create a job that runs an Analysis Task on the source file, a QC Task that ensures the source file meets a set of acceptance criteria that we’ll define, and then performs a Transcode Task if the criteria have been met. This tutorial assumes you are already familiar with the basic Hybrik JSON; if you have not already done so, please take a moment to review the Hybrik JSON Tutorial.
General Properties
There are two categories of properties that can be specified in an analyze
task: general_properties
and deep_properties
. The general properties are those that can be extracted from the header content of the file. Deep properties require that the individual frames of the media be decompressed and analyzed, which takes more time.
The available general properties of a media file include:
- Container
- Kind
- Faststart
- Duration
- Size
- Overall bitrate
- Video
- PID
- Codec
- Profile
- Level
- Bitrate mode (CBR, VBR, etc.)
- Video stream bitrate
- Frame rate
- Frame dimensions
- Interlacing
- Display Aspect Ratio
- Pixel Aspect Ratio
- Chroma and color space info
- Duration
- etc
- Audio (one for each audio track)
- PID
- Codec
- Bitrate info
- Sample Rate
- Channel count
- Duration
- Profile
- etc
When you specify that you would like analysis of the general_properties
of a file, Hybrik will return all of these parameters.
Deep Properties
The deep_properties
of a media file require examining each frame (both audio and video) of the file to determine the specific property. This often requires both analyzing the frame and comparing it with other frames in the file. The deep properties are generally configurable, allowing you to set specific thresholds for detection and control the sensitivity of the analysis.
For example, there is a deep property to identify black frames within the video. In the qc
task definition, you may select both what percentage of pixels in the frame have to be black in order for the whole frame to be considered black, as well as the brightness value that registers that a pixel is black.
In the case of black detection, you must determine the boundary conditions for that particular analysis – e.g., if a frame is almost entirely black, but has a few pixels that are dark gray, does it count as a black frame?
The available deep properties include:
- Video
- Black
- detect frames that are black within the video
- Black borders
- detect letter- or pillar-boxing
- Interlace mode
- analyze frame-by-frame for interlacing artifacts
- Levels
- analyze the video for min/max, Y, Cb, Cr, etc.
- Blockiness
- detect compression block artifacts
- HDR Levels
- calculate the MaxFALL and MaxCLL for HDR encoding
- Source complexity
- measure for how complex the content is over time
- Content variance
- analyze how much the content changes over time
- Scene change detection
- detect scene changes probabilities
- PSE
- detect Photo-Sensitive Epilepsy (PSE) artifacts
- Compressed Statistics
- determine compressed frame sizes etc.
- Compressed Quality
- determine quality of the underlying bitstream like PQ values etc.
- SSIM
- determine the SSIM value between the media and a reference
- PSNR
- determine the PSNR value between the media and a reference
- VMAF
- use the Netflix VMAF method to compare the media and a reference
- Black
- Audio
- Levels
- DC offset, RMS peak, etc.
- EBU R.128
- loudness analysis
- Dolby Professional Loudness
- Perform Dolby loudness analysis on the audio
- Volume
- simple volume measurement
- Silence
- detect silent segments in the audio tracks
- PSNR
- compare the asset to a reference audio track
- Emergency Alert
- detect emergency alert signals in the audio
- Channel Compositions
- determine channel composition in the audio
- Levels
Analyze Task JSON
When creating an analyze
Task, you must specify which properties are going to be analyzed. In the example below, we are specifying that we want the general properties to be analyzed, as well as the following deep properties: audio levels, silence, black video, and black borders. More specifically, we are also saying that in order for a segment to be considered “silent”, it must be silent, under -60 dB, for more than 1 second. For a segment to be considered “black”, less than it must be black for more than 1 second. This allows us, for example, to skip over false positive results for brief moments of silence or black in our media.
NOTE: In Hybrik version 1.217, we introduced a change to the structure of the analyzer results that have timed events. The new result version can be activated by setting "response_version": 2
in your analyzer's options. The default version will become version 2 in a future release. If you need the documentation for the legacy version of this tutorial, please visit the legacy link.
{
"uid": "analyze_task",
"kind": "analyze",
"payload": {
"options": {
"response_version": 2
},
...
}
{
"uid": "analyze_task",
"kind": "analyze",
"payload": {
"general_properties": {
"enabled": true
},
"deep_properties": {
"audio": {
"levels": {
"enabled": true,
"is_optional": true
},
"silence": {
"enabled": true,
"is_optional": true,
"noise_db": -60,
"duration_sec": 1
}
},
"video": {
"black": {
"enabled": true,
"is_optional": true,
"black_level": 0.03,
"duration_sec": 1
},
"black_borders": {
"enabled": true,
"is_optional": true
}
}
}
}
}
Any time we do not specify a property in the analysis, Hybrik will use the default values. In the case of the black detection, our default value for black pixel detection is 0.1
(i.e., 10% of white). In terms of the number of pixels on a screen that must be black (black_pixel_ratio
), we default to 98%. You could change both of these values in the JSON by setting them explicitly, as with this example:
{
"black": {
"enabled": true,
"black_level": 0.2,
"black_pixel_ratio": 0.99,
"duration_sec": 2.0
}
}
Please see the Hybrik API documentation for a complete list of the available settings for each analysis type.
Analyze Task Results
When Hybrik completes a job, it returns a JSON structure with all of the data that resulted from the job. Each task returns its own section of JSON. You can access this data either programmatically using the Hybrik API or as a file via the Hybrik UI. If you are using the UI, go to the Completed Jobs view, select a specific job, and then choose “Export Job Summary”. This will export a JSON file which contains the summary of the job. This file may contain a significant amount of information, including:
- Any errors
- Job statistics
- Execution time
- Task count
- Retries
- etc
- For each task
- Machine info
- CPU utilization
- Memory usage
- etc
- Task statistics
- Execution Time
- Start
- Stop
- Retries
- etc
- Task Results
You can search in the job summary for the “Analyzer” section, which is where Analyzer results are returned. If you have multiple Analyze tasks in your workflow (for example, for both the input and output video), then your Job Summary will have multiple Analyze results. The results are broken into the following sections:
probes
- this lists all the analysis components that were run
general_properties
- the general properties of the file
deep_properties
- the deep properties of the file – each sub property is in its own object
The “General Properties” object looks like the following
{
"general_properties": {
"container": {
"duration_sec": 60.022,
"bitrate_kb": 1312.112,
"size_kb": 9614,
"kind": "mp4",
"faststart": false
},
"audio": [
{
"pid": 2,
"duration_sec": 60.022,
"codec": "aac_lc",
"bitrate_mode": "cbr",
"bitrate_kb": 127.628,
"sample_rate": 48000,
"channels": 2,
"language": "en"
}
],
"video": {
"pid": 1,
"duration_sec": 60.019,
"codec": "h264",
"profile": "High",
"level": "L3",
"frame_rate": 23.976023976023978,
"bitrate_kb": 1180.352,
"height": 480,
"width": 848,
"frame_rate_mode": "constant",
"interlace_mode": "progressive",
"dar": 1.767,
"par": 1,
"clean_aperture_height": 480,
"clean_aperture_width": 848,
"chroma_format": "cs_420",
"bit_resolution": 8
}
}
}
general_properties
has container
, audio
, and video
sections. The container
section contains general information about the media container. You will notice that the audio section is an array, and there will be one array object for each audio track. The video section gives you basic information about the video track.
deep_properties
has an object for each type of audio or video analysis performed. Remember that each type of desired analysis must be specified. Deep property analysis tasks can be quite CPU intensive, so Hybrik only performs the deep analysis tasks specified in the job.
Each analysis will return a set of values, often including statistics like the mean value, standard deviation, minimum value, maximum value, etc. These values can be helpful in your QC process. Some analysis types also return charting data, which is a sampling of the specified value over time. Charting data is returned as a set of 100 values that are evenly spaced out over the duration of the file.
{
"deep_properties": {
"video": {
"levels": {
"frames": 1439,
"bpp": 8,
"y": {
"broadcast_safe": false,
"mean": 121.2,
"stddev": 12.58,
"mean_frame_max": 154.6,
"min": 37,
"min_pos_frame": 901,
"min_pos_sec": 37.6,
"max": 255,
"max_pos_frame": 829,
"max_pos_sec": 34.6
},
"charts": [
{
"datasets": [
{
"data": [
125.55,
126.613,
...
Quality Control Task
The qc
task allows you define a set of conditions that an asset needs to meet, and evaluates the results of an analyze
task using those conditions. If all of the supplied conditions are not met, the QC Task will fail. QC Tasks can evaluate parameters associated with the file format/wrapper/container, the video stream, or the audio streams, depending on the type of criteria being tested.
Each QC test has a condition
string, which defines the test using an expression compatible with the syntax used in the math.js JavaScript library. In addition the QC test can include message_pass
and message_fail
messages that are inserted into log files and reports.
For example, if we want to ensure that a file is not missing audio tracks, we could verify that the file contains at least one audio track by testing that the number of channels detected in the first audio track (audio[0]
) during the analyze
Task is greater than or equal to 1:
{
"uid": "qc_task",
"kind": "qc",
"payload": {
"tests": [
{
"conditions": [
{
"condition": "general_properties.audio.size() >= 1",
"message_pass": "PASS: Number of audio tracks is greater than or equal to 1, there are {general_properties.audio.size()} tracks",
"message_fail": "FAIL: There are no audio tracks."
}
]
}
]
}
},
Below is a complete qc
task that performs the following evaluations against the source file:
- File size is greater than 10 megabytes
- Video width is at least 1920 pixels
- Video height is at least 1080 pixels
- Number of audio channels for the first audio track is at least 1
- Audio sample rate is more than 44.1kHz
- Any there is less than 5 seconds of black at the start and end of the video
- Letterboxing (black borders at top and bottom) is not present
- Average audio RMS level is at least -60db
- Any detected periods of silent audio are less than 5 seconds long
Remember that whenever you create a qc
task, you will also need to create a matching analyze
task that generates all the values that the qc
task will validate.
Regarding audio tracks and channels, the code below only evaluates the first track (the value 0
in general_properties.audio[0].channels
), to evaluate other tracks replace the 0
with another integer.
{
"uid": "qc_task",
"kind": "qc",
"payload": {
"tests": [
{
"conditions": [
{
"condition": "general_properties.container.size_kb > 10000",
"message_pass": "PASS: Source file size is greater than 10 megabytes, size = {general_properties.container.size_kb} kb.",
"message_fail": "FAIL: Source file size is less than or equal to 10 megabytes, size = {general_properties.container.size_kb} kb."
},
{
"condition": "general_properties.video.width >= 1920",
"message_pass": "PASS: Video frame width is greater than or equal to 1920, width is {general_properties.video.width} pixels.",
"message_fail": "FAIL: Video frame width is less than 1920, width is {general_properties.video.width} pixels."
},
{
"condition": "general_properties.video.height >= 1080",
"message_pass": "PASS: Video frame height is greater than or equal to 1080, height is {general_properties.video.height} pixels.",
"message_fail": "FAIL: Video frame height is less than 1080, height is {general_properties.video.height} pixels."
},
{
"condition": "general_properties.audio.size() >= 1",
"message_pass": "PASS: Number of audio tracks is greater than or equal to 1, there are {general_properties.audio.size()} tracks",
"message_fail": "FAIL: There are no audio tracks."
},
{
"condition": "general_properties.audio[0].sample_rate >= 44100",
"message_pass": "PASS: Audio sample rate is greater than or equal to 44.1 kHz, sample rate is {general_properties.audio[0].sample_rate} Hz.",
"message_fail": "FAIL: Audio sample rate is less than 44.1 kHz, sample rate is {general_properties.audio[0].sample_rate} Hz."
},
{
"condition": "deep_properties.video.black.leading_sec <= 5",
"message_pass": "PASS: There is less than 5 seconds of black at the start of this video. Actual value: {deep_properties.video.black.leading_sec}",
"message_fail": "FAIL: There is more than 5 seconds of black at the start of this video. Actual value: {deep_properties.video.black.leading_sec}"
},
{
"condition": "deep_properties.video.black.trailing_sec <= 5",
"message_pass": "PASS: There is less than 5 seconds of black at the end of this video. Actual value: {deep_properties.video.black.trailing_sec}",
"message_fail": "FAIL: There is more than 5 seconds of black at the end of this video. Actual value: {deep_properties.video.black.trailing_sec}"
},
{
"condition": "deep_properties.video.black_borders.top == 0 && deep_properties.video.black_borders.bottom == 0",
"message_pass": "PASS: Video letterboxing is 0 pixels.",
"message_fail": "FAIL: Video letterboxing is more than 0 pixels, top = {deep_properties.video.black_borders.top} pixels, bottom = {deep_properties.video.black_borders.bottom} pixels."
},
{
"condition": "deep_properties.video.black_borders.left == 0 && deep_properties.video.black_borders.right == 0",
"message_pass": "PASS: Video pillarboxing is 0 pixels.",
"message_fail": "FAIL: Video pillarboxing is more than 0 pixels, left = {deep_properties.video.black_borders.left} pixels, right = {deep_properties.video.black_borders.right} pixels."
},
{
"condition": "deep_properties.audio[0].levels.rms_level_db >= -60",
"message_pass": "PASS: Audio RMS level is higher than or equal to -60 dB, level is {deep_properties.audio[0].levels.rms_level_db} dB.",
"message_fail": "FAIL: Audio level is lower than -60dB, level is {deep_properties.audio[0].levels.rms_level_db} dB."
},
{
"condition": "deep_properties.audio[0].silence.events.size() == 0 || deep_properties.audio[0].silence.max_sec <= 5",
"message_pass": "PASS: Longest audio silence is shorter than 5 seconds.",
"message_fail": "FAIL: At least one audio silence is longer than or equal to 5 seconds: {deep_properties.audio[0].silence.max_sec}"
}
]
}
]
}
}
Some of these tests are dependent on specific deep properties from the analyze
Task. The video.black
, video.black_borders
, audio[].levels.rms_level
, and audio[].silence[]
tests would fail if the Analysis Task did not include the video.black
, video.black_borders
audio.levels
, and audio.silence
analyses in the deep_properties
section.
In our connections
array, we connect the transcode
Task to the success state of the qc
task, meaning that the transcode
Task is triggered only if all of the tests within the QC Task have passed. We’ve connected an email notification to the error state of the QC Task, so that the transcode
Task will not be triggered and someone can be notified to remedy the issue.
For more examples of conditions, see the Conditionals Tutorial.
Quality Control Task Results
Results of the comparisons made in a qc
Task are available in the task result JSON. In the example below, the qc
Task failed because one criterion wasn’t met - letterboxing bars measuring 140 pixels high were detected. The rest of the test passed: file size, video width/height, number of audio tracks, audio sample rate, black frame duration, pillarboxing bars, audio level, and audio silence were all within the limits we defined.
The failure of any condition will cause the job to fail and the message will come from the failure test case:
{
"errors": [
{
...
"result_define": "SYS_TSK_QC_FAILURE",
"message": "qc checks failed",
"details": "FAIL: Video letterboxing is more than 0 pixels, top = 140 pixels, bottom = 140 pixels.",
"diagnostic": {
"results": [
...
Each condition returned from the QC task shows the condition tested and the message
displays the pre-defined message based on whether the test passes or fails. You can also see explicitly how the condition was evaluated in the condition_evaluation
and the values
array will return the results of what the calculated value used for comparison was:
"results": [
{
"tests": [
{
"test_result": "Fail",
"conditions": [
{
"condition": "general_properties.container.size_kb > 10000",
"message_pass": "PASS: Source file size is greater than 10 megabytes, size = {general_properties.container.size_kb} kb.",
"message_fail": "FAIL: Source file size is less than or equal to 10 megabytes, size = {general_properties.container.size_kb} kb.",
"condition_evaluation": true,
"message": "PASS: Source file size is greater than 10 megabytes, size = 10251983 kb.",
"values": [
{
"parameter": "general_properties.container.size_kb",
"value": 10251983
}
]
},
{
"condition": "general_properties.video.width >= 1920",
"message_pass": "PASS: Video frame width is greater than or equal to 1920, width is {general_properties.video.width} pixels.",
"message_fail": "FAIL: Video frame width is less than 1920, width is {general_properties.video.width} pixels.",
"condition_evaluation": true,
"message": "PASS: Video frame width is greater than or equal to 1920, width is 1920 pixels.",
"values": [
{
"parameter": "general_properties.video.width",
"value": 1920
}
]
},
{
"condition": "general_properties.video.height >= 1080",
"message_pass": "PASS: Video frame height is greater than or equal to 1080, height is {general_properties.video.height} pixels.",
"message_fail": "FAIL: Video frame height is less than 1080, height is {general_properties.video.height} pixels.",
"condition_evaluation": true,
"message": "PASS: Video frame height is greater than or equal to 1080, height is 1080 pixels.",
"values": [
{
"parameter": "general_properties.video.height",
"value": 1080
}
]
},
{
"condition": "general_properties.audio.size() >= 1",
"message_pass": "PASS: Number of audio tracks is greater than or equal to 1, there are {general_properties.audio.size()} tracks",
"message_fail": "FAIL: There are no audio tracks.",
"condition_evaluation": true,
"message": "PASS: Number of audio tracks is greater than or equal to 1, there are 4 tracks",
"values": [
{
"parameter": "general_properties.audio.size()",
"value": 4
}
]
},
{
"condition": "general_properties.audio[0].sample_rate >= 44100",
"message_pass": "PASS: Audio sample rate is greater than or equal to 44.1 kHz, sample rate is {general_properties.audio[0].sample_rate} Hz.",
"message_fail": "FAIL: Audio sample rate is less than 44.1 kHz, sample rate is {general_properties.audio[0].sample_rate} Hz.",
"condition_evaluation": true,
"message": "PASS: Audio sample rate is greater than or equal to 44.1 kHz, sample rate is 48000 Hz.",
"values": [
{
"parameter": "general_properties.audio[0].sample_rate",
"value": 48000
}
]
},
{
"condition": "deep_properties.video.black.leading_sec <= 5",
"message_pass": "PASS: There is less than 5 seconds of black at the start of this video. Actual value: {deep_properties.video.black.leading_sec}",
"message_fail": "FAIL: There is more than 5 seconds of black at the start of this video. Actual value: {deep_properties.video.black.leading_sec}",
"condition_evaluation": true,
"message": "PASS: There is less than 5 seconds of black at the start of this video. Actual value: 0",
"values": [
{
"parameter": "deep_properties.video.black.leading_sec",
"value": 0
}
]
},
{
"condition": "deep_properties.video.black.trailing_sec <= 5",
"message_pass": "PASS: There is less than 5 seconds of black at the end of this video. Actual value: {deep_properties.video.black.trailing_sec}",
"message_fail": "FAIL: There is more than 5 seconds of black at the end of this video. Actual value: {deep_properties.video.black.trailing_sec}",
"condition_evaluation": true,
"message": "PASS: There is less than 5 seconds of black at the end of this video. Actual value: 0",
"values": [
{
"parameter": "deep_properties.video.black.trailing_sec",
"value": 0
}
]
},
{
"condition": "deep_properties.video.black_borders.top == 0 && deep_properties.video.black_borders.bottom == 0",
"message_pass": "PASS: Video letterboxing is 0 pixels.",
"message_fail": "FAIL: Video letterboxing is more than 0 pixels, top = {deep_properties.video.black_borders.top} pixels, bottom = {deep_properties.video.black_borders.bottom} pixels.",
"condition_evaluation": false,
"message": "FAIL: Video letterboxing is more than 0 pixels, top = 140 pixels, bottom = 140 pixels.",
"values": [
{
"parameter": "deep_properties.video.black_borders.top",
"value": 140
},
{
"parameter": "deep_properties.video.black_borders.bottom",
"value": 140
}
]
},
{
"condition": "deep_properties.video.black_borders.left == 0 && deep_properties.video.black_borders.right == 0",
"message_pass": "PASS: Video pillarboxing is 0 pixels.",
"message_fail": "FAIL: Video pillarboxing is more than 0 pixels, left = {deep_properties.video.black_borders.left} pixels, right = {deep_properties.video.black_borders.right} pixels.",
"condition_evaluation": true,
"message": "PASS: Video pillarboxing is 0 pixels.",
"values": [
{
"parameter": "deep_properties.video.black_borders.left",
"value": 0
},
{
"parameter": "deep_properties.video.black_borders.right",
"value": 0
}
]
},
{
"condition": "deep_properties.audio[0].levels.rms_level_db >= -60",
"message_pass": "PASS: Audio RMS level is higher than or equal to -60 dB, level is {deep_properties.audio[0].levels.rms_level_db} dB.",
"message_fail": "FAIL: Audio level is lower than -60dB, level is {deep_properties.audio[0].levels.rms_level_db} dB.",
"condition_evaluation": true,
"message": "PASS: Audio RMS level is higher than or equal to -60 dB, level is -15.357603 dB.",
"values": [
{
"parameter": "deep_properties.audio[0].levels.rms_level_db",
"value": -15.357603
}
]
},
{
"condition": "deep_properties.audio[0].silence.events.size() == 0 || deep_properties.audio[0].silence.max_sec <= 5",
"message_pass": "PASS: Longest audio silence is shorter than 5 seconds.",
"message_fail": "FAIL: At least one audio silence is longer than or equal to 5 seconds: {deep_properties.audio[0].silence.max_sec}",
"condition_evaluation": true,
"message": "PASS: Longest audio silence is shorter than 5 seconds.",
"values": [
{
"parameter": "deep_properties.audio[0].silence.events.size()",
"value": 1
},
{
"parameter": "deep_properties.audio[0].silence.max_sec",
"value": 1.4267
}
]
}
]
}
]
}
]
}
Examples
- Analyze and QC example
- Analyze and QC example with PDF reports
- For more examples of conditions, see the Conditionals Tutorial