Connections Array

The connections array in a Hybrik job (often at the bottom of the job json) is how you direct the flow of tasks, routing tasks to one or more other tasks. You might route a source to a transcode or an analyze task. You might take the results of 1 or more transcode tasks and route them to a package task. If a job fails, you might want to send a special notification and route to a notify task.

It is possible to connect a single task to one or more tasks. Or you can have a task depend on multiple other tasks before executing.

A Simple example

At it’s most basic form, the connections array will route from a source to a transcode task

"connections": [
    {
        "from": [
            {
                "element": "source"
            }
        ],
        "to": {
            "success": [
                {
                    "element": "transcode_task"
                }
            ]
        }
    }
]

From the source element to additional tasks. Common routes are:

Multiple Routes and Concurrency

Here is an example with multiple connections. In this example the routing looks like:

Here is some psuedocode for how it works, starting from the qc_task:

if (qc_task succeeds)
  proceed to (transcode_a && transcode_b)
    if ((transcode_a && transcode_b) == "success")
      proceed to transcode_success_notify
    else if (either transcode_task fails)
      proceed to transcode_fail_notify
else if (qc_task fails)
  proceed to qc_error_notify_task

Because the from and to routes are arrays, you can route from one task to multiple tasks which can run concurrently.

In the following example

• a source file is first analyzed for general video/audio properties in one analyze task.
• Then the qc task is run.
• If the conditions in the qc task are met (PASS), two separate transcode tasks will start.
• If the conditions of the qc task are not met (FAIL/ERROR), the QC Error notify task is run.
• The combined results of the two transcode tasks will determine which of the final notify tasks are run.

"connections": [
    {
      "from": [
        {
          "element": "source"
        }
      ],
      "to": {
        "success": [
          {
            "element": "analyze_general"
          }
        ]
      }
    },
    {
      "from": [
        {
          "element": "analyze_general"
        }
      ],
      "to": {
        "success": [
          {
            "element": "qc_task"
          }
        ]
      }
    },
    {
      "from": [
        {
          "element": "qc_task"
        }
      ],
      "to": {
        "success": [
          {
            "element": "transcode_a"
          },
          {
            "element": "transcode_b"
          }
        ],
        "error": [
          {
            "element": "qc_error_notify_task"
          }
        ]
      }
    },
    {
      "from": [
        {
          "element": "transcode_a"
        },
        {
          "element": "transcode_b"
        }
      ],
      "to": {
        "success": [
          {
            "element": "transcode_success_notify"
          }
        ],
        "error": [
          {
            "element": "transcode_fail_notify"
          }
        ]
      }
    }
  ]

Operational Notes

• When a from connection routes to 2 or more to tasks, if any of these tasks fail, the entire job fails and any downstream tasks in the job will not be run.
• If a connection is reached on error, the “virtual document” that is normally passed between tasks (containing source and job data/elements) will not be passed to the next to connection. Only an error message is passed.

Examples

• Connections array example