Task Graphs

Dask internally converts the algorithms into a graph format made of python dicts, tuples, and functions.

For example,

dsk = {
 'x': 1,
 'y': 2,
 'z': (add, 'x', 'y')
}

A key can be a str, bytes, float, or a tuple.

A task is a tuple with first element as callable and rest are arguments. Arguments again can be any other key, literal value or any computations.

So,

Key
---
'x'
('x', 1, 2) 


Tasks
----
(add, 'x', 'y')
(sum [1, 2, 3], 2)

A computation can be

• a literal value such as 1.
• a task
• any key present in dask graph
• a list of computations such as ['x', 1, (add 1, 2)]

Evaluate Graph

get method from dask is served as entry point for the execution. It computes the value of the key from the dask graph.

from dask.threaded import get
 
get(dsk, 'x')
 
1

get takes the dask graph and key to compute.