bitPhile

visualize-task-graphs

4 min read

import dask.array as da
import dask

x = da.ones((15, 15), chunks=(5, 5))

y = x.sum()

dask.visualize(y)

png

y.visualize(rankdir='BT')

png

dask.visualize(y, optimize_graph=True)

png

Visualize High level dask graph

y.dask.visualize(filename='test.svg')

svg

High level graph HTML representation

y.dask

HighLevelGraph

HighLevelGraph with 4 layers and 23 keys from all layers. 

        <div style="">
<svg width="24" height="24" viewBox="0 0 32 32" fill="none" xmlns="http://www.w3.org/2000/svg" style="position: absolute;">

    <circle cx="16" cy="16" r="14" fill="#8F8F8F" style="stroke: var(--jp-ui-font-color2, #1D1D1D);" stroke-width="2"/>

</svg>

<details style="margin-left: 32px;">
    <summary style="margin-bottom: 10px; margin-top: 10px;">
        <h4 style="display: inline;">Layer1: ones_like</h4>
    </summary>
    <p style="color: var(--jp-ui-font-color2, #5D5851); margin: -0.25em 0px 0px 0px;">
        ones_like-44817800100989a0a789e6d1be702b8f
    </p>

    <table>
    <tr>
        <td>
            <table>

                <tr>
                    <th style="text-align: left; width: 150px;">layer_type</th>
                    <td style="text-align: left;">Blockwise</td>
                </tr>

                <tr>
                    <th style="text-align: left; width: 150px;">is_materialized</th>
                    <td style="text-align: left;">True</td>
                </tr>

                <tr>
                    <th style="text-align: left; width: 150px;">number of outputs</th>
                    <td style="text-align: left;">9</td>
                </tr>

                <tr>
                    <th style="text-align: left; width: 150px;">shape</th>
                    <td style="text-align: left;">(15, 15)</td>
                </tr>

                <tr>
                    <th style="text-align: left; width: 150px;">dtype</th>
                    <td style="text-align: left;">float64</td>
                </tr>

                <tr>
                    <th style="text-align: left; width: 150px;">chunksize</th>
                    <td style="text-align: left;">(5, 5)</td>
                </tr>

                <tr>
                    <th style="text-align: left; width: 150px;">type</th>
                    <td style="text-align: left;">dask.array.core.Array</td>
                </tr>

                <tr>
                    <th style="text-align: left; width: 150px;">chunk_type</th>
                    <td style="text-align: left;">numpy.ndarray</td>
                </tr>


            </table>
        </td>
        <td>
            <svg width="250" height="250" style="stroke:rgb(0,0,0);stroke-width:1" >

15 15 

</details>

        <div style="">
<svg width="24" height="24" viewBox="0 0 32 32" fill="none" xmlns="http://www.w3.org/2000/svg" style="position: absolute;">

    <circle cx="16" cy="16" r="14" fill="#8F8F8F" style="stroke: var(--jp-ui-font-color2, #1D1D1D);" stroke-width="2"/>

</svg>

<details style="margin-left: 32px;">
    <summary style="margin-bottom: 10px; margin-top: 10px;">
        <h4 style="display: inline;">Layer2: sum</h4>
    </summary>
    <p style="color: var(--jp-ui-font-color2, #5D5851); margin: -0.25em 0px 0px 0px;">
        sum-1eee5d86c351bbfd9246db6fcdd2ab33
    </p>

    <table>
    <tr>
        <td>
            <table>

                <tr>
                    <th style="text-align: left; width: 150px;">layer_type</th>
                    <td style="text-align: left;">Blockwise</td>
                </tr>

                <tr>
                    <th style="text-align: left; width: 150px;">is_materialized</th>
                    <td style="text-align: left;">True</td>
                </tr>

                <tr>
                    <th style="text-align: left; width: 150px;">number of outputs</th>
                    <td style="text-align: left;">9</td>
                </tr>

                <tr>
                    <th style="text-align: left; width: 150px;">shape</th>
                    <td style="text-align: left;">(15, 15)</td>
                </tr>

                <tr>
                    <th style="text-align: left; width: 150px;">dtype</th>
                    <td style="text-align: left;">float64</td>
                </tr>

                <tr>
                    <th style="text-align: left; width: 150px;">chunksize</th>
                    <td style="text-align: left;">(5, 5)</td>
                </tr>

                <tr>
                    <th style="text-align: left; width: 150px;">type</th>
                    <td style="text-align: left;">dask.array.core.Array</td>
                </tr>

                <tr>
                    <th style="text-align: left; width: 150px;">chunk_type</th>
                    <td style="text-align: left;">numpy.ndarray</td>
                </tr>



                    <tr>
                        <th style="text-align: left; width: 150px;"> depends on </th>
                        <td style="text-align: left;">ones_like-44817800100989a0a789e6d1be702b8f</td>
                    </tr>


            </table>
        </td>
        <td>
            <svg width="250" height="250" style="stroke:rgb(0,0,0);stroke-width:1" >

15 15 

</details>

        <div style="">
<svg width="24" height="24" viewBox="0 0 32 32" fill="none" xmlns="http://www.w3.org/2000/svg" style="position: absolute;">

    <circle cx="16" cy="16" r="14" fill="#8F8F8F" style="stroke: var(--jp-ui-font-color2, #1D1D1D);" stroke-width="2"/>

</svg>

<details style="margin-left: 32px;">
    <summary style="margin-bottom: 10px; margin-top: 10px;">
        <h4 style="display: inline;">Layer3: sum-partial</h4>
    </summary>
    <p style="color: var(--jp-ui-font-color2, #5D5851); margin: -0.25em 0px 0px 0px;">
        sum-partial-0259629db3a875f0182cc02372216a68
    </p>

    <table>
    <tr>
        <td>
            <table>

                <tr>
                    <th style="text-align: left; width: 150px;">layer_type</th>
                    <td style="text-align: left;">MaterializedLayer</td>
                </tr>

                <tr>
                    <th style="text-align: left; width: 150px;">is_materialized</th>
                    <td style="text-align: left;">True</td>
                </tr>

                <tr>
                    <th style="text-align: left; width: 150px;">number of outputs</th>
                    <td style="text-align: left;">4</td>
                </tr>

                <tr>
                    <th style="text-align: left; width: 150px;">shape</th>
                    <td style="text-align: left;">(2, 2)</td>
                </tr>

                <tr>
                    <th style="text-align: left; width: 150px;">dtype</th>
                    <td style="text-align: left;">float64</td>
                </tr>

                <tr>
                    <th style="text-align: left; width: 150px;">chunksize</th>
                    <td style="text-align: left;">(1, 1)</td>
                </tr>

                <tr>
                    <th style="text-align: left; width: 150px;">type</th>
                    <td style="text-align: left;">dask.array.core.Array</td>
                </tr>

                <tr>
                    <th style="text-align: left; width: 150px;">chunk_type</th>
                    <td style="text-align: left;">numpy.ndarray</td>
                </tr>



                    <tr>
                        <th style="text-align: left; width: 150px;"> depends on </th>
                        <td style="text-align: left;">sum-1eee5d86c351bbfd9246db6fcdd2ab33</td>
                    </tr>


            </table>
        </td>
        <td>
            <svg width="250" height="250" style="stroke:rgb(0,0,0);stroke-width:1" >

2 2 

</details>

        <div style="">
<svg width="24" height="24" viewBox="0 0 32 32" fill="none" xmlns="http://www.w3.org/2000/svg" style="position: absolute;">

    <circle cx="16" cy="16" r="14" fill="#8F8F8F" style="stroke: var(--jp-ui-font-color2, #1D1D1D);" stroke-width="2"/>

</svg>

<details style="margin-left: 32px;">
    <summary style="margin-bottom: 10px; margin-top: 10px;">
        <h4 style="display: inline;">Layer4: sum-aggregate</h4>
    </summary>
    <p style="color: var(--jp-ui-font-color2, #5D5851); margin: -0.25em 0px 0px 0px;">
        sum-aggregate-00894334ef002abf8d298cbe8aa1aab4
    </p>

    <table>
    <tr>
        <td>
            <table>

                <tr>
                    <th style="text-align: left; width: 150px;">layer_type</th>
                    <td style="text-align: left;">MaterializedLayer</td>
                </tr>

                <tr>
                    <th style="text-align: left; width: 150px;">is_materialized</th>
                    <td style="text-align: left;">True</td>
                </tr>

                <tr>
                    <th style="text-align: left; width: 150px;">number of outputs</th>
                    <td style="text-align: left;">1</td>
                </tr>

                <tr>
                    <th style="text-align: left; width: 150px;">shape</th>
                    <td style="text-align: left;">()</td>
                </tr>

                <tr>
                    <th style="text-align: left; width: 150px;">dtype</th>
                    <td style="text-align: left;">float64</td>
                </tr>

                <tr>
                    <th style="text-align: left; width: 150px;">chunksize</th>
                    <td style="text-align: left;">()</td>
                </tr>

                <tr>
                    <th style="text-align: left; width: 150px;">type</th>
                    <td style="text-align: left;">dask.array.core.Array</td>
                </tr>

                <tr>
                    <th style="text-align: left; width: 150px;">chunk_type</th>
                    <td style="text-align: left;">numpy.ndarray</td>
                </tr>



                    <tr>
                        <th style="text-align: left; width: 150px;"> depends on </th>
                        <td style="text-align: left;">sum-partial-0259629db3a875f0182cc02372216a68</td>
                    </tr>


            </table>
        </td>
        <td>

        </td>
    </tr>
    </table>

</details>

    </div>
</div>

Graph View