Skip to content

Mesh interconnect

This example builds a parameterized N×M toroidal mesh network. Each node performs a unit of compute work every cycle, then injects a packet to a randomly chosen destination every few cycles. In-transit packets are routed hop-by-hop using dimension-order (XY) routing: each intermediate node forwards the packet East until it reaches the correct column, then South until it reaches the correct row.

What this example demonstrates:

  • Two-dimensional submodule_array node[N][M] declarations
  • Two-dimensional net_array net_e[N][M] and net_s[N][M] declarations
  • Nested structural for loops for 2D connection generation
  • Wrap-around (toroidal) connections with explicit per-case for loops
  • Parent init block initializing per-instance member variables (row, col) across a 2D array
  • C++ pointer arrays (inport<W>*, outport<W>*) for indexed port access inside code blocks

Topology

Each node has one East outport (out_e) and one South outport (out_s), plus matching inports (in_e receiving East-going tokens, in_s receiving South-going tokens). All connections wrap around — the rightmost column's East output connects to the leftmost column's East input, and the bottom row's South output connects to the top row's South input — forming a torus. Every node therefore always has a valid East and South neighbor, which simplifies the connection code and the routing logic.

Mesh structure

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
module Mesh
    parameter int N = 3           // rows
    parameter int M = 3           // columns
    parameter int INJECT_EVERY = 4
    parameter int RUN_CYCLES   = 20

    submodule_array node[N][M] : MeshNode<N, M, INJECT_EVERY, RUN_CYCLES>

    // East-going nets: net_e[i][j] connects node[i][j].out_e to node[i][j+1].in_e
    net_array net_e[N][M] : capacity 2 width 12

    // South-going nets: net_s[i][j] connects node[i][j].out_s to node[i+1][j].in_s
    net_array net_s[N][M] : capacity 2 width 12

    // East connections (columns 0..M-2)
    for i in 0 to (N-1)
        for j in 0 to (M-2)
            node[i][j].out_e      => net_e[i][j]
            node[i][(j+1)].in_e   <= net_e[i][j]
        end for
    end for
    // East wrap-around: last column -> first column
    for i in 0 to (N-1)
        node[i][(M-1)].out_e => net_e[i][(M-1)]
        node[i][0].in_e      <= net_e[i][(M-1)]
    end for

    // South connections (rows 0..N-2)
    for i in 0 to (N-2)
        for j in 0 to (M-1)
            node[i][j].out_s      => net_s[i][j]
            node[(i+1)][j].in_s   <= net_s[i][j]
        end for
    end for
    // South wrap-around: last row -> first row
    for j in 0 to (M-1)
        node[(N-1)][j].out_s => net_s[(N-1)][j]
        node[0][j].in_s      <= net_s[(N-1)][j]
    end for

    // Initialize each node with its (row, col) position
    init $
    for (int i = 0; i < N; i++)
        for (int j = 0; j < M; j++) {
            node[i][j].row = i;
            node[i][j].col = j;
        }
    $
end module
Download 4_mesh.sitar

Key structural features:

2D submodule_array: submodule_array node[N][M] : MeshNode<N,M,INJECT_EVERY,RUN_CYCLES> creates an N×M grid of MeshNode instances. Elements are accessed as node[i][j].

2D net_array: net_array net_e[N][M] : capacity 2 width 12 creates a 2D array of nets. East-going token from node[i][j] travels on net_e[i][j]; South-going on net_s[i][j].

Nested for loops: Connection statements inside nested for i ... for j ... end for end for blocks generate all interior connections. The wrap-around connections (last column → first column, last row → first row) are handled by separate single-dimension loops without requiring modulo arithmetic in the index expressions.

Parent init block: Since all MeshNode instances share the same type and template arguments, their row and col positions cannot be passed as compile-time parameters. The parent sets them at construction time using a C++ nested loop in the init block:

init $
for (int i = 0; i < N; i++)
    for (int j = 0; j < M; j++) {
        node[i][j].row = i;
        node[i][j].col = j;
    }
$

Node behavior

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
module MeshNode
    parameter int N            = 3
    parameter int M            = 3
    parameter int INJECT_EVERY = 4
    parameter int RUN_CYCLES   = 20

    inport  in_e  : width 12   // arrives from West (token traveled East)
    inport  in_s  : width 12   // arrives from North (token traveled South)
    outport out_e : width 12   // departs to East neighbor
    outport out_s : width 12   // departs to South neighbor

    include $#include <cstdlib>$

    decl $
    int  row;       // position, set by parent in init
    int  col;
    int  received;  // total tokens consumed at this node

    // Single in-flight packet slot (one packet in transit per cycle)
    bool      has_pkt;
    token<12> pkt;
    int       pkt_dr, pkt_dc, pkt_data;
    bool      ok;
    $
    init $row = 0;  col = 0;  received = 0;  has_pkt = false;$

    behavior
        do
            // ---- Phase 0: receive ----
            // Accept at most one incoming packet per cycle (East inport first).
            wait until (this_phase == 0);
            $
            // Simulate per-cycle compute work (placeholder)
            volatile int dummy = row * M + col + (int)this_cycle;
            (void)dummy;

            token<12> t;
            if (!has_pkt && (in_e.pull(t) || in_s.pull(t))) {
                sitar::unpack(t, pkt_dr, pkt_dc, pkt_data);
                if (pkt_dr == row && pkt_dc == col) {
                    // Packet has arrived at its destination
                    log << endl
                        << "(" << row << "," << col
                        << ") consumed data=" << pkt_data;
                    received++;
                } else {
                    // Load into transit slot for forwarding in phase 1
                    pkt     = t;
                    has_pkt = true;
                }
            }
            $;

            // ---- Phase 1: forward or inject ----
            wait until (this_phase == 1);
            $
            if (has_pkt) {
                // XY routing: move East until column matches, then South
                ok = (pkt_dc != col) ? out_e.push(pkt) : out_s.push(pkt);
                if (ok) {
                    log << endl
                        << "(" << row << "," << col
                        << ") fwd " << (pkt_dc != col ? "E" : "S")
                        << " to (" << pkt_dr << "," << pkt_dc << ")";
                    has_pkt = false;
                }
                // If push fails (net full), keep packet and retry next cycle
            } else if ((int)this_cycle % INJECT_EVERY == 0) {
                // Slot empty: inject a new packet to a random destination
                srand(row * M + col + (int)this_cycle * 7);
                int dr = rand() % N;
                int dc = rand() % M;
                if (dr != row || dc != col) {
                    token<12> inj;
                    int d = row * M + col;   // payload = source node index
                    sitar::pack(inj, dr, dc, d);
                    ok = (dc != col) ? out_e.push(inj) : out_s.push(inj);
                    if (ok)
                        log << endl
                            << "(" << row << "," << col
                            << ") inject to (" << dr << "," << dc << ")";
                }
            }
            $;
            wait;
        while (this_cycle < RUN_CYCLES) end do;
        stop simulation;
    end behavior
end module
Download 4_mesh.sitar

Phase 0 — receive: Each node accepts at most one incoming packet per cycle. If the packet is destined for this node, it is consumed and logged. Otherwise it is held in the node's transit slot.

Phase 1 — forward or inject: If the transit slot holds a packet, the node forwards it one hop toward its destination. XY routing: move East while pkt_dc != col; then South. If the target net is full, the packet is held and retried next cycle. If the transit slot is empty and it is an injection cycle, the node generates a new packet to a random destination.

One packet in transit per node

This model allows at most one forwarded packet per node per cycle. A production router would maintain per-direction queues; this example keeps the behavior minimal to focus on structural constructs.

Expected output (3×3 mesh, 20 cycles)

(0,1) TOP.mesh.node[0][0] : (0,0) inject to (2,1)
(0,1) TOP.mesh.node[0][0] : (0,0) fwd E to (2,1)
(1,0) TOP.mesh.node[0][1] : (0,1) fwd E to (2,1)
(2,0) TOP.mesh.node[0][2] : (0,2) fwd S to (2,1)
(3,0) TOP.mesh.node[1][2] : (1,2) fwd S to (2,1)
(4,0) TOP.mesh.node[2][1] : (2,1) consumed data=0
...
Simulation stopped at time (20,0)

Each packet takes at most N+M−2 hops. With a 3×3 mesh the longest path is 4 hops. Multiple packets circulate concurrently; their interleaved log lines show the hop-by-hop traversal.

Scaling the mesh

Change the top-level instantiation to Mesh<4,4> or Mesh<8,8> to generate larger meshes. The structure, connection loops, and routing logic are all parameterized on N and M — no other changes are needed.

Full example

The complete model below is self-contained and runnable. The Top module instantiates a 3×3 mesh; change the template arguments to scale up.

1
2
3
module Top
    submodule mesh : Mesh<3, 3>
end module
Download 4_mesh.sitar
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
module Mesh
    parameter int N = 3           // rows
    parameter int M = 3           // columns
    parameter int INJECT_EVERY = 4
    parameter int RUN_CYCLES   = 20

    submodule_array node[N][M] : MeshNode<N, M, INJECT_EVERY, RUN_CYCLES>

    // East-going nets: net_e[i][j] connects node[i][j].out_e to node[i][j+1].in_e
    net_array net_e[N][M] : capacity 2 width 12

    // South-going nets: net_s[i][j] connects node[i][j].out_s to node[i+1][j].in_s
    net_array net_s[N][M] : capacity 2 width 12

    // East connections (columns 0..M-2)
    for i in 0 to (N-1)
        for j in 0 to (M-2)
            node[i][j].out_e      => net_e[i][j]
            node[i][(j+1)].in_e   <= net_e[i][j]
        end for
    end for
    // East wrap-around: last column -> first column
    for i in 0 to (N-1)
        node[i][(M-1)].out_e => net_e[i][(M-1)]
        node[i][0].in_e      <= net_e[i][(M-1)]
    end for

    // South connections (rows 0..N-2)
    for i in 0 to (N-2)
        for j in 0 to (M-1)
            node[i][j].out_s      => net_s[i][j]
            node[(i+1)][j].in_s   <= net_s[i][j]
        end for
    end for
    // South wrap-around: last row -> first row
    for j in 0 to (M-1)
        node[(N-1)][j].out_s => net_s[(N-1)][j]
        node[0][j].in_s      <= net_s[(N-1)][j]
    end for

    // Initialize each node with its (row, col) position
    init $
    for (int i = 0; i < N; i++)
        for (int j = 0; j < M; j++) {
            node[i][j].row = i;
            node[i][j].col = j;
        }
    $
end module
Download 4_mesh.sitar
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
module MeshNode
    parameter int N            = 3
    parameter int M            = 3
    parameter int INJECT_EVERY = 4
    parameter int RUN_CYCLES   = 20

    inport  in_e  : width 12   // arrives from West (token traveled East)
    inport  in_s  : width 12   // arrives from North (token traveled South)
    outport out_e : width 12   // departs to East neighbor
    outport out_s : width 12   // departs to South neighbor

    include $#include <cstdlib>$

    decl $
    int  row;       // position, set by parent in init
    int  col;
    int  received;  // total tokens consumed at this node

    // Single in-flight packet slot (one packet in transit per cycle)
    bool      has_pkt;
    token<12> pkt;
    int       pkt_dr, pkt_dc, pkt_data;
    bool      ok;
    $
    init $row = 0;  col = 0;  received = 0;  has_pkt = false;$

    behavior
        do
            // ---- Phase 0: receive ----
            // Accept at most one incoming packet per cycle (East inport first).
            wait until (this_phase == 0);
            $
            // Simulate per-cycle compute work (placeholder)
            volatile int dummy = row * M + col + (int)this_cycle;
            (void)dummy;

            token<12> t;
            if (!has_pkt && (in_e.pull(t) || in_s.pull(t))) {
                sitar::unpack(t, pkt_dr, pkt_dc, pkt_data);
                if (pkt_dr == row && pkt_dc == col) {
                    // Packet has arrived at its destination
                    log << endl
                        << "(" << row << "," << col
                        << ") consumed data=" << pkt_data;
                    received++;
                } else {
                    // Load into transit slot for forwarding in phase 1
                    pkt     = t;
                    has_pkt = true;
                }
            }
            $;

            // ---- Phase 1: forward or inject ----
            wait until (this_phase == 1);
            $
            if (has_pkt) {
                // XY routing: move East until column matches, then South
                ok = (pkt_dc != col) ? out_e.push(pkt) : out_s.push(pkt);
                if (ok) {
                    log << endl
                        << "(" << row << "," << col
                        << ") fwd " << (pkt_dc != col ? "E" : "S")
                        << " to (" << pkt_dr << "," << pkt_dc << ")";
                    has_pkt = false;
                }
                // If push fails (net full), keep packet and retry next cycle
            } else if ((int)this_cycle % INJECT_EVERY == 0) {
                // Slot empty: inject a new packet to a random destination
                srand(row * M + col + (int)this_cycle * 7);
                int dr = rand() % N;
                int dc = rand() % M;
                if (dr != row || dc != col) {
                    token<12> inj;
                    int d = row * M + col;   // payload = source node index
                    sitar::pack(inj, dr, dc, d);
                    ok = (dc != col) ? out_e.push(inj) : out_s.push(inj);
                    if (ok)
                        log << endl
                            << "(" << row << "," << col
                            << ") inject to (" << dr << "," << dc << ")";
                }
            }
            $;
            wait;
        while (this_cycle < RUN_CYCLES) end do;
        stop simulation;
    end behavior
end module
Download 4_mesh.sitar