13.4. NoC
Once you accept that complex, 21st-century SOC designs must incorporate large numbers of processors, the next logical step is to think about efficiently connecting those on-chip processors together. NoCs do not attempt to replicate the deep, multilayer, protocol-heavy schemes used for macro-level, Ethernet-style networking amongst computer systems. Instead, NoCs attempt to create lightweight, high-bandwidth networks that are agile enough to serve as links amongst on-chip processor cores. NoC designs and concepts vary widely, but there are some common NoC characteristics:
They are more than a single, shared medium (like a bus). They are truly networks.
They provide point-to-point connections between any two hosts attached to the network either by employing true, point-to-point crossbar switches or through virtual point-to-point connections.
They provide high aggregate bandwidth through parallelism.
They clearly separate communication from computation.
They take a layered approach to communications, such as used in other macro networking schemes although they may not employ as many network layers because such complexity is prohibitively expensive in terms of on-chip area.
They have implicit pipelining and provide intermediate storage points for the data as it moves from sender to receiver.
When evaluating NoC performance, there are two key figures of merit for performance: throughput and latency. Throughput is simply the maximum amount of data that senders can pump through the network at any time. Complex network topologies can make it difficult to measure the true throughput of a network. One approach to measuring this parameter is to measure aggregate throughput. Another method is to measure the bisection throughput: the amount of data crossing an imaginary line drawn to bisect the network.
Latency measures the time needed for data to traverse the NoC from sender to receiver. The latency depends on the number of network switches the message must travel through, the amount of buffering or storage there is in each network switch, and the amount of NoC traffic. Throughput and latency are closely intertwined and both depend on network loading. Research indicates that network congestion becomes a concern as the NoC messaging load increases above about 35%.
There are two fundamental approaches to building NoCs: circuit switching and packet switching. Circuit-switched NoCs establish a connection between sender and receiver before the data transfer starts. Once established, the circuit connection is reserved for the entire duration of the message transfer and then closed when the transmission ends or when there’s no longer a need for that connection. Packet-switched NoCs chop the data into packets. Each packet contains a header that directs it to a destination address. Separate packets (also called sub-packets, flits, or phits) can take different routes through the network and must then be reassembled, in the proper order, by the receiver. Circuit- and packet-switched networking schemes are hardly new. The original telephone network was a circuit-switched network and today’s Internet is the world’s largest and most familiar packet-switched network.
NoCs are also classified by network topology. Mesh networks arrange on-chip sender and receiver blocks in a regular grid. Each sender or receiver within the mesh is associated with a network switch and each network switch is connected to its nearest four neighbors (North, South, East, and West). Figure 13.9 shows a 2D mesh NoC.
The 2D mesh network is an intuitive network topology. It allows any sender to talk to any receiver and it seems to have won the popularity contest among academic NoC researchers. Variations of the mesh network are the torus and folded torus, which respectively link network switches at two or four of the mesh edges to the corresponding switch at the opposite side of the mesh. This topology reduces the number of switches a message must traverse (the number of “hops”). Figure 13.10 shows a torus version of the 2D mesh network shown in Figure 13.9.
The ring network is another familiar networking topology and appears in Figure 13.11. Rings can be unidirectional or bidirectional. However, published research suggests that a ring is the least efficient NoC topology.
Although the NoCs shown above make excellent university research vehicles, the amount of logic needed to implement some of these NoCs can easily cause many commercial SOC design teams to discard any ideas for NoC use. There is a different line of NoC research that has a goal of developing very lightweight on-chip networks. Advocates of such networks believe that the processing units or clusters on the chip that perform the actual work of the SOC should get the majority of the silicon and that the on-chip network should only be sufficiently complex to serve the needs of these processing units.