22.1 Introduction

Biological systems give us examples of amorphous, unstructured devices capable of noise- and fault-tolerant information processing. They excel in massively parallel spatial problems, as opposed to digital processors, which are rather weak in that area. The morphic architecture was thus introduced in the Emerging Research Architecture section of ITRS 2007, to refer to biologically inspired architectures that embody a new kind of computation paradigm in which adaptation plays a key role to effectively address the particulars of problems [1]. This chapter focuses on recent progress of two morphic architectures that offer opportunities for emerging nanoelectronic devices: neuromorphic architectures and cellular automaton architectures.

22.2 Neuromorphic Architectures

The term neuromorphic was introduced by Carver Mead in the late 1980s to describe VLSI systems containing electronic analog circuits that mimic neuro-biological architectures in the nervous system [2]. Traditional neurocomputers employ components that are biologically rather implausible, like static threshold elements that represent neurons, whereas neuromorphic architectures are closer to biology. An example of a neuromorphic VLSI system is the silicon retina [3] (Figure 22.1) that is modeled after the neuronal structures of the vertebrate retina.

The appeal of neuromorphic architectures lies in: (i) their potential to achieve (human-like) intelligence based on the unreliable devices typically found in neuronal tissue, (ii) their strategies to deal with anomalies, emphasizing not only tolerance to noise and faults, but also the active exploitation of noise to increase the effectiveness of operations, and (iii) their potential for low-power operation. Traditional von Neumann machines are less suitable with regard to item (i), since for this type of tasks they require a machine complexity (the number of gates and computational power), that tends to increase exponentially with the complexity of the environment (the size of the input). Neuromorphic systems, on the other hand, exhibit a more gradual increase of their machine complexity with respect to the environmental complexity [4] (Figure 22.2). Therefore, at the level of human-like computing tasks, neuromorphic machines have the potential to be superior to von Neumann machines. Points (ii) and (iii) are strongly related to each other in von Neumann machines, since tolerance to noise runs counter to lowering power supply voltage and bias currents. Neuromorphic architectures, on the other hand, suffer much less from this trade-off. Unlike von Neumann machines, which can correct bit errors only to a certain extent through the use of error control techniques, neuromorphic machines tend to keep working even under high error rates.

Like the areas of the human brain, neuromorphic machines (LSIs) are application-specific. Significant performance benefits can be achieved by employing them as supplemental CMOS, as an addition to a von Neumann system, which provides universal computation ability. Neuromorphic systems thus have a diversified functionality, and consequently can be categorized as more than Moore candidates. Table 22.1 shows trends in the development of neuromorphic systems and their applications. The application category “Information processing” is only a single item in the table, which understates the potentially large benefits in terms of machine complexity of human-like information processing, since functionalities like prediction, associative memory, and inference offer new opportunities require huge machine complexity if they were implemented by von Neumann architectures. The Emerging Research Architecture section of ITRS 2009, for example, introduced an inference engine based on Bayesian neural networks [5], and in 2010 Lyric Semiconductor introduced NAND gates where the input probabilities are combined using Bayesian logic rather than the binary logic of conventional processors. Lyric Error Correction (LEC) uses this probabilistic logic to perform error detection and correction with about 3% of the circuitry and 8% of the power that would be needed for the equivalent conventional error correction scheme [6].

ITRS 2007 (Emerging Research Architecture section) did not address neuromorphic “Intelligent sensors,” since they were considered ancillary to the central focus on information processing of ITRS at the time. However, intelligent sensors have been re-spotlighted, because there exist vast opportunities for high-performance architectures that combine them with emerging nanoelectronic devices. So far CMOS implementations (in the items “Vision” and “Others”; see Table 22.1) and blueprints of SET implementations (in item “Vision”) have been proposed.

Another approach to build neuromorphic systems is inspired by biochemical reactions in living organisms. Reaction–diffusion computers, for example, are based on a biochemical reaction named after Belousov and Zhabotinsky [38], and they are able to efficiently solve combinatorial problems through the use of natural parallelism. Electronic implementations of this type of information processing require strong nonlinear I-V characteristics to mimic the chemical reactions involved, and there exist many opportunities for emerging nanoelectronic devices in this respect.

On the technology side, one of the key issues for neuromorphic systems is how neuronal elements are implemented. An important consideration in this respect is the level of abstraction of a biological neuron, which can range from (almost) truthful to a very simple model, such as an integrate and fire neuron. Depending on the technology used [single electron transistor (SET) neurons, RTD, memristors, etc.], this level may vary, and opportunities for emerging nanoelectronic devices exist in this respect. Another important issue is how nonvolatile analog synapses are implemented. Many attempts have been made to design analog synaptic devices based on existing flash memory technologies, but they have experienced difficulties in designing appropriate controllers for electron injection and ejection as well as increasing the limit on the number of rewriting times. Memristive devices (e.g., resistive RAMs and atomic switches) offer a promising alternative for the implementation of nonvolatile analog synapses. They are applied in the CMOL architecture, which combines memristive nano-junctions with CMOS neurons and their associated controllers. In ITRS 2007, CMOL (CrossNets) was introduced in terms of nanogrids of (ideally) single molecules fabricated on top of a traditional CMOS layer, but the concept has since been expanded to use a nanowire crossbar add on as well as memristive (two-terminal) crosspoint devices like nanowire resistive RAMs [39]. The CMOL architecture will be further expanded to include multiple stacks of CMOS layers and crossbar layers. This may result in the implementation of large-scale multi-layer neural networks, which have thus far evaded direct implementations by CMOS devices only.

A final important issue is noise tolerance and noise utilization in neural systems and their possible application in electronics. Noise and fluctuations are usually considered obstacles in the operation of both analog and digital circuits and systems, and most strategies to deal with them focus on suppression. Neural systems, on the other hand, tend to employ strategies in which the properties of noise are exploited to improve the efficiency of operations. This concept may be especially useful in the design of computing systems with noise-sensitive devices (e.g., extremely low-power devices like SET and subthreshold analog CMOS devices).

Table 22.2 shows examples of noise-driven neural processing and their possible applications in electronics. Stochastic resonance (SR) is a phenomenon where a static or dynamic threshold system responds stochastically to a subthreshold or suprathreshold input with the help of noise. In biological systems SR is utilized to detect weak signals in a noisy environment. SR on some emerging nanoelectronic devices (a SET network and GaAs nanowire FETs) has been demonstrated. SR can be observed in many bi-stable systems and will be utilized to facilitate the state transitions in emerging logic (bi-stable) memory devices. Noise-driven fast signal transmission is observed in neural networks for the vestibulo-ocular reflex, where signals are transmitted with an increased rate over a neuronal path when nonidentical neurons and dynamic noise are introduced. Implementation in terms of a SET circuit has demonstrated that, when several nonidentical pulse-density modulators were used as noisy neurons, performances on input–output fidelity of the population increased significantly as compared to that of a single neuron circuit. Phase synchronization among isolated neurons can be utilized for skew-free clock distribution where independent oscillators are implemented on a chip as distributed clock sources, while the oscillators are synchronized by a common temporal noise. Noise in synaptic depression can be used to facilitate the operation of a neuromorphic burst-signal detector, where the output range of the detector is significantly increased by noise. Noise shaping in inhibitory neural networks has been demonstrated in subthreshold CMOS, where static and dynamic noises can positively be taken if one could not remove a certain level of noise or device mismatches. The circuits exploit properties of device mismatches and external (temporal) noise to perform noise shaping one-bit AD conversion (pulse-density modulation).

22.3 Cellular Automata Architectures

A Cellular Automaton is an array of cells, organized in a regular grid. Each cell can be in one of a finite number of states from a predefined state set, which is usually a set of integers. The state of each cell is updated according to transition rules, which determine the cell's next state from its current state as well as from the states of the neighboring cells. The neighbors of a cell are usually the cells directly adjacent in orthogonal directions of a cell, like the north, south, east, and west neighbor in the case of a two-dimensional grid (von Neumann neighborhood), but other neighborhoods have also been used in experiments. The functionality of each cell is defined by the transition rules of the cellular automaton. The transition rules are usually the same for all cells, but heterogeneous sets of rules have also been considered, as well as programmable rules. A cell can typically be expressed in terms of a Finite Automaton, which is a model in computer science well-known for its simple but effective structure.

Cellular automata were initially proposed by von Neumann in the 1940s as a model of self-reproduction, but most of the interest they have attracted since then has been motivated by their ability to conduct computation in a distributed way. Though cellular automata have the name of their inventor in common with von Neumann architectures, they represent a radically different concept of computation.

The appeal of cellular automata as emerging research architectures lies in a number of factors. First, their regular structure has the potential for manufacturing methods that can deliver huge numbers of cells in a cost-effective way. Candidates in this respect are bottom-up manufacturing methods, such as those based on molecular self-assembly. Second, regularity also facilitates reuse of logic designs. The design of a cell is relatively simple as compared to that of a microprocessor unit, so design efforts are greatly reduced for cellular automata. Third, errors are more easily managed in the regular structures of cellular automata, since a unified approach for all cells can be followed. Fourth, wire lengths between cells are short, or wires are completely unnecessary if cells can interact with their neighboring cells through some physical mechanism. Fifth, cells can be used for multiple purposes, from logic or memory to the transfer of data. This makes cellular automata configurable in a flexible way. Sixth, cellular automata are massively parallel, offering a huge computational power for applications of which the “logical structure” fits the topology of the grid of cells.

Cellular automata may be less suitable for certain application due to the following factors. First, there is a relatively large overhead in terms of hardware. Cells tend to require a certain minimal level of complexity in order to be useful for computation [54]. In practice this means that they are configurable for logic, memory, or data transfer. The density of such functionality per unit of area tends to be lower than that of more conventional architectures. A large hardware overhead may be acceptable, though, if cells are available in huge numbers at a low cost, especially if the cellular automaton can be mapped efficiently on a certain application. Second, input and output of data to cells may be difficult. The use of the border cells of a grid for input and output is infeasible in case of huge numbers of cells, because it fails to employ all cells in parallel. Parallel input and output to cells through optical means or by wires addressing individual cells like in memory have more potential in this context. Third, it is difficult to configure cells into various patterns of states. Such configuration and reconfiguration functionality is required to give the cellular automaton its functionality for a certain computational task. Here, similar solutions as those for the input and output of data need to be employed to access cells in parallel.

There are two approaches for implementing cellular automata in hardware: fine-grained and tiny-grained. Systems that are coarse-grained are considered outside the class of cellular automata, since they are associated with multi-core architectures. Fine-grained cellular automata have cells that can be configured each as one or a few logic gates, or as a simple hub for data transfer. A cell typically contains a limited amount of memory in the order of 10–100 Bytes. Cells are usually addressed on an individual basis for input and output, or for configuration. Typically, the transition rules governing the functionality of a cell are changed during configuration. An example of a fine-grained approach is the Cell Matrix [55], which is a model capable of universal computation. Fine-grained cellular automata have a good degree of control over configuration and computation, but this comes at the price of relatively complex cells, limiting the use of cost-effective manufacturing methods that can exploit the regularity of these architectures.

The other approach to cellular automata is tiny-grained. Cells in this model have extremely simple functionality, in the order of a few states per cell and a limited number of (fixed) transition rules. The small number of states translates into memory requirements of only a few bits per cell, whereas the nonprogrammable nature of the transition rules drastically reduces the complexity of cells. The simple nature of rules poses no problems if the rules are designed to cover the functionality intended for the cellular automaton. An example of a tiny-grained approach is proposed [56] which is capable of universal computations as well as the correction of errors. Tiny-grained cellular automata have the potential of straightforward realizations of cells on nanometer scales. The challenge is to design models with as few states and transition rules per cell as possible. The theoretical minimum is two states and one transition rule. In synchronously timed models this has been approached by the Game of Life cellular automaton (two states, two rules) and in asynchronously timed models (no clock) by the Brownian Cellular Automata [57] (three states, three rules). Both models are universal. The number of states and rules should be considered only as rough yardsticks, since ultimately the most important measure is the efficiency at which cells can be realized in a technology.

Most hardware realizations of cellular automata to date are Application-Specific. In this context cellular automata are used as part of a larger system to conduct a specific set of operations with great efficiency. Applications have typically a structure that can be mapped efficiently on the hardware, and the approach followed is generally tiny-grained, since cells are optimized for one or a few simple operations. Image processing applications are the most common in such hardware realizations [58–60], since they can be mapped with great efficiency on two-dimensional cellular automata. Though the focus in the past was mostly on operations like filtering, thinning, skeletonizing, and edge detection, recent applications of cellular automata include the watermarking of images with digital image copyright [61,62]. Cellular automata have also been used for the implementation of a Dictionary Search Processor [63], memory controllers [64], and the generation of test patterns for Built-In Self-Test (BIST) of VLSI chips [65,66]. An overview of application-specific cellular automata is given in [67].

It is possible that the role of cellular automata in architectures will gradually increase with technological progress, from being merely used as dedicated subprocessor to the main part of the architecture. To do that, cellular automata would need a capacity that their application-specific cousins lack: computation universality, that is, the ability to carry out the same class of computations as our current computers do. This term is mostly used in a theoretical context, to prove equivalence to a universal Turing machine or more narrowly, to support a complete set of Boolean logic functions. The extreme inefficiency of Turing machines carries with it the misunderstanding that generallity is equivalent to inefficiency, but this is often far from the truth. A general approach to carry out operations efficiently on a cellular automaton is to configure it as a logic circuit. Cells will then be used as logic gates or for transferring data between logic gates. In fine-grained cellular automata a cell is typically sufficiently complex to be able to function as one or a few gates. In tiny-grained cellular automata, on the other hand, clusters of cells need to work together in order to obtain logic gate functionality. A cluster typically consists of up to 10 tiny-grained cells, its size depending on the functionality covered. This may seem a large overhead, but cells tend to be much less complex than in fine-grained cellular automata, making this approach feasible. Furthermore, cells used merely to transfer data – and this is the majority of the cells – see much less of their hardware unused when carrying out this simple task.

Cellular automata have seen only limited attempts at realization on a nanometer scales Molecule Cascades [68] use CO molecules on a Cu(111) grid to conduct simple logic operations. The CO molecules jump from grid point to grid point, triggering each other sequentially, like dominos. This process is quite slow and error-prone, though there appears to be potential for improvement. The mechanical nature of the operations, though, means that this cellular automaton is unlikely to reach competitive speeds. Another attempt uses layers of organic molecules on a gold grid [69]. Interactions between molecules take place via the tunneling of electrons between them. The rules that have been identified as governing those interactions appear to be influenced by the presence of electrons in the grid. This may limit the control over the operation of the cellular automaton, but it also carries the promise of efficient ways to configure the grid.

22.4 Taxonomy of Computational Ability of Architectures

Whereas von Neumann architectures generally refer to the use of memory resources separated from computational resources to store data and programs, there is an increasing need for taxonomy of those architectures that are based on different concepts. Figure 22.3 illustrates the world of “information processing” including from present Boolean-based processing of von Neumann machines to nonNeumann machines and introducing four possible architectures on it, that is, More Neumann, More than Neumann, Less than Neumann, and Beyond Neumann.

The term More Neumann refers to those architectures that differ from the classical von Neumann architecture only in terms of numbers (e.g., presenting multi- or many-core architectures). While the stored memory concept is still followed in More Neumann architectures, a certain level of parallelism is assumed, like in multi-core systems. The More Neumann architecture has been grown from ancient Less than Neumann architectures (Figure 22.4, left), and thus the performance lies in the metrics in terms of parallelism, computational ability, and programmability, as shown in Figure 22.4, right.

More than Neumann refers to architectures that do not suffer from the von Neumann bottleneck between computation and memory resources, that is, these resources are integrated to a high degree. These architectures tend to have a highly distributed character in which small elements have extremely limited memory and computation resources to the extent that each element individually is Less than Neumann (i.e., incapable of being used as a full-fledged von Neumann architecture, like a “logic element” unit in FPGA), yet the combination of these elements lifts them to a higher level of competence for certain applications (Figure 22.4). In More than Neumann architectures reorganization or reconfiguration usually plays the role that programmability has in von Neumann architectures. Programming a More than Neumann architecture thus involves an appropriate organization or configuration of the individual elements in order to make them perform a certain function. This reorganization may take the form of setting/adjusting the memories of the individual elements, but it may also involve a relinking of interconnections between the elements. In the context of neuromorphic architectures the elements take the form of neurons and synaptic connections between them, and synapses can be adjusted based on a learning process, while in some architectures new synaptic connections can be created and old ones destroyed. In the case of cellular automata, the elements are the cells, and their functionality is changed by setting their memory states to appropriate values. More than Neumann architectures are typically capable of high performance on certain classes of problems, but much less so on other problems (or may be even unable to handle other problems). Neuromorphic architectures have their strengths in problems that involve learning, classification, and recognition, but they do less well on traditional computing problems. Cellular automata are strong in applications that demand a regular structure of logic or data and a huge degree of parallelism.

Beyond Neumann refers to architectures that can solve certain computational problems fundamentally faster than would be possible on the architectures outlined above (Figure 22.4), like quantum computing architectures. Problems such as these typically require computation times that are exponential as measured in terms of their input. The fundamental limits that restrict the computational power of architectures ranging from von Neumann to More than Neumann are exceeded in Beyond Neumann architectures through adopting novel operating principles. Schemes that use analog values instead of digital (neuromorphic architectures, dynamic systems, etc.), that use superposition of bit values (quantum computing schemes), or that use an analog timing scheme (asynchronous architectures) are prime candidates for this category. The flow of information in architecture may also characterize it as Beyond Neumann. While Turing machines embody the traditional Input–Processing–Output flow, modern computers (even von Neumann ones) are used in a more interactive mode with humans, like in gaming, or with other computers, when connected in networks. Biological brains have a somewhat related concept of input and output, but different in its implementation: their processing of information appears to be an autonomous process, that may (or may not) be modulated by the input signals in the environment [70]. This allows biological organisms to flexibly select important signals from the environment, while ignoring irrelevant ones. Underneath this all lies an impressive neural machinery, yet to be uncovered, that can solve problems with unrivaled efficiency. Many of the above elements (analog-valued signals, asynchronous timing in combination with selective synchronization, chaotic dynamics) are thought to play an important role in neural information processing. While Beyond Neumann architectures are promising in principle, it needs to be emphasized that currently no practical implementations of them have been reported.

22.5 Summary

This chapter introduced a concept of the morphic architecture that refers to architectures adapted to effectively address a particular problem set, and exhibited recent progress of two morphic architectures that offer opportunities for emerging nanoelectronic devices: neuromorphic architectures and cellular automaton architectures.

Morphic architectures will be employed in a broad class of mixed-signal systems that are focused on a particular application and that draw inspiration for their structure from the application. In some cases, computation is performed in the analog manner, which may offer orders of magnitude improvement in performance and power dissipation, albeit with reduced accuracy. As an example, biologically inspired inference networks for cognition may yield to a partial analog implementation and provide substantial gains in performance relative to their digital counterparts.

Finally, taxonomy for emerging models of computation was introduced, because of an increasing need of classifying emerging architectures that are based on nonNeuman concepts. Concepts of four possible architectures were introduced: More Neumann, More than Neumann, Less than Neumann, and Beyond Neumann architectures. The morphic and cellular automaton architectures introduced in this chapter will be categorized into More than Neumann architectures that consist of huge collection of ancient Less than Neumann (i.e., incapable of being used as fully fledged von Neumann architecture) elements.