Analog AI Accelerators: Computing with Physics

A lander with no memory to spare

During the final descent of Chandrayaan-3's Vikram lander in August 2023, the spacecraft had to identify a safe touchdown site on its own. Ground control on Earth could not do it — the round-trip light delay and the speed of the descent made real-time human intervention impossible. Onboard cameras fed images to an onboard processor, which had to run terrain-analysis algorithms — at their core, convolutions, which are matrix multiplications — fast enough to pick a landing spot before the lander ran out of altitude, on a power budget that bears no resemblance to a data centre. A single NVIDIA H100 GPU alone draws up to 700 watts. A spacecraft's entire compute stack, camera, and actuator suite has to survive on a small fraction of that, powered by batteries charged from a solar array folded into a rocket fairing.

This is the real constraint behind a question that sounds abstract: where does the energy in a matrix multiplication actually go? Not into the multiplication itself, as it turns out. It goes into moving numbers from where they are stored to where they are multiplied. That single fact — verified, measured, and published repeatedly in circuit-design literature — is the entire motivation for analog AI accelerators. Instead of storing a weight in memory and shipping it to a digital multiplier every time it is needed, these chips store the weight as a physical property of a device and let the laws of electric circuits do the multiplying and adding on the spot. The chip does not run an algorithm for matrix multiplication. It builds a physical circuit whose electrical behaviour is the matrix multiplication.

Where digital multiply-accumulate energy actually goes

Every layer of a neural network you have studied so far — a fully connected layer, a convolution, the query-key-value projections inside a transformer's attention block — reduces to the same atomic operation repeated billions of times: multiply an input value by a weight, and add the result into a running sum. This is called a multiply-accumulate, or MAC. A forward pass through even a modest model executes MACs by the billions; a large language model executes them by the trillions per token generated.

In a conventional digital processor — a CPU, a GPU, a digital NPU — a MAC requires three separate actions: fetch the weight from memory, perform the multiply in an arithmetic logic unit, and perform the add. Mark Horowitz's widely cited ISSCC 2014 keynote, "Computing's Energy Problem (and what we can do about it)," measured the energy cost of each of these actions in a modern CMOS process, and the figures were later reproduced in Sze, Chen, Yang, and Emer's 2017 Proceedings of the IEEE survey, "Efficient Processing of Deep Neural Networks." The numbers, in order of magnitude, are striking:

• A 32-bit integer multiply costs roughly 3.1 picojoules (pJ).
• A 32-bit integer add costs roughly 0.1 pJ.
• Reading a 32-bit word from an on-chip SRAM cache costs roughly 5 pJ.
• Reading a 32-bit word from off-chip DRAM costs roughly 640 pJ.

Read those four numbers again. The arithmetic — the part everyone thinks of as "the computation" — costs about 3.2 pJ per MAC. Fetching the weight that arithmetic needs, if it has to come from DRAM rather than a nearby cache, costs roughly 200 times more energy than doing the multiply and add combined. This is the von Neumann bottleneck: a processor built around a separate memory bank and a separate compute unit spends most of its energy budget shuttling data across the wire between them, not computing on it. A neural network with hundreds of millions of weights cannot keep them all in a small, energy-cheap SRAM cache, so most weight fetches during a full forward pass pay something close to that DRAM tax.

# Order-of-magnitude energy accounting
# (figures from Horowitz, ISSCC 2014, reproduced in Sze et al., Proc. IEEE 2017)

mult_32b_pJ  = 3.1     # 32-bit integer multiply
add_32b_pJ   = 0.1     # 32-bit integer add
dram_read_pJ = 640.0   # 32-bit word, off-chip DRAM

compute_only = mult_32b_pJ + add_32b_pJ
ratio = dram_read_pJ / compute_only
print(round(compute_only, 2), round(ratio, 1))

This prints 3.2 200.0. Fetching a single weight from off-chip memory costs about two hundred times what it costs to actually multiply and accumulate with it. This is the number analog accelerators are built to attack — not the compute, the movement.

Ohm's Law and Kirchhoff's Current Law as a matrix multiplier

An analog in-memory-compute accelerator eliminates the fetch by never separating the weight from the compute in the first place. The core structure is a resistive crossbar array, built from non-volatile memory devices — resistive RAM (ReRAM) or phase-change memory (PCM) cells — arranged at the intersections of horizontal and vertical wires, like a grid.

Each cell is programmed once, by applying a specific write voltage, to hold a particular electrical conductance, G (measured in siemens, the reciprocal of resistance). That conductance value is the stored weight. To feed an input, you don't send a number down a bus to an ALU — you apply a voltage, V, proportional to the input activation, along the row wire. Ohm's Law is not a metaphor here; it is the literal circuit law governing every cross-point: the current flowing through that cell is I = V × G. The multiplication of activation by weight has already happened, in the time it takes charge to redistribute across the device — sub-nanosecond, and with no clock cycle spent on a digital multiplier circuit.

The accumulation step is equally physical. Every cell in a column feeds its current onto the same vertical wire, and Kirchhoff's Current Law says the total current flowing out of that wire equals the sum of every current flowing into it: I_col = Σᵢ Vᵢ·Gᵢ. That sum is exactly a dot product — one row of a weight matrix multiplied against the input vector, accumulated over every input, in a single physical step. A crossbar with m input rows and n output columns computes an entire m-by-n matrix-vector product simultaneously, the instant the input voltages are applied — not sequentially, the way a digital ALU would step through each multiply-add pair.

Worked example: tracing a 3×2 crossbar by hand

Consider a tiny fully connected layer with 3 inputs and 2 outputs — small enough to trace by hand, structurally identical to a full-width layer. The three input activations are encoded as row voltages: V₁ = 0.2 V, V₂ = 0.5 V, V₃ = 0.8 V. The weight matrix is programmed as six conductances (in millisiemens, mS):

Column 1's output current, by Ohm's Law at each cell and Kirchhoff's Current Law at the column:

I₁ = V₁G₁₁ + V₂G₂₁ + V₃G₃₁ = (0.2)(0.1) + (0.5)(0.3) + (0.8)(0.2) = 0.02 + 0.15 + 0.16 = 0.33 mA

Column 2's output current, the same way:

I₂ = V₁G₁₂ + V₂G₂₂ + V₃G₃₂ = (0.2)(0.4) + (0.5)(0.1) + (0.8)(0.05) = 0.08 + 0.05 + 0.04 = 0.17 mA

Every step of this is the matrix-vector product y = Vᵀ G that you would compute digitally with a loop of multiply-accumulate instructions — except here it required no loop, no ALU, and no weight fetch, because the weight never left the device that stored it. Verify the arithmetic with a direct matrix multiplication:

import numpy as np

V = np.array([0.2, 0.5, 0.8])            # activations, encoded as row voltages (V)
G = np.array([[0.1, 0.4],
              [0.3, 0.1],
              [0.2, 0.05]])              # conductances (mS): rows = inputs, cols = outputs

I = V @ G                                # Ohm's Law (V*G per cell) + Kirchhoff's Law (column sum)
print(np.round(I, 3))

This prints [0.33 0.17], matching the hand trace exactly — floating-point rounding is why the code rounds to three decimals rather than printing the raw sums, which carry tiny binary-representation error in the fifteenth decimal place.

How the crossbar fits into a real chip

From lab bench to silicon: what shipped chips actually look like

This is not a paper idea. IBM Research published a 64-core mixed-signal chip built on phase-change memory in Nature Electronics in 2023 (Le Gallo, Khaddam-Aljameh, and colleagues), where each core is a crossbar array performing analog in-memory matrix-vector multiplication for deep neural network inference, with digital logic handling everything around the crossbar — activation functions, routing between layers, and the analog-to-digital conversion at the boundary. Mythic, a US-based accelerator company, built its Analog Matrix Processor around embedded flash memory cells doing the same job: each flash transistor's threshold voltage is tuned to store a weight as an analog quantity, and rows of them compute dot products the same way a ReRAM crossbar does. On the architecture-research side, two influential ISCA 2016 papers, ISAAC (Shafiee et al.) and PRIME (Chi et al.), both proposed ReRAM crossbar-based accelerators for convolutional neural networks and estimated order-of-magnitude energy and throughput gains over contemporary digital accelerators by keeping weights stationary in the array across an entire inference pass.

Every one of these designs shares a structural feature worth naming: the crossbar's output current is analog, but everything downstream of it — the next layer's input, the residual connection, the final classification decision — is digital. That means the current has to be converted back into a digital number, by an analog-to-digital converter (ADC), before it can go anywhere else in the system. This conversion is not free, and in several published crossbar accelerator designs, including ISAAC's own analysis, the ADC circuitry accounts for a substantial share, sometimes the majority, of the chip's total power and area — because high-resolution, fast ADCs are themselves power-hungry digital-adjacent circuits. The crossbar wins the multiply-accumulate battle almost for free; it does not automatically win the whole war, because every column still has to hand its answer across the analog-digital boundary.

The misconception: "analog means more precise"

A natural assumption, especially once you've spent a year thinking in terms of float32 versus int8 quantization, is that analog computation must be more accurate than digital, because it works with continuous physical quantities instead of numbers rounded to a fixed number of bits. This is backwards, and it is worth correcting explicitly, because it is the single most common misreading of what "analog AI accelerator" means.

A continuous physical quantity is not a noise-free quantity. Every resistive memory cell in a crossbar is subject to thermal (Johnson-Nyquist) noise, to device-to-device manufacturing variation — no two ReRAM cells programmed to "the same" conductance will land on exactly the same value — and to non-linear, sometimes drifting, current-voltage response, especially in phase-change devices, where the resistance can relax over time after programming. Long wires across a large array add resistive voltage drop (IR drop) that shifts the effective voltage seen by cells far from the input driver. None of this is quantization error in the digital sense, but all of it behaves like noise added to the computed sum, and published crossbar chips typically report effective precision in the range of 4 to 8 bits — well below the 32-bit floating point a GPU computes in by default, and often below even the 8-bit integer quantization used for efficient digital inference.

The correct framing is a trade, not a free upgrade: analog crossbars exchange precision for energy efficiency. That trade is acceptable for inference on a model that tolerates a few percent of additional error, and it is why every production analog accelerator you will find is deployed for inference only, running weights that were trained to full precision on a digital system and then quantized and "written" onto the analog array afterward. It is not acceptable for training, where gradient computations need to be precise and exactly reproducible for optimization to converge reliably — which is also why backpropagation, to this day, runs on digital GPUs and TPUs, never directly on the crossbar's analog physics.

Active recall

Attempt each question before reading its answer.

Q1. A crossbar has 2 input rows and 3 output columns. Row voltages are V₁ = 0.4 V, V₂ = 0.6 V. Conductances (mS) are: column 1 — G₁₁=0.20, G₂₁=0.10; column 2 — G₁₂=0.05, G₂₂=0.30; column 3 — G₁₃=0.15, G₂₃=0.15. Compute I₁, I₂, I₃.

Q2. A digital accelerator spends 3.2 pJ on the multiply-add itself, but 640 pJ fetching the needed weight from DRAM. An idealized analog crossbar cell already holds the weight, spending 0 pJ on fetch and an estimated 4 fJ (0.004 pJ) on the physical MAC settling process. Find the total energy per MAC for each approach and the fold-improvement of analog over digital.

Q3. Take the worked 3-input × 2-output crossbar and scale it to 3 inputs × 200 outputs, a realistic hidden-layer width. Each column needs one 8-bit ADC conversion per inference pass, costing roughly 50 fJ per conversion. Using the same 4 fJ-per-MAC idealized compute cost from Q2, find (a) total crossbar compute energy for this layer's 600 MACs, (b) total ADC energy for its 200 columns, and (c) which one dominates.

Q4. Explain, in terms of what training actually requires, why backpropagation is not run directly on an analog crossbar's physics, even though the forward pass could be.

Q5. A classmate says: "Analog compute must be more accurate than digital, because it's continuous instead of rounded to bits." What's wrong with this claim, and what precision do real crossbar chips typically achieve instead?

Q6. You must choose between an analog crossbar accelerator and a GPU for two workloads: (a) an always-on, battery-powered keyword-spotting model on a wearable, processing one sample at a time, with weights that almost never change; (b) a cloud transformer-serving system handling large batches of requests, frequently swapping between different fine-tuned adapters. Which workload fits the crossbar, and why does the other one not?

A1. I₁ = (0.4)(0.20) + (0.6)(0.10) = 0.08 + 0.06 = 0.14 mA. I₂ = (0.4)(0.05) + (0.6)(0.30) = 0.02 + 0.18 = 0.20 mA. I₃ = (0.4)(0.15) + (0.6)(0.15) = 0.06 + 0.09 = 0.15 mA. Each output current is a Kirchhoff-summed set of Ohm's-Law products across its column, exactly as in the worked example.

A2. Digital total = 3.2 pJ + 640 pJ = 643.2 pJ. Analog total (idealized, fetch-free) = 0.004 pJ. Fold-improvement = 643.2 / 0.004 = 160,800×. This number is intentionally extreme, because it ignores every cost outside the crossbar cell itself — most importantly the ADC. Treat it as an upper bound the physics alone could theoretically offer, not a number any shipped chip achieves.

A3. (a) Crossbar compute energy = 600 MACs × 4 fJ = 2,400 fJ = 2.4 pJ. (b) ADC energy = 200 columns × 50 fJ = 10,000 fJ = 10 pJ. (c) ADC energy is about 4.2× larger than the idealized crossbar compute energy for the same layer. This is the ripple effect of widening the array from 2 outputs to 200: MAC count scales with rows × columns, but ADC count scales with columns alone, so a tall, narrow crossbar (few inputs summed per column, many columns) is ADC-dominated, while a wide crossbar (many inputs summed per column) amortizes each column's fixed ADC cost over more physics-computed MACs. Real accelerator designs deliberately maximize the number of inputs feeding each column for exactly this reason.

A4. Training needs exact, reproducible gradients and precise, controllable weight updates so that gradient descent converges predictably. Analog crossbar cells cannot deliver this: writing a new conductance to a ReRAM or PCM cell is itself imprecise and non-linear, cells have limited write endurance, and the same nominal write does not always land on the same conductance twice. Digital hardware, working in exact fixed- or floating-point arithmetic, is what makes reliable backpropagation possible; the trained, quantized weights are only converted to conductances and "programmed" onto the analog array afterward, for the forward-pass-only job of inference.

A5. The claim conflates "continuous" with "noise-free." Continuous physical quantities are still corrupted by thermal noise, device-to-device conductance variation, non-linear and drifting current-voltage behaviour (especially in PCM), and IR drop across long wires. None of that is quantization error, but all of it adds effective error to the computed sum. Published crossbar chips typically achieve effective precision in the 4-to-8-bit range — below, not above, standard digital int8 or float32 inference precision. Analog trades precision for energy efficiency; it does not exceed digital precision.

A6. Workload (a), the keyword-spotting wearable, fits the crossbar well: the model is small, the weights are essentially fixed once deployed, inference is single-sample and latency-tolerant, and the power budget is the binding constraint — exactly the profile a fixed-conductance analog array was designed for. Workload (b) does not fit: cloud serving needs to batch many requests through the same hardware for throughput (a strength of a GPU's parallel digital pipelines, not of a physically fixed crossbar), and frequently swapping fine-tuned adapters means frequently reprogramming conductances, which is slow and energy-costly compared to a GPU simply loading a different set of weights into SRAM. The crossbar's core advantage — near-zero-cost weight fetch — depends on the weights staying put; a workload that keeps changing them gives that advantage away.

Think About It

Think about this: How would you explain analog ai accelerators: computing with physics to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.

Practice Exercises

Now it is time to practice! Complete these challenges to solidify your understanding:

• Exercise 1: Write a short program that demonstrates the core concept from this chapter. Test it with at least 3 different inputs.
• Exercise 2: Find a real-world example where analog ai accelerators: computing with physics is used in an Indian company (like TCS, Infosys, Flipkart, or ISRO). Write a paragraph explaining the connection.
• Exercise 3: Create a mind-map connecting analog ai accelerators: computing with physics to at least 3 other topics you have studied.

Key Takeaways — Summary and Recap

Let us recap what we covered: the core ideas behind analog ai accelerators: computing with physics, how they connect to real-world applications, and why they matter for your journey in computer science. Remember these key points as you move forward. For competitive exam preparation (CBSE, JEE, BITSAT), focus on understanding the WHY behind each concept, not just the WHAT.