Optical AI Chips Promise 10–80x Lower Energy Use Than GPUs
AI responses (inference) have run into the limits of electricity, so startups are once again trying to compute with light. Optics multiplies a matrix of numbers by a vector with almost no energy cost, but encoding data into light is expensive. The realistic goal is to use 10–80x less energy than GPUs, not the 100,000x suggested by lab estimates. For practitioners, this is a signal: light will accelerate some workloads, not replace GPUs.
AI-processed from IEEE Spectrum AI; edited by Hamidun News
Every answer from ChatGPT or another chatbot costs electricity, and at some point electricity, not the money spent on chips, became the main constraint. Inference, meaning the answers a trained model gives to user queries, has already pushed companies to build enormous data centers. That is why an old idea, computing with light instead of electric current, has moved from the lab into business. According to IEEE Spectrum, the combined valuation of companies in this niche has reached $2.22B, compared with $0.4B in 2022. Their promise is to perform the same math GPUs (Nvidia graphics cards and their equivalents) do, but to spend 10–80x less energy on it.
"Interest in photonics is largely driven by the fact that AI inference has hit a power limit, and quite abruptly," says Peter McMahon, a professor at Cornell University who has studied optical computing for more than a decade. Optical startups are appearing in the US, China, Germany, and Israel.
How Light Multiplies Matrices
The core operation of inference is matrix-vector multiplication. Think of the vector as your query converted into a set of numbers, and the matrix as the table of model weights. Graphics cards multiply one by the other countless times per second.
As far back as 1978, Joseph Goodman and colleagues proposed a simple optical scheme. A row of light-emitting diodes encodes the vector's numbers in brightness: the brighter the diode, the larger the value. A cylindrical lens stretches this light vertically, effectively copying the vector N times. The light then passes through an N-by-N pixel attenuator: each pixel dims the light according to the matrix value, and this is the multiplication. A second lens collects the light along rows and sums it, and an array of detectors reads the resulting vector.
The advantage lies in scaling. In electronics, each of the N² multiplications requires energy, so costs grow as N². In optics, multiplication is passive: as long as the matrix does not change, light passing through it costs almost nothing. The payment goes to encoding the vector and reading out the answer, which scales roughly as N.
In 2022, McMahon's team demonstrated an optical neural network that spent less than one photon, about 2.5×10⁻¹⁹ J, per multiplication. In a conventional computer, a multiplication costs between 10⁻¹⁴ and 10⁻¹² J, so on paper the savings reach 100,000x.
Why the Promises Are More Modest Than the Lab Numbers
McMahon himself cautions against such estimates. First, the calculation leaves out the cost of encoding data into light. Suppose an electronic multiplication costs 1 picojoule, while encoding one vector element in optics costs 1,000 picojoules. Then electronics wins for any vector shorter than 1,000 elements. According to McMahon, this is not far from reality: optics begins to win somewhere between N = 1,000 and N = 10,000.
Second, Amdahl's law applies. If an accelerator handles 90% of the code but cannot speed up the remaining 10%, the maximum gain is tenfold. Matrix multiplication is only part of inference; there are also nonlinear operations, and electronics still handles those. That is why optical companies are aiming not for 100,000x, but for 10–80x in energy efficiency relative to GPUs. The open question is whether they can reach mass adoption before conventional chips themselves become more efficient.
Who Is Building What
Inference is usually divided into two phases: prefill, when the model processes the input query, and decode, when the response is generated one token at a time. Prefill is heavier computationally, because the same matrix is multiplied by many different vectors. That is where optics logically aims.
UK startup Lumai, from Oxford, drew more heavily on Goodman's scheme than others. Instead of a fixed attenuator, it uses a programmable spatial light modulator, a pixelated membrane resembling an advanced liquid-crystal display that changes both the brightness and the phase of light. Light travels through air rather than waveguides on a chip. The vector is encoded into light, copied by a lens, passes through the modulator, is collected by a second lens, and is converted back into a digital signal. Electronics handles the nonlinear functions. According to product lead Phillip Burr, a full matrix-vector multiplication completes in a single optical cycle.
Another UK company, Optalysys, takes a different approach: it embeds optical elements into the networking path of data centers, where light is already used for data transmission. Its basic on-chip element is a Fourier transform calculator.
What This Means for Practitioners
All the gain figures remain company targets, not production measurements. Still, the conclusions are already useful. Optics is unlikely to replace graphics cards entirely; a hybrid is more realistic, in which light handles the heavy multiplications and electronics handles everything else. Moreover, the gains appear only on sufficiently large matrices, so the first candidates will be large models and long contexts. Those planning infrastructure several years ahead should watch this niche, but should not build it into budgets until there are commercial products. Logan Wright of Yale adds that even if optics does not win the AI race, the technologies developed in this wave will enable photonic systems with unique capabilities.
- Nov 21, 2026Альтман назвал «ужасной идеей» передачу решений ИИ, а OpenAI бросила 25% инженеров на защиту
- Nov 21, 2026Anthropic shares opened at $111.50, valuing the seven co-founders' stakes at $180 billion
- Nov 21, 2026Phota Labs, Backed by a16z With $5.6M, Teaches AI to Reshoot Bad Photos Without Changing the Face
- Nov 21, 2026Author Makes a Narrative Video on Home LTX 2.3 (22B Parameters) Without Subscriptions
Key Questions Answered
How Light Multiplies Matrices?
The core operation of inference is matrix-vector multiplication. Think of the vector as your query converted into a set of numbers, and the matrix as the table of model weights. Graphics cards multiply one by the other countless times per second. As far back as 1978, Joseph Goodman and colleagues proposed a simple optical scheme. A row of light-emitting diodes encodes the vector's numbers in brightness: the brighter the diode, the larger the value.
Why the Promises Are More Modest Than the Lab Numbers?
McMahon himself cautions against such estimates. First, the calculation leaves out the cost of encoding data into light. Suppose an electronic multiplication costs 1 picojoule, while encoding one vector element in optics costs 1,000 picojoules. Then electronics wins for any vector shorter than 1,000 elements. According to McMahon, this is not far from reality: optics begins to win somewhere between N = 1,000 and N = 10,000. Second, Amdahl's law applies.
Who Is Building What?
Inference is usually divided into two phases: prefill, when the model processes the input query, and decode, when the response is generated one token at a time. Prefill is heavier computationally, because the same matrix is multiplied by many different vectors. That is where optics logically aims. UK startup Lumai, from Oxford, drew more heavily on Goodman's scheme than others.
Ship AI Faster with Founders, Engineers & Weekly Architecture Reviews
Inside Vibe Club (hamidun.com/club) — weekly live masterminds with Zhemal Khamidun, Tech Radar, full LMS access, and 370+ production Claude Code skills.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.