Safe AI for
Cyber‑Physical Systems

How do we trust AI-using machines —
when they fly, drive, and make real-world decisions?

Ben Wooding

Introductions

Ben Wooding

Portrait of Ben Wooding

Postdoctoral Researcher, Vanderbilt University

Institute for Convergent Software Integrated Systems (ICONS)

Formal methods for safe AI in cyber‑physical systems

🌍Originally from England — ten years in Uganda as a missionary kid

💍Married to Lucy

⛪Immanuel Church Nashville

The last two weeks

Two weeks of headlines

News clipping: NPR · 10 Sep 2026 NPR · 10 Sep 2026
News clipping: AP, via ABC News · 14 Sep 2026 AP, via ABC News · 14 Sep 2026
News clipping: NPR · 7 Sep 2026 NPR · 7 Sep 2026
News clipping: The Washington Post · 14 Sep 2026 The Washington Post · 14 Sep 2026
News clipping: AP, via PBS NewsHour · 14 Sep 2026 AP, via PBS NewsHour · 14 Sep 2026
News clipping: MIT Technology Review · 14 Sep 2026 MIT Technology Review · 14 Sep 2026
News clipping: NPR · 2 Sep 2026 NPR · 2 Sep 2026
News clipping: NPR · 13 Sep 2026 NPR · 13 Sep 2026

How do we know what’s real? And what’s just noise?

Why I care

“Love the Lord your God with all your heart… and love your neighbour as yourself.” Matthew 22 : 37–39

“The Lord God took the man and put him in the Garden of Eden to work it and take care of it.” Genesis 2 : 15

Developing new technologies and machines with behavioral guarantees is loving to the neighbor who will fly in them, be diagnosed by them, and live beside them.
We want new technologies to be fruitful not harmful. Machines cannot take responsibility — but the humans who design and build them can.

Scope, honestly

I’m staying in my lane

Relationships with AI

People are forming attachments to chatbots in the middle of an epidemic of loneliness.

Deepfakes & fraud

Convincing fake voices, images and video — used for scams, humiliation and deception.

Children & minds

AI is proliferating social media, with concerns already for the young and the vulnerable.

Work & dignity

Genuine concerns about the future of work, jobs, and livelihoods.

AI & the climate

The electricity and water required for this compute — and its climate cost.

Economy

Anthropic and OpenAI looking to go public soon, many believe an AI market bubble.

Genuine concerns, but outside my field of expertise.
Tonight: AI in the physical world — and how we try to guarantee safety.

01An old story

Therac‑25, 1985–87: dangers with new technologies

electron gun flattening target target in the beam — dose as prescribed race condition — target out, raw beam 1001,00010,000100,000 rads → typical ≈200 est. 15–20,000

A radiation‑therapy machine. Six massive overdoses in two years, across the US and Canada. Three patients died of the overdose.

The design

Earlier models had hardware interlocks — physical switches that made dangerous behaviors impossible. Software checks were substituted for many traditional hardware interlocks.

The bug

A race condition: an operator who edited the prescription too quickly could trigger the raw beam to fire without shielding.

The detail that should stop us

The Therac‑20 contained the same software error but its hardware caught what the software missed.

Three years of investigation: “system testing alone is not adequate.”

01An old story

Intel Pentium, 1994: an expensive typo

Pentium 1994 4,195,835 ÷ 3,145,727 should be 1.333820449… chip said 1.333739068… the FDIV lookup table — 5 cells wrong out of 2,048
Floating‑point division wrong from the fourth decimal place — found by a mathematician counting primes.
Cost of the recall
$0

Intel's answer ever since:
mathematically prove the arithmetic,
don't just test it.

01An old story

Ariane 5, 1996: one number too big for its box

H0 +10 s +20 s +30 s +37 s +39 s SRI 1 ✓ SRI 1 ✕ SRI 2 ✓ SRI 2 ✕ identical software — halted 0.05 s apart 64‑bit value → 16‑bit box overflow breakup ≈4 km — self‑destruct 4 June 1996 — one loop ≈ the whole flight
The bug

A 64‑bit decimal was forced into a 16‑bit integer slot and overflowed, in code that serves no purpose after lift‑off, kept from Ariane 4 “for commonality”.

The failure

Two inertial reference computers ran identical software — so both halted, 0.05 s apart. The autopilot then steered on their error message as if it were flight data.

The code had flown for years — on Ariane 4, a slower rocket. “Correct” is always relative to a specification — change the rocket and the guarantee changes too.

01An old story

Mars Climate Orbiter, 1999: pounds versus newtons

ground s/w: lbf·s spec & nav: N·s ×4.45 Mars ≈80 km survival line planned: 150–170 km actual: est. 57 km ✕ signal lost — never reacquired 23 September 1999 — nine months of drift, one dip
The bug

The ground software reported thruster impulses in pound‑force seconds; the interface specification said newton seconds.

The failure

Nine months of navigation quietly absorbed the 4.45× error. The $125 M orbiter’s first pass came in at an estimated 57 km — below the ≈80 km survival line.

The right unit was written down. A specification only protects you if something checks it.

01An old story

Every new technology asks an old question

🚂 1800s steam boilers explode → pressure‑vessel codes 🌉 1879 Tay Bridge falls → structural standards ✈️ 1954 Comet jets break up → fatigue testing & certification 💾 1994 Pentium FDIV bug → formal verification 🤖 now AI enters the world → ?
“When you invent the ship, you also invent the shipwreck.”
— Paul Virilio, Politics of the Very Worst, 1999
02Getting our words straight

So what is AI?

ARTIFICIAL INTELLIGENCE rules, search, planning, learning MACHINE LEARNING learns a model from examples NEURAL NETWORKS affine layers + nonlinearities GENERATIVE AI LLMs like ChatGPT
  • AI
    chess engines, sat‑nav route planning
  • Machine learning
    spam filters, film recommendations
  • Neural networks
    face recognition, aircraft collision avoidance
  • Generative AI
    ChatGPT and other large language models
02Getting our words straight

Cyber‑physical systems: software in a control loop

📡 SENSE 🧠 DECIDE ⚙️ ACT the physical world
The closed loop runs many times a second — no human in it.
✈️Aircraft

autopilots, collision avoidance

🚗Cars

braking, lane‑keeping, cruise control

🫀Medical devices

pacemakers, insulin pumps

⚡Power grids

balancing supply & demand

02Getting our words straight

Neural networks are moving into the control loop

Research figure: two aircraft in an encounter, with the range and angles the collision-avoidance networks read
Collision avoidance — 45 small networks issue the advisory (ACAS‑Xu).
Bak & Tran 2022, CC BY 4.0
Research figure: indoor photographs beside their predicted depth maps
Monocular depth — one photo in, a distance for every pixel out.
Huang et al., Sensors 2020, CC BY 4.0
Research figure: a forward camera view from a car with detection boxes over vehicles and people
Object detection — people, cars and riders boxed in real time.
Alibeigi et al. 2023 (ZOD), CC BY 4.0
Research figure: chest X-rays with disease findings boxed by a detection network
Medical imaging — chest X‑rays with the findings boxed by the network.
Cheng et al., Diagnostics 2024, CC BY 4.0
Research figure: a road photograph with a traffic sign detected and classified
Traffic‑sign classification — signs found and named in the camera frame.
Wang et al., PLOS One 2026, CC BY 4.0
Research figure: motorway photographs with the detected lane lines drawn on
Lane following — the lane lines the controller steers by.
Zhang et al., Sensors 2025, CC BY 4.0

So why are we letting them in at all?

03Why we’re turning to AI

Dull, dirty, dangerous: work to hand to machines

☢️ nuclear decommissioning Fukushima Unit 2 debris, 2024 🧍 → 🤖 ⛏️ open‑pit mining 8.3 bn tonnes hauled, driverless 🧍 → 🤖 💣 bomb disposal ‘Wheelbarrow’, Northern Ireland, 1972 🧍 → 🤖 🚛 road transport 38% of US work deaths, 2024 🧍 → 🤖 🌊 offshore platforms Spot on BP’s Mad Dog rig, 2020 🧍 → 🤖 🚀 another planet Perseverance: 88% self‑driven 🧍 → 🤖
Autonomy takes the person out of the blast radius — if the machine can be trusted alone.
≈3 million
work‑related deaths a year, worldwide

ILO estimate, 2023 — 330,000 in accidents, 2.6 million from occupational disease.

1.19 million
road‑traffic deaths a year

WHO, 2021 data — the leading cause of death at ages 5–29.

The strongest argument for automation is not simply convenience.

03Why we’re turning to AI

AI capabilities have jumped in the last 10 years

0%10%20%30% top‑5 error · ImageNet winner 20102011 28.2%25.8% hand‑engineered features + SVM 2012 16.4% AlexNet 201320142015 11.7%6.7%3.6% deep convolutional networks trained human annotator · 5.1%
ILSVRC: 1.2 million labelled photographs, 1,000 classes — top‑5 error of each year’s winner.
To 2011 — hand‑designed features

A person decided in advance what to measure — edges at particular angles, corners, patches of texture — and wrote it down as code. The computer only scored features a human had chosen.

2012 onward — learned features

Nobody writes the features down. The network starts from random numbers, sees 1.2 M labelled photographs, and every mistake nudges each number a little. Useful features appear on their own.

Where no first‑principles model exists, fitting to data beats writing rules. Retinal scans, skin lesions, speech, driving: the same curve. Caveat: 3.6% is an average over one fixed test set.

03Why we’re turning to AI

Learned controllers are beating classical control

🏁
Drone racing — Swift Kaufmann et al., Nature 2023

Beat three champion pilots — two of them world champions — 15 races to 10.

☀️
Fusion plasma — TCV tokamak Degrave et al., Nature 2022

One reinforcement‑learned policy replaced a bank of separate PID loops.

🐕
Legged robots — ANYmal Lee 2020 · Miki 2022, Sci. Robotics

Trained in simulation; crossed mud and rubble where the model‑based controller fell.

🌦️
Weather — GraphCast Lam et al., Science 2023

A 36.7 M‑parameter graph neural network beat ECMWF’s operational physics model.

Classical control is still vital but data can help us close gaps to real behavior.

03Why we’re turning to AI

Why did AI suddenly get so good?

1 · Labels were the bottleneck

ImageNet: 1.2 M photographs, each labelled by a person.

2 · Self‑supervised learning (GPT, 2018)

Objective: predict the next token — the data labels itself, so the training set can be the whole internet.

3 · Transformer (2017) + scaling laws (2020)

An architecture that introduces context — and more data and compute keep helping, predictably.

Love your neighbour as your . ? the model’s guesses for the hidden word self family own ✓ the right answer is already in the text — no human labelling needed
“Predict the next word” — still baffling it can be this effective.
“General methods that leverage computation are ultimately the most effective, and by a large margin.”
— Richard Sutton, The Bitter Lesson, 2019
03Why we’re turning to AI

The second recent jump in capability — an example

wav2vec 2.0 Large — trained on LibriSpeech (960 h) Whisper large‑v2 — zero‑shot, no tuning LibriSpeech audiobooks · the benchmark 2.7 2.7 TED‑LIUM conference talks 10.5 4.0 Common Voice crowd‑sourced, home mics 29.9 9.0 CallHome telephone conversations 34.8 17.6 CHiME‑6 dinner parties, distant mics 65.8 25.5 Average 13 other test sets 29.3 12.8 −55.2% errors
Word error rate (%), lower is better — Radford et al. 2022, Table 2. Whisper never trained on these test sets.
Previously task specific tools (gray)

Trained on 960 h of clean audiobooks. Superb there — brittle everywhere else.

Whisper (OpenAI, 2022)

Trained on 680,000 hours of everyday audio from the web — 700× more, most of it noisy.

55.2% fewer errors out of distribution

On speech neither had seen — and it still ties the specialist on the specialist’s own benchmark.

The breakthrough: breadth also appears to bring robustness.

03Why we’re turning to AI

Foundation models are entering the control loop

🦾
Robot arms — RT‑2 (2023)

A web‑trained vision‑language model that outputs robot actions directly.

🚗
Cars — end‑to‑end driving (2023)

One network, straight from camera pixels to steering and braking.

🏭
Humanoids — the factory floor (2024)

A vision‑language model directing a humanoid that loads car parts.

🎙️
Speech — the control tower (2024)

A fine‑tuned speech model transcribing live air‑traffic radio.

the input space benchmarks measure the average safety isdecided here
Accuracy is a property of a distribution. Safety is a property of every input.

They work brilliantly on average —
but what about edge cases people meet in reality?

04The problem

Images are points; perturbations lie in an ε‑ball

A photo is a vector of pixel values — a point in a million‑dimensional input space.

All imperceptible perturbations of the photo form an ε‑ball of radius ε around that point.

An adversarial perturbation slides the point in the ε‑ball — across the network’s decision boundary.

label “panda” “gibbon” ε the ε‑ball: all inputs within distance ε of the photo
The perturbation never leaves the ε‑ball — the label changes anyway.
04The problem

Is there any point in the ε‑ball
that flips the label?

The ε‑ball contains infinitely many images.
No amount of testing can try them all.

“Program testing can be used to show the presence of bugs, but never to show their absence!”
— Edsger Dijkstra, 1970
04The problem

Adversarial examples: imperceptible perturbations

🐼 “panda” · 57.7% + ε = 0.007 = 🐼 “gibbon” · 99.3%
Do you think your eyes could tell the image has been distorted?

Audio adversarial examples: a clip that sounds like ordinary talk transcribes as “okay google, browse to evil.com” — any audio, any target phrase.

Physical‑world adversarial example: a few stickers, and a stop sign is classified as “Speed limit 45.”

LLM jailbreaks are adversarial examples too: one optimised suffix of gibberish tokens, and an aligned chatbot writes the malicious code it just refused to write.

04The problem

The Turing Award keeps going to formal methods

The ACM A.M. Turing Award — “the Nobel Prize in Computing” — has 81 laureates. Twelve of them laid the foundations of formal methods: the logics, automata and algorithms for proving that programs do what they should.

Reasoning about programs
Automata · semantics · proof assistants
Edsger DijkstraEdsger Dijkstra1972

“composed correctly, not just debugged into correctness”

Robert Floyd1978

inductive assertions (1967) — the root of program verification

Tony HoareTony Hoare1980

Hoare logic (1969): {P} S {Q}

Michael RabinMichael Rabin1976

nondeterministic finite automata (1959, with Scott)

Dana ScottDana Scott1976

the same paper; later denotational semantics

Robin Milner1991

LCF: machine‑assisted proof; ML; CCS

Complexity
Temporal logic · model checking
Concurrency
Stephen CookStephen Cook1982

NP‑completeness (1971) — the wall we hit five slides from now

Amir PnueliAmir Pnueli1996

temporal logic for programs (1977): “always”, “eventually”

Edmund ClarkeEdmund Clarke2007

model checking (1981, with Emerson): search every state

Allen EmersonAllen Emerson2007

model checking; CTL, a branching‑time temporal logic

Joseph SifakisJoseph Sifakis2007

model checking, independently (1982, with Queille)

Leslie LamportLeslie Lamport2013

safety and liveness (1977); logical clocks; TLA+

Recently AI too: 2018 — Bengio, Hinton, LeCun; 2024 — Barto & Sutton.

Portraits via Wikimedia Commons — Dijkstra: Hamilton Richards, CC BY‑SA 3.0 · Rabin, Cook: Andrej Bauer, CC BY‑SA 2.5 · Scott: George M. Bergman, CC BY‑SA 4.0 · Hoare, Sifakis: Rama, CC BY‑SA 2.0 FR · Pnueli: David Monniaux, CC BY‑SA 1.0 · Clarke: Dennis Hamilton, CC BY 2.0 · Emerson: E. Allen Emerson, CC BY‑SA 4.0 · Lamport: lamport.org, free use

06From images to machines

One method, many specifications

Robustness

Every image in the ε‑ball around this photo gets the same label — noise, dirt, glare and all.

Fairness

Two applicants who differ only in a protected attribute must get the same decision — provably.

Reachability

Where can the closed‑loop system — network and physics — be, one second from now?

The ε‑ball of “nearby images” becomes a set of people, scripts of code , or of aircraft states. The maths doesn’t mind.

05Opening the box

A neural network: affine layers + ReLU activations

pixels in affine: Wx + b affine: Wx + b label out
Affine map, then ReLU. Repeat a few hundred times.
1 · Affine layer — multiply & add

Wx + b — like 3×flour + 2×sugar. Well‑behaved, fully predictable.

2 · ReLU — max(0, x)

The only non‑linearity. All the intelligence — and all the trouble — lives here.

05Opening the box

Affine layers propagate the whole set at once

affine: Wx + b the input set: the ε‑ball the exact reachable set
Stretched, rotated, translated — no point escapes: the polytope maps to a polytope, exactly.

So instead of testing images one at a time, we propagate the entire infinite ε‑ball through the layer in one step.

05Opening the box

ReLU is not linear — it is piecewise linear

xReLU(x) = max(0, x)kink at x = 0x ≤ 0 : y = 0x ≥ 0 : y = x
One kink at x = 0 — and either side of it, a straight line: y = 0 below, y = x above.
x₁x₂x₁ = 0ReLU(x₁) = 0ReLU(x₁) = x₁every x₁ becomes 0unchanged
Cut the set at the kink and each half meets an affine map — exact.

Approach 1 — exact: split the set and each half meets an affine map again — handled exactly, no approximation anywhere. The price: one set becomes two.

05Opening the box

Every straddled ReLU doubles the sets

ReLU (x = 0) inactive → 0 active → identity 1 → 2 → 4 → 8 → 16 …
Exact — but every straddled ReLU doubles the activation patterns.

Inside a real network, every ReLU the set straddles forces the split — one set becomes two.

Each set meets more ReLUs in the next layer.
Two becomes four. Four becomes eight…

State‑space explosion

Just 50 straddled ReLUs → 2⁵⁰ ≈ 1,000,000,000,000,000 activation patterns. More than we can store.

05A $1,000,000 aside

Verifying a ReLU network is NP‑complete

In 2017, researchers proved ReLU network verification is NP‑complete — it joins the class of problems computer science believes admit no polynomial‑time algorithm.

Whether that class truly has no polynomial‑time algorithm is the P vs NP problem — one of the seven Millennium Prize Problems. Solve it: $1,000,000.

Only one of the seven has ever fallen — though this month an AI‑generated proof of Navier–Stokes is under review. For P vs NP, ~9 in 10 experts bet P ≠ NP. So we don’t wait for a polynomial‑time algorithm — we over‑approximate.

Riemann Hodge Navier–Stokes ✓?claimed 2026 — under review Yang–Mills Birch–S‑D Poincaré ✓ solved — prize declined P vs NP $1,000,000 — still open the seven Millennium Prize Problems (Clay Institute, 2000)
05Opening the box

Approach 2 — over‑approximation

l u triangle relaxation two sides are the ReLU the chord is the error no case split, no blow‑up — one convex set
Two sides are ReLU(x) itself; only the chord adds error.
layer 1 layer 2 layer 3 the true set stays inside the over‑approximation — but error compounds
Fast and scalable — at the price of over‑approximation error.
05Opening the box

Abstract domains: box, zonotope, polyhedron

interval box cheap & loose · 💰 zonotope tighter · 💰💰 polyhedron tight & costly · 💰💰💰 the craft: the cheapest domain that still proves the specification
Recreated after Aws Albarghouthi, Introduction to Neural Network Verification. Our lab’s own abstract domain: the star set.
05Opening the box

Two methods: both sound, only one complete

Exact — sound & complete
✅ “Safe” means verified safe
❌ “Unsafe” comes with a real counterexample
…but the state space explodes. Only small networks finish.
Over‑approximate — sound
✅ “Safe” still means verified safe — it contains the true reachable set
⚠️ “Unsafe” might be spurious — a real counterexample, or just looseness?
…but it scales to real networks, fast.

In practice, a hybrid: over‑approximate first; case‑split only where “unsafe” appears.

06From images to machines

Zoom out: where formal methods already run

☁️
Amazon’s cloud database

Formal verification caught a data‑loss bug that needed a 35‑step chain of events to appear. No test plan on Earth finds that.

✈️
Airbus fly‑by‑wire

Formal verification proves the flight code is free of runtime errors — for every input, on every flight.

🔒
seL4 — an operating‑system kernel

Ships with a machine‑checked proof that the code matches its formal specification, line by line.

🛠️
CompCert — a verified compiler

Bug‑hunters found hundreds of bugs in mainstream rivals — and none in its formally verified core.

Industry’s sentence is changing — from “we tested it extensively”
to “we proved it cannot do the bad thing.”

06From images to machines

Reachability: one reach tube, every possible future

⚠ unsafe set (a building, a storm, another aircraft) initial set — every possible start now+1sreach sets+2s reach tube avoids the unsafe set — verified
The controller is a network, the aircraft is the plant — we propagate both forward together.
06From images to machines

Formal specifications: what “behaves” means

Safety

the bad thing never happens

Reach‑avoid

reach the goal set, avoiding the unsafe set

Liveness

the good thing eventually happens

Resilience

after a disturbance, recover within k steps

Written in temporal logic — mathematics for sentences about time:
□ ¬crash  ·  ◇ goal  ·  dip → ◇≤k recover

06From images to machines

The safety certificate can itself be a network

⚡ unsafe (boiler bursts, grid collapses) barrier · B(x) = 0 the reachable set — all trajectories
A barrier certificate: one function proving the unsafe set is never reached.

A barrier certificate is a function B(x), non‑increasing along every trajectory — so its zero level set is never crossed.

1 · A network proposes the barrier

Learned from data — even when the system’s dynamics are unknown.

2 · A formal checker verifies it

Conditions hold? Keep it — the certificate. Violated? The counterexample goes back into training. Repeat.

The network is allowed to be creative, because it never gets the last word. The checker does.

07Open challenges

Challenge 1 — LLMs are a million‑fold too big

10 thousand 1 million 100 million 10 billion 1 trillion number of learned parameters — each step is ×10 ← formally verified to date LLMs → ACAS‑Xu — one aircraft network · 13 thousand LeNet — digit classifier · 60 thousand ResNet‑50 — image classifier · 26 million VGG16 — the largest ever verified · 138 million GPT‑3 · 175 billion GPT‑4 · ~1.8 trillion (rumoured) probabilistic certificates me — under review Whisper‑ATC · 1.5 billion
Propagate an ε‑ball through a trillion parameters: exponential blow‑up exhausts every computer on Earth.
07Open challenges

Challenge 2 — no global Lipschitz constant

a Lipschitz function output change ≤ L × input change — always
A Lipschitz constant L: a global bound on sensitivity.
tiny step in →huge leap out 1/x near 0 — no Lipschitz constant
Transformer attention: provably not globally Lipschitz.

Even a sound and complete verifier can only report the truth — and sometimes the truth is: this model really is that sensitive.

07Open challenges

Challenge 3 — our proofs are local to one input

A photographed give-way sign — the one verified input
✓
verified robust — THIS image
Another photographed road sign — an unseen input?
Another photographed road sign — an unseen input?
Another photographed road sign — an unseen input?
Another photographed road sign — an unseen input?
Another photographed road sign — an unseen input?
Another photographed road sign — an unseen input?
Another photographed road sign — an unseen input?
Another photographed road sign — an unseen input?
the camera feed — each frame an unseen input

Local robustness proofs start from the image we tested.

A flying drone meets unseen inputs every frame — it can’t pause mid‑air to verify each one.

Can a guarantee generalise from the photos we tested to the next photo?

09Where the field is heading

2024: the year the big prizes went to machine learning

John HopfieldGeoffrey Hinton
Nobel Prize in Physics — Hopfield & Hinton

“for foundational discoveries and inventions that enable machine learning with artificial neural networks”. Hopfield network (1982): 30 nodes, 435 connections. Boltzmann machine (1985). Hinton also holds the 2018 Turing Award.

David BakerDemis HassabisJohn Jumper
Nobel Prize in Chemistry — Baker · Hassabis & Jumper

Half “for computational protein design”; half “for protein structure prediction”. AlphaFold2: structures for nearly all ~200 million known proteins; at CASP14 (2020), median backbone error 0.96 Å against 2.8 Å.

Richard Sutton
ACM Turing Award — Barto & Sutton

“for developing the conceptual and algorithmic foundations of reinforcement learning”. Temporal‑difference learning (Sutton, 1988); the textbook (1998). The paradigm behind the tokamak and ANYmal controllers of slide 16.

“…when we create digital beings that are more intelligent than ourselves. We have no idea whether we can stay in control.”
— Geoffrey Hinton, Nobel banquet speech, 10 December 2024
08What we can do

Answer 1 — probabilistic certificates

🎙️
Air‑traffic control, by radio

“Speedbird 245, descend flight level 80” — heard by a 1.5‑billion‑parameter speech model over a noisy VHF channel.

Sample channel noise 1,000 times

Independent draws of real VHF‑band noise. Count how often the meaning survives — which aircraft was addressed, which altitude was given.

The certificate 📜

“For the next, unheard burst of noise: under 5% probability the meaning changes — Clopper–Pearson, confidence 99.9999%.”

No exact verifier scales this far; sampling needs only forward passes.

random perturbations — adversarial attack
Honest small print: guards against random noise, not adversarial attack.
08What we can do

Answer 2 — input‑agnostic certificates

Level 1 — per‑input (local) certificates
photographed road sign
photographed road sign
photographed road sign
photographed road sign✗
photographed road sign
photographed road sign
photographed road sign
photographed road sign

each sign: 1,000 noise samples → its own local certificate

Level 2 — population certificate
📜 “At least 9 in 10 of the images this camera will ever meet are robust — confidence 99.9999%.”

The tested images become random samples themselves — a second scenario‑approach bound covers unseen inputs: images nobody has seen yet.

08What we can do

Answer 3 — runtime monitoring

predicted reach tube observed trajectory divergence! 🔔 a second controller corrects the course time →
A verified controller flies — the monitor watches, every split‑second, for the moment model and reality diverge.

A verified controller is flying — but its proof was made against a model of the world. On board, every split‑second, the monitor checks model against reality.

The moment they diverge — the world doing something the model never promised — a second controller steps in and corrects the course.

Figure 3(b) of work under review: six doorway-transit flights; blue monitored tracks stay inside the certified set, orange unmonitored ones leave it 177 times

We flew it — a union of neural control barrier certificates, in our cluttered two‑room lab, through a 90 cm doorway. With the monitor: 0 of 4,810 states left the certified set. Without it: 177.
Fig. 3, work under review

09Where the field is heading

Neuro‑symbolic AI: neural proposer, symbolic checker

neural language modelguesses an auxiliaryconstruction: “add point D…”System 1 · intuitionsymbolic deduction enginederives every consequence;each step checked (DD + AR)System 2 · rulesproposalretryproof found — every step verified(verified logically and numerically)ABCDauxiliary point D — the creative step
AlphaGeometry (Trinh et al., Nature 2024): intuition proposes, deduction disposes.
AlphaGeometry — Nature, January 2024

25 of 30 Olympiad geometry problems — the previous best method solved 10.

IMO 2024 — AlphaProof + AlphaGeometry 2

Silver‑medal standard — and every proof machine‑checked in Lean.

IMO 2025 — Gemini Deep Think

Gold‑medal standard, in plain language, graded by humans. Faster — no certificate.

the network proposes the solution, the symbolic checker certifies it

09Where the field is heading
“We are not trying to meet some abstract production quota of definitions, theorems and proofs. The measure of our success is whether what we do enables people to understand and think more clearly and effectively about mathematics.”
— William Thurston, Fields Medallist · “On Proof and Progress in Mathematics”, 1994

Do we lose something by taking this approach? Was math only ever about finding an answer?

09Where the field is heading

New architectures, new verification problems

Mixture of experts — sparse activation

A router wakes 2 of 8 experts per token — most of the network stays dark.

top‑k routing is a hard case split — the ReLU split of slide 28, over whole sub‑networks

Kolmogorov–Arnold networks — learnable edges

The learned pieces live on the edges, as splines — interpretable, ≈10× slower to train.

small symbolic pieces can be handed to an SMT solver: KAN barrier certificates (IJCAI 2025)

Graph neural networks — message passing

Each node listens to its neighbours — molecules, power grids, the weather model of slide 16.

unbounded graphs: reachability undecidable; bound the node degree and it becomes decidable (Sälzer & Lange, ICLR 2023)

Every new architecture re‑asks the question of slide 28: where do the case splits hide?

09Where the field is heading

Neuromorphic hardware: the brain runs on ≈20 W

clocked digital hardwareevery unit, every tick — whether or not anything changedspiking (neuromorphic) hardwarefires only when input crosses threshold — silence is almost freen1n2n3n4n5n6the model: a brain, ≈20 W
Two ways to spend energy: on every tick, or only on events.
Spiking, event‑driven chips

Energy spent only on events: TrueNorth classified video at 63 mW; Hala Point packs 1.15 B neurons — research prototypes.

The fine print

Intel’s own survey: ordinary feedforward networks see “modest if any benefit” — the gains need spikes used as spikes.

Why it matters

Data centres: ≈415 TWh in 2024, heading for ≈945 TWh by 2030, AI the main driver (IEA 2025).

robustness must cover spike timing too — today’s SNN verifiers are “barely scalable” (Seong et al., 2024)

09Where the field is heading

“Recommendations to my fellow AI scientists”

Yann LeCun on stage at ECCV 2026 in Malmö beside his keynote slide listing four abandon-in-favour-of recommendations and a red final line about LLMs
Yann LeCun’s keynote slide — ECCV 2026, Malmö, 8‑12 September
The programme, in four lines
generative models→joint‑embedding architectures probabilistic models→energy‑based models contrastive methods→regularised methods reinforcement learning→model‑predictive control

the fine print under the fourth line: “Use RL only when planning doesn’t yield the predicted outcome, to adjust the world model or the critic.”

“IF YOU ARE INTERESTED IN HUMAN‑LEVEL AI, DON’T WORK ON LLMs”

the last line of the slide — verbatim, and red in the original

New machines.
Old questions.
Thank you.

Happy to share my slides — and more resources.

Ben Wooding · woodingben.com

1 / 1
→ advance · F fullscreen