The WebNN Programming Model

WebNN follows a graph-based execution model. You build a computational graph, compile it for a specific hardware backend, then execute it:

// Check for WebNN support
if (!('ml' in navigator)) {
 console.log('WebNN not supported, falling back to ONNX Runtime WASM');
 return fallbackToWasm(modelPath);
}

// Create a context targeting the NPU
const context = await navigator.ml.createContext({ deviceType: 'npu' });
const builder = new MLGraphBuilder(context);

// Build a simple inference graph
const input = builder.input('input', {
 dataType: 'float32',
 dimensions: [1, 3, 224, 224], // NCHW format
});

// Conv2d → ReLU → Pool → ... (simplified)
const conv1Weight = builder.constant(
 { dataType: 'float32', dimensions: [32, 3, 3, 3] },
 new Float32Array(conv1WeightData),
);

const conv1 = builder.conv2d(input, conv1Weight, {
 padding: [1, 1, 1, 1],
 strides: [2, 2],
});

const relu1 = builder.relu(conv1);
const pool1 = builder.averagePool2d(relu1, {
 windowDimensions: [2, 2],
 strides: [2, 2],
});

// Compile the graph
const graph = await builder.build({ output: pool1 });

// Execute with real data
const inputBuffer = new Float32Array(1 * 3 * 224 * 224);
// ... fill inputBuffer with image data ...

const inputTensor = await context.createTensor({
 dataType: 'float32',
 dimensions: [1, 3, 224, 224],
 writable: true,
});
context.writeTensor(inputTensor, inputBuffer);

const outputTensor = await context.createTensor({
 dataType: 'float32',
 dimensions: [1, 32, 56, 56],
 readable: true,
});

await context.dispatch(graph, { input: inputTensor }, { output: outputTensor });
const results = await context.readTensor(outputTensor);

Building the graph manually is tedious. In practice, you load an ONNX model and let the runtime handle the graph construction.

ONNX Model Execution

The practical path to WebNN is through ONNX Runtime Web, which can use WebNN as a backend:

import * as ort from 'onnxruntime-web';

// Configure ONNX Runtime to use WebNN
ort.env.wasm.numThreads = 4;

async function runInference(imageData) {
 const session = await ort.InferenceSession.create(
 'mobilenet_v3_small_int8.onnx',
 {
 executionProviders: [
 {
 name: 'webnn',
 deviceType: 'npu', // 'npu', 'gpu', or 'cpu'
 powerPreference: 'default', // 'default', 'low-power', 'high-performance'
 },
 ],
 // Fall back to WASM if WebNN is unavailable
 fallbackExecutionProviders: ['wasm'],
 },
 );

 // Preprocess image to tensor
 const tensor = new ort.Tensor('float32', preprocessImage(imageData), [1, 3, 224, 224]);

 // Run inference
 const startTime = performance.now();
 const results = await session.run({ input: tensor });
 const elapsed = performance.now() - startTime;

 console.log(`Inference time: ${elapsed.toFixed(2)}ms`);

 // Post-process results
 const probabilities = softmax(results.output.data);
 return getTopK(probabilities, 5);
}

function preprocessImage(imageData) {
 // Resize to 224x224, normalize to ImageNet mean/std
 const canvas = new OffscreenCanvas(224, 224);
 const ctx = canvas.getContext('2d');
 ctx.drawImage(imageData, 0, 0, 224, 224);

 const pixels = ctx.getImageData(0, 0, 224, 224).data;
 const float32 = new Float32Array(3 * 224 * 224);

 const mean = [0.485, 0.456, 0.406];
 const std = [0.229, 0.224, 0.225];

 for (let i = 0; i < 224 * 224; i++) {
 float32[i] = (pixels[i * 4] / 255 - mean[0]) / std[0]; // R
 float32[i + 224 * 224] = (pixels[i * 4 + 1] / 255 - mean[1]) / std[1]; // G
 float32[i + 2 * 224 * 224] = (pixels[i * 4 + 2] / 255 - mean[2]) / std[2]; // B
 }

 return float32;
}

Real Benchmarks

Three models were benchmarked across different backends on two machines. The key numbers:

Machine 1: Intel Core Ultra 7 (Meteor Lake), 16GB RAM, Chrome 126

Model WASM (CPU) WebGPU WebNN (NPU) WebNN (GPU)
MobileNetV3-Small (INT8) 18ms 4.2ms 2.8ms 3.1ms
MobileNetV3-Large (FP32) 45ms 8.5ms , 7.2ms
ResNet-50 (INT8) 125ms 15ms 11ms 12ms
DistilBERT (INT8, seq=128) 340ms 42ms 28ms 35ms

Machine 2: Apple M3 Pro, 18GB RAM, Chrome 126

Model WASM (CPU) WebGPU WebNN (GPU)*
MobileNetV3-Small (INT8) 12ms 3.8ms 3.2ms
MobileNetV3-Large (FP32) 32ms 6.1ms 5.8ms
ResNet-50 (INT8) 95ms 12ms 10ms
DistilBERT (INT8, seq=128) 280ms 38ms 32ms

*WebNN on macOS uses the CoreML delegate, which routes to ANE (Apple Neural Engine) for supported operations.

Key observations:

  1. WebNN NPU is fastest for INT8 models on hardware that has an NPU. The INT8 constraint is real, most NPUs only support integer operations.

  2. WebGPU is the reliable second choice. It works everywhere with a capable GPU and handles FP32 models well.

  3. WASM is 5-15x slower but is the universal fallback. For models that need to run everywhere, ship WASM as the baseline and opportunistically use WebNN/WebGPU.

Privacy-Preserving AI Patterns

The most compelling use case for on-device inference isn't performance, it's privacy. When the model runs in the browser, the data never leaves the device.

// Medical image analysis, data stays on device
async function analyzeSkinLesion(image) {
 const session = await getOrCreateSession('dermnet_classifier_int8.onnx');

 // Preprocess locally
 const tensor = preprocessDermatologyImage(image);

 // Inference locally, image never sent to a server
 const results = await session.run({ input: tensor });

 // Only the classification result (a single integer) is sent to the server
 // for logging/analytics, not the image
 await reportClassification(results.classification, {
 confidence: results.confidence,
 // No PII, no image data
 });

 return results;
}

This pattern has been applied successfully in document classification for legal tech applications. Documents are OCR'd locally (using Tesseract.js), then classified by a DistilBERT model running through WebNN. The document content never leaves the device. For firms handling attorney-client privileged documents, this is not a nice-to-have, it is a compliance requirement.

Model Optimization for On-Device

The model has to be small and fast. Here is the recommended optimization pipeline:

# Step 1: Export to ONNX from PyTorch
import torch
import onnx

model = load_trained_model()
dummy_input = torch.randn(1, 3, 224, 224)
torch.onnx.export(model, dummy_input, "model_fp32.onnx", opset_version=17)

# Step 2: Quantize to INT8 with ONNX Runtime
from onnxruntime.quantization import quantize_dynamic, QuantType

quantize_dynamic(
 "model_fp32.onnx",
 "model_int8.onnx",
 weight_type=QuantType.QInt8,
)

# Step 3: Optimize graph for WebNN
import onnxoptimizer
model = onnx.load("model_int8.onnx")
optimized = onnxoptimizer.optimize(model, [
 'eliminate_deadend',
 'fuse_bn_into_conv',
 'fuse_consecutive_transposes',
])
onnx.save(optimized, "model_int8_optimized.onnx")

Result: MobileNetV3-Small goes from 14.3MB (FP32) to 3.8MB (INT8 quantized). That's small enough to cache in a Service Worker.

// Cache the model in a Service Worker for offline use
self.addEventListener('install', (event) => {
 event.waitUntil(
 caches.open('ml-models-v1').then((cache) =>
 cache.addAll([
 '/models/mobilenet_v3_small_int8.onnx',
 '/models/distilbert_sentiment_int8.onnx',
 ]),
 ),
 );
});

Wrapping Up

WebNN bridges the gap between "ML in the browser is a toy" and "ML in the browser is production-viable." The hardware is there, NPUs ship in every new Intel and Qualcomm laptop. The API is there, Chrome supports it today. The models are there, quantized ONNX models run at interactive frame rates.

The use cases where this matters most: privacy-sensitive applications where data can't leave the device, latency-sensitive applications where network round-trips are unacceptable, and offline-capable applications that need ML without connectivity.

For everything else, calling an API is still simpler. But the set of "everything else" is shrinking fast.