Matrix light transport · Ari Vasvaria

Most realistic renderers are path tracers. They shoot random rays through each pixel, follow them as they bounce, and average the results. With few rays the picture is noisy, so games add ray-tracing hardware and a denoiser to clean it up.

I wanted to see how far I could get with no random sampling at all. This renderer splits every surface into small patches and builds a matrix that says how much light each patch sends to every other patch. Solving with that matrix gives the lighting for the whole scene at once.

The matrix multiplications run on the GPU's tensor cores, the part normally used for neural networks. The output has no noise because nothing in it is random.

The box room with the ceiling light changing colour. — Eight frames with the light changing colour. Each one is solved again from the start and takes about 30 ms. These are the renderer's real output at 128×128, enlarged.

Eight frames with the light changing colour. Each one is solved again from the start and takes about 30 ms. These are the renderer's real output at 128×128, enlarged.

Five renders in a row with the shorter box moving sideways a little in each. — The short box moving one centimetre per frame. A frame takes 26 ms including rebuilding the matrices for the new geometry.

Two nearly identical renders of the box room side by side. — Left: this renderer at 256×256. Right: a path tracer I wrote as the reference, run to 32,768 samples per pixel.

Measurements

More patches make the solve slower. Past a point they stop making it more accurate, because the remaining error comes from how pixels are sampled.

Surface patches	Lighting solve	Whole frame	Error vs reference
608	1.45 ms	22 ms	3.9%
2,432	4.54 ms	45.2 ms	3.32%
5,472	16.07 ms	97 ms	3.33%

It only handles matte surfaces so far: no mirrors, glass or gloss. I also have not yet compared its speed against a path tracer at the same accuracy. Both are next.

CUDA C++ builds the operators: patch-to-patch form factors with visibility tested at patch centres, and a camera matrix that projects patch radiosity to pixels with 2×2 up to 8×8 samples per pixel. The solve and projection are FP16-operand, FP32-accumulate products through an explicit nvcuda::wmma kernel, checked against an FP64 product of the same quantised operands.

Timings are from an RTX 5070 laptop GPU at 128×128. With 5,472 patches fixed, going from 2×2 to 8×8 pixel samples lowers the error from 3.33% to 1.14%. The 256×256 frame above has 1.43% error.

Because patches are finite and visibility is tested at centres, this is not an unbiased renderer. The reference path tracer is: PCG32 streams per pixel and sample, cosine-weighted sampling, light sampling combined with the power heuristic, Russian roulette after the fourth bounce, no depth cap, no clamping.

Four path-traced renders of the box room, from very noisy to clean. — The reference path tracer with more and more samples. This is the noise the matrix renderer does not have.