MLE & AI Engineering

The test was
protecting
the bug.

TensorFlow returned the wrong gradient.
An existing test expected it.

I work on ML frameworks and the systems around them. These are the bugs I followed, the patches that merged, and the reasoning a diff can't quite hold.

Read the TensorFlow story
Fig. 01 / TensorFlowAt the origin
A zero value with a nonzero slope The straight line f(x) = x times log(3.1) crosses the origin. Its slope is about 1.1314, including at zero. The old gradient incorrectly returned zero at that point. xf(x)0 slope = log(3.1)old gradient: 0 f(x) = x · log(3.1)
Forward value0.0000
Correct gradient1.1314
Old gradient0.0000
The value is zero. The slope is still there.

A small counterexample.
A surprisingly persistent assumption.

01 / From the debugging notebook

Follow the failure.

Four stories, with the code
and tests behind each one.

01

TensorFlowML framework correctness

The test was protecting the bug

TensorFlow returned a zero gradient, and an existing test agreed. A straight line through the origin gave me a reason to doubt them both.

4 min read
03

CeleryConcurrency

The socket race started in a destructor

I followed a Redis socket race out of Celery's polling loop and into result cleanup. The fix needed a lock that could survive callbacks and a fork.

4 min read

02 / Beyond the write-ups

A few merged patches.

All selected contributions