In March 2014, a passenger jet disappeared over the Indian Ocean with 239 people aboard. International search teams scanned a stretch of sea floor larger than Pennsylvania and came back with nothing. More than a decade later, the families still have no answers. That absence is exactly what drives John Leonard, a roboticist at MIT CSAIL who has spent nearly 40 years asking one question: how does a robot find its way through a place no one has ever mapped for it?
The ocean breaks every assumption AI was built on
The scale of the problem is hard to absorb. The ocean covers more than 70% of the planet, and less than a third of its floor has ever been mapped in detail. There is no Wi-Fi underwater. There is no GPS. The acoustic signals that robots use to communicate are slower than a 1980s telephone modem. In murky water, cameras go nearly blind. Every shortcut that makes land-based AI reliable simply disappears.
To illustrate just how hard the task is, Leonard frames it around a deliberately ordinary object. ‘I have a challenge problem,’ he explains, ‘which is to take a coffee mug and throw it off a ship into the ocean, and send a robot down to find it.’ It sounds almost trivial until you consider the darkness, the current, the absence of landmarks, and the fact that the robot has to orient itself while building a map of a place no one has charted before.
The answer to the visibility problem borrows from nature. Dolphins navigate by echolocation, sending out sound and reading what bounces back. Leonard and his colleagues are building robots that do the same thing, using sonar sensors that capture a horizontal field of acoustic returns from surrounding objects. The challenge is that sonar images look nothing like camera images, and the datasets available for training are tiny by AI standards.
The team’s solution is to let the camera teach the sonar. In clear water, matched pairs of camera images and sonar images are captured at the same moment. Two models are trained side by side so that both produce similar internal representations of the same object. Once that relationship is learned, a sonar image taken in murky water, with no camera image available, can still be interpreted accurately. Leonard has built up tens of thousands of images of oyster baskets and lobster pots shot from multiple angles to make this work, though the approach requires far less labeling effort than manually annotating raw sonar data.
Finding something is only half the job
At the MIT Marine Autonomy Lab on the Charles River, the research is scaling up. Rather than a single vehicle searching alone, Leonard envisions cooperative swarms: underwater robots sharing navigation data, surface vehicles relaying GPS and satellite communications down acoustically, and aerial drones covering the wider search area above. Behind him in the lab sit half a dozen prototype vehicles already being tested in exactly that configuration.
But retrieval is where things get genuinely hard. Today, bringing an object back from the sea floor still requires a human scuba diver. The robots can navigate, but picking something up and setting the surrounding environment back the way it was before is a frontier no machine has crossed. Leonard calls the goal ‘undo for the physical world,’ a kind of Ctrl-Z for physical space. A robot would need to register the original arrangement of objects, interact with them, and restore everything to its prior state. He describes this as ‘a sort of Turing Test of Physical AI,’ because it demands not just perception but reasoning about uncertainty and objects in relation to each other.
Leonard is candid about where the field stands. Looking ahead, he suspects that the models in use today will seem like brute force in retrospect, built on billions of training images to do what a child manages in minutes. But the ocean, precisely because it breaks so many of AI’s standard assumptions, may be the pressure test that forces genuinely new architectures to emerge, ones with applications far beyond the sea floor.
The coffee mug, still on the ocean floor
Somewhere at the bottom of an unmapped stretch of water sits Leonard’s coffee mug, waiting.
A robot that can find it, pick it up, and leave everything else exactly as it was would represent something no machine has managed yet. The families who lost someone over the Indian Ocean in 2014 are still waiting for a different kind of answer, and the distance between those two facts is exactly the problem Leonard has chosen to spend his career closing.
