Terrapattern is an interface for finding "more like this, please" in satellite photos. Simply click an interesting spot on Terrapattern's map, and it will find other locations that look similar. Terrapattern is particularly useful for locating things that aren't usually indicated on maps. We think Terrapattern is important because it's open-source, open-access, and open-ended. Terrapattern can be briefly described as "a visual search engine for satellite imagery", "similar-image search for satellite photos", or "a prototype for visual query-by-example in satellite imagery". We hope it will help citizen scientists, humanitarians, journalists, and other curious people to discover "patterns of interest". Technically, the project uses a Deep Convolutional Neural Net (DCNN) to assist with image recognition. For more information, please see http://terrapattern.com. Terrapattern is an open-source project created by a collaborative team of artists, creative technologists, and students. Terrapattern is not a company or startup; it is an experimental research prototype, developed in a university setting, whose purpose is to present a new way of exploring, understanding, and organizing the world. Terrapattern went live on May 24th, 2016. Terrapattern was created by Golan Levin, David Newbury, and Kyle McDonald with the assistance of Irene Alvarado, Aman Tiwari, and Manzil Zaheer. The project was developed at the CMU Frank-Ratchye STUDIO for Creative Inquiry with support from the John S. and James L. Knight Foundation.

Overview

Terrapattern is a prototype for helping people quickly scan extremely large geographical areas for specific visual features. It seeks to help people identify, characterize and track indicators which have not been detected or measured previously, and which have sociological, humanitarian, scientific, or cultural significance. Instead of searching for a location based on a text query, with Terrapattern one can click on a place in a map to find other similar-looking places.

Terrapattern Tool

Go to the Terrapattern website to try the service. The about and FAQ pages contain more information about our technical process, mainly covering data gathering and the deep convolutional neural net (CNN) we used to assist with image recognition.

This version of the system is meant to be a proof of concept - a prototype suggesting new possibilities for using maps and searching through visual information. A more robust version 2 is currently in development.

Tools: Ruby, Processing, Javascript, HTML, CSS, Sketch, geo libraries

Year: 2016

Team: Golan Levin, David Newbury, Kyle McDonald, Aman Tiwari, Manzil Zaheer

Role: Project ideation, dataset creation for CNN, prototype web design and development

Link: Website, Github, Golan’s project description

Examples

Terrapattern is ideal for discovering, locating and labeling typologies that aren't customarily indicated on maps. These might include ephemeral or temporally-contingent features (such as vehicles or construction sites), or the sorts of banal infrastructure (like fracking wells or smokestacks) that only appear on specialist blueprints, if they appear at all. It is important to point out that the Terrapattern system was not trained on any of the categories shown below, but instead recognizes them because of their common visual features.

Here, for example, are golf course sand traps identified in the Pittsburgh metro region:

Here, Terrapattern identified some of Pittsburgh's school bus depots:

A collection nautical wrecks in NYC-area coastal waters:

There are more examples on the Terrapattern site.

Terrapattern Becomes a Product

Terrapattern ended up inspiring Descartes Lab to create their product GeoVisual Search. From their Medium article:

“Last summer a group of artists and coders created Terrapattern, a ground-breaking demonstration of visual search over satellite imagery. We loved it. The demo aligned with many ideas we had been kicking around at Descartes Labs, and it was great to see somebody just go out and do it. It got us thinking about how we could extend visual search beyond cities, out to entire countries, or even the whole world.
Today we’re sharing our own demonstration of this technology, GeoVisual Search.”

Selected Press

Blakemore, Erin. "This New Satellite Project Helps People Find Patterns in City Spaces". Smithsonian, 6/9/2016. [PDF]

Brown, Eileen. "Terrapattern search engine finds patterns in the Google Earth landscape". ZDNet, 7/14/2016. [PDF]

Heller, Piotr. "Suchmaschine für Satellitenbilder: Wir wissen, wo euer Flugzeug steht". Spiegel Online, 6/3/2016. [PDF]

Manaugh, Geoff. "Subverting Our New Space Overlords". The Atlantic, 6/9/2016. [PDF]

Meyer, Robinson. "The Thrill of Terrapattern, a New Way to Search Satellite Imagery". The Atlantic, 5/27/2016. [PDF]

Stinson, Liz. "Terrapattern is Like a Search Engine for Satellite Imagery". WIRED.com, 5/27/2016. [PDF]

Twilley, Nicola. "Meet Terrapattern, Google Earth's Missing Search Engine". The New Yorker, 5/25/2016. [PDF]

Visnjic, Filip. "Terrapattern – Neural network visual search tool for satellite imagery". CreativeApplications.net, 5/25/2016. [PDF]

Whitehead, Timothy. "Terrapattern, the search engine for imagery". Google Earth Blog, 5/27/2016. [PDF]