Cloudless ‘Synthetic’ Sentinel-2 Data

Before/After True-Color Image (Left: Sentinel-2, Right: Synthetic Sentinel-2 Data).

The Elevator Pitch:

For anyone unfamiliar with what we do or for the people who just stumbled upon this article on Medium, I’ll quickly give you an elevator pitch for our company, ClearSky Vision:

Synthetic Data:

I think it’s important to first clarify what we mean by ‘Synthetic Data’ because it’s already a term used in machine learning and many other industries. That makes the term a bit ambiguous and maybe even misleading. But I also think it’s the most descriptive word of what we are doing with a fair disclaimer for anyone using it (if you know a better word please tell me).

The Easy and The Hard Cases:

It should come as no surprise that there are easy cases and very difficult cases when increasing the temporal frequency of optical satellites. Our favorite weather is either hazy, has a lot of cumulus clouds (the small clouds you see in the summer), or has a lot of cirrus clouds (thin high altitude clouds). In these situations, several optical wavelengths penetrate the clouds nicely and make it possible to recreate the entire image and all spectral bands (more or less) perfectly. That’s the easy stuff! However, it is far from trivial (as it requires state-of-the-art deep learning and very smart coders) but relatively speaking it’s much easier than cases with dense clouds looming over Europe in autumn or winter.

Before/After True-Color Image.
Before/After True-Color Image.
Before/After True-Color Image.
Sentinel-2 image from the 17th of May 2020. The lime-green fields in the lower part are rapeseed blooming.

The Future We Want to Build:

I strongly believe the market is going in this direction even though synthetic data, mosaicking tools, cloud-free composites, and data fusion techniques are rarely used in agricultural monitoring and food security applications. Don’t get me wrong, I understand the hesitation and the need for innovation before utilization, however, the amount of discarded optical imagery is staggering. It feels like a waste not utilizing all this data better and, sadly, just efficiently spotting and discarding this data is being recognized as state-of-the-art innovation... That is, however, not entirely false since it’s very inefficient sending clouded data (white pixels) from server to server to customer while disrupting the workflow of everyone interacting with it. So when we say cloudless imagery we do not mean “good cloud mask” as is the case with s2cloudless but we mean “making clouded imagery cloudless” or “making unusable data useful again”. Ehh, the last one sounds too much like a cheap slogan.

Continuous Monitoring:

It’s understandable why precision agriculture has such a slow and low uptake in Europe since all decision-making information is lagging behind — and sporadic information from when the sky is blue is mere snapshots of that particular time. What are the odds of you receiving a usable image when it really matters? I would say highly unlikely. So the information you utilize will almost always be outdated imagery based on sporadic (unpredictable) weather-based timing — or god’s will if you are religious. It limits the positive impacts of precision agriculture and makes you uncertain about when the next information might come around — a day, three days, a week, or a month?

One column for each day. Rows from top to bottom: Sentinel-2 True-Color, Synthetic True-Color, Sentinel-2 NDRE, and Synthetic NDRE. NDRE is a Red-Edge Biomass Index.
Growth of Spring Barley across 20 days in May (Denmark, 2020)
11 Days of Sentinel-2 and ClearSky Vision (Denmark, 2020)

Time-Lapse:

I have a small treat for anyone still reading, as I think time-lapses are the most fascinating use-case that doesn’t directly bring any value. It’s more for ‘show and tell’ — unless you actually use the georeferenced data behind the video. Anyhow, below is a time-lapse showing seven months from April to October.

True-Color and NDVI Time-Lapse (April to October)

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Morten Fjord Pedersen

Morten Fjord Pedersen

Co-founder of ClearSky Vision | Cloudless Sentinel-2 Data | Data Fusion with Deep Learning | I’m always out of free disk space.