Remote Sensing Foundation Models: How to Adapt AI for Satellite Data

Research published on arXiv as a review chapter examines the application of foundation models to Earth observation through satellites and airborne sensors.

What are Remote Sensing Foundation Models

Remote Sensing Foundation Models (RSFM) are large models trained on archives of satellite and airborne imagery. Unlike traditional computer vision models, which learn on photos of cats and faces, RSFM see the planet: cities, fields, forests, clouds, oceans.

Example: you want to predict wheat yield in a specific region. Instead of writing a model from scratch for just this task, you take an already trained RSFM on satellite imagery, adapt it to your task — and the model starts working, because it already knows how to read satellite data.

Why a Universal Foundation Model Doesn't Work

Here's the problem: satellite data is physically different from regular photos.

• Different sensors, different wavelengths (infrared, red, blue, short-wave)
• Atmospheric distortions: clouds, haze, aerosols change what the satellite sees
• Low spatial resolution (10-100 meters per pixel, not like smartphone photos)
• Time limitations: a satellite flies over a location once every few days, you need to catch the moment
• Physics of measurements: spectral reflectance follows physical laws, not arbitrary patterns in training data

There is no single universal model that works equally well for all Earth observation tasks. Benchmarks are contradictory. Model A is better at agriculture prediction, model B is better at building type classification. This is because each RSFM was engineered for its own dataset, and knowledge transfer requires tuning.

Examples: When Adaptation is Critical

Harmful algae prediction. A neural network learns to predict toxic algal blooms in lakes and seas. Just spectral characteristics are not enough — you need to account for a spectral mask that tells you where exactly in the image to look for blooms. This requires physics-informed masking: the model must understand not just patterns, but the physics of why algae appear there.

Monitoring station selection. Another task: where to place a new environmental monitoring station? A model with adaptation, trained with reinforcement learning, knows how to select locations that provide maximum information about climate and nature's state.

What's Next

The next generation of RSFM should be evaluated not just by standard accuracy metrics (accuracy, F1), but also by:

• Modality-specific adaptation: the model should transfer well to new sensor types and wavelengths
• Physical plausibility: representations in the neural network should correspond to real satellite measurement physics, not just data patterns

This means that in the future RSFM will be built taking into account not just machine learning, but domain expertise — physics, astronomy, climatology.

What This Means

In complex specialized domains (Earth observation, medicine, scientific simulations) foundation models are not a panacea. A large model trained on billions of parameters is simply not enough. We need adaptation to the physics of the task, to the specifics of the data, to real limitations. The next step in AI is not just more parameters, but more physical meaning.