AI Error in Estimating Hidden Solar Generation Rises from 0.104 to 0.185 kW

Grid operators want to see how much electricity rooftop solar panels generate, but most homes are visible only through "net" smart meters: the device reports the difference between consumption and generation, not each quantity separately. Deep models can separate this signal, but a new arXiv paper shows that their usefulness comes down to a question: for whom, and when, is the model applied? On Ausgrid data, the best model's daytime mean absolute error (MAE) was 0.104 kW for already-known homes in a future test period and 0.185 kW for homes the model had not seen, over the same period. That is a gap of almost 1.8 times.

Four Regimes Instead of One Averaged Number

Such models are usually evaluated with a single figure on a shared test set. The authors split application scenarios into four deployment regimes: the model serves known customers, new customers, future periods, or new regions. This separates transfer between households from extrapolation over time. The baseline was Ausgrid smart meter data; the Pecan Street dataset was used to test in another region.

The researchers fixed training under a single protocol and then compared representative deep sequence architectures. The result is awkward for the usual benchmark logic: with the training contract fixed, the deployment regime explains a much larger share of the error spread than the choice of model itself. Changing the architecture for a few percent makes less sense than honestly determining what data the system will work on.

Key Figures

• Context: a fixed 48-hour window.
• Daytime MAE of the best model on known homes in a future period: 0.104 kW.
• Daytime MAE on unknown homes in the same period: 0.185 kW.
• Fine-tuning on five homes in the target region reduces the transfer error of the original model by 28–36%.
• Transfer used normalization by the target domain.

Transfer to Another Region and Window Length

When transferring from Ausgrid to Pecan Street, a model trained in one region loses accuracy: climate, typical installations, and consumption habits differ. The authors show that a small adjustment is enough to recover a substantial part of the loss. Normalizing by the target domain's statistics plus fine-tuning on five homes in the region reduces the error by 28–36%. For an operator, this is a concrete number: there is no need to collect labels for thousands of homes; a small sample with known generation is enough.

A separate question concerns the length of the input window. A long context helps with extrapolation for known customers, when the model has already seen the behavior of specific homes. For new customers, the result depends on the architecture: for some models a long window brings a gain, for others it does not. The paper does not confirm a universal "the longer, the better" rule.

How to Use This in Practice

For grid companies and analytics providers, three practical steps follow. First, set a separate error budget for each regime: the figure for known customers in a future period cannot serve as a promise for new homes, and even less for a new region. Second, when launching in a new region, plan a small pilot with measured generation at roughly five homes, to fine-tune the model and normalize the data. Third, choose the context length to fit the task: forecasting the behavior of known subscribers and estimating newly connected ones call for different settings.

One limitation should be kept in mind: the work covers two datasets and benchmark evaluation, not production use. Still, the method of separate evaluation regimes carries over to other energy analytics tasks, where a model is trained on some customers and applied to others.