A proposed first-experiment protocol informed by FPP3, ERA5 documentation and TimeSeriesSplit. It contains no newly measured forecasting results.
01 / A one-line task contract
Example task specification: at each hourly issue time, predict power one hour ahead for one site, using only inputs released by that issue time. This is an illustrative setup; a real dataset must establish the site, frequency, units and availability delay.
- Record site_id, issue_time, target_time, available_at, value, unit and quality_flag. Keep timezone explicit.
- Separate missing observations, sensor faults and operational curtailment. A power value near zero is not automatically an invalid record.
02 / Keep simple models in the comparison
Persistence predicts the last available power value. A seasonal baseline uses the same time on the previous day. Both may fail during rapid weather changes, but they make the added value of a complex model measurable. The seasonal interval must match the actual sampling frequency.
If a required historical input is missing, mark the baseline as unavailable or apply a predeclared fallback. Compare models on a common valid subset and also report their availability over all requested predictions.
03 / Split by issue time, protect the target window
Use chronological training, validation and final test periods. Fit imputers, scaling and feature selection on the training period only. For a forecast at t with labels through t+h, ensure that no training label reaches into the validation period. An embargo should follow the horizon and data-release rules.
TimeSeriesSplit provides ordered folds and a gap, but does not validate forecast availability, site grouping or delayed revisions for you. Its equal-spacing assumption matters when comparing test folds of equal duration.
04 / Show the error distribution, not just one mean
- Report MAE and RMSE with the power unit. If normalizing by installed capacity, state the capacity source and how changes over time are handled.
- Avoid unqualified MAPE near zero power. Inspect daylight/nighttime, forecast horizon and high-change intervals separately, with sample counts and an event threshold chosen before testing.
- If adding prediction intervals later, report empirical coverage together with interval width. A narrow band alone is not evidence of reliability.
05 / Before adding sources, plan how they can fail
A useful comparison matrix is: power history only; history plus weather; history plus weather with delayed or missing weather inputs. Keep the chronological split and target definition fixed. Randomly masking cells is not a substitute for realistic outages of an entire source.
Save the split manifest, source versions, configuration, prediction table and failure cases before writing a conclusion. These are planned artifacts; this note does not claim that a fusion model has already improved performance.
Sources & further reading
Compiled from code and documents, with AI-assisted drafting and translation. Examples and proposed checks are learning material; only linked evidence establishes what was run.