ERA5(T) Rectangular Grid: Difference between revisions

From Wiki-WindPRO
Jump to navigation Jump to search
m (Morten moved page ERA5 CDS Grid Data to ERA5 (Geographic Grid): Renaming of dataset - release of ERA5T on CDS grid)
 
(100 intermediate revisions by 3 users not shown)
Line 1: Line 1:
[[Category:Online Data]][[Category:Wind Data]][[File:ERA5_WindsAt100m.png|thumb|right|300px|Winds @ 100m]]
[[Category:Online Data]][[Category:Wind Data]][[File:GunfleetSands_ERAT.png|thumb|right|600px|ERA5 - most recent data available in windPRO: ERA5 (red, until 2019-10-30) and ERA5(T) (green, until 2019-12-31). Site is Gunfleet Sands Offshore Wind Farm, UK.  Data accessed 2020-01-11.]]
== Introduction ==  
== Introduction ==  
The initial [[ERA5 Data|ERA5 windPRO dataset]] released in 2017 is based on the raw ERA5 data on a reduced gaussian grid. Unfortuantely the newly released ERA5T (updated with a shorter delay) data is not available on the reduced gaussian grid, but only on the 0.25x0.25 grid available in the Climate Data Store. In order to have a dataset in windPRO with ERA5T data this ERA5 CDS Grid dataset is made available.
[[Image:ERA5_RectangularVsGaussianGrid.png|right|600px|thumb]]
"ERA5(T) Rectangular Grid" is the most up-to date ERA5 data available in windPRO, provided by a seamless merge of the ERA5 data and the preliminary ERA5(T) data. The dataset hold a focused climate-data parameter subset of the ERA5 data dating back more than 20 years. Data available in windPRO are updated twice a month. For a general introduction to the ERA5 data in windPRO along with validation reports, please refer to our [[ERA5_Gaussian_Grid|ERA5 Gaussian Grid]] wiki-page.<br>
'''Why an additional "ERA5(T) Rectangular Grid" dataset in windPRO?'''<br>
The first ERA5-based dataset in windPRO was released in summer 2017 and now named [[ERA5_Gaussian_Grid|ERA5 Gaussian Grid]]. The original ERA5 data is based on model output from a reduced gaussian grid, which has been a recommended first choise for many modellers and analysts. Unfortuantely, the newly released ERA5(T) data from December 2019 (preliminary data, but updated with a shorter delay) is currently unavailable for the reduced gaussian grid. As such, ERA5(T) data is currently only offered by ECMWF on an interpolated, rectangular geographical grid in a 0.25x0.25 degree resolution. From our understanding, it is due to current technical limitations in the data-distribution systems at ECMWF and CDS. The windPRO "ERA5(T) Rectangular Grid" dataset is created in order to provide the most recent and up-to date data in windPRO - with both ERA5 data and ERA5(T) included.


== Resolution and Coverage ==
'''From Preliminary ERA5T to Final ERA5 Data'''<br>
'''Resolution:'''<br> The dataset grid is an interpolated regular lat/lon grid at 0.25x0.25 resolution (around 27.8 km). It holds hourly values. Coverage is global for land-areas and coastal regions. In windPRO, offshore coverage is expected to cover an area of approximately 300 km from the coastline.<br>
ERA5(T) is initial release data with availability 5 days behind real time. ERA5 is final data with availability 2-3 months behind real time. In the "ERA5(T) Rectangular Grid" dataset of windPRO, data from ERA5(T) and ERA5 are seamlessly merged, with ERA5(T) data being the most recent 3 months. Users can easily verify which data is ERA5 or ERA5(T) as the windPRO-data is flagged by the 'era5t' binary parameter - see the table below. In the rather unlikely event that ECMWF detects serious errors or flaws in the ERA5(T) data, the final ERA5 data could be different to the preliminary ERA5(T) data. If corrections are applied then users will be notified by ECMWF (via their mailing list and forum) and by EMD (by wiki and direct email). ECMWF<ref>ECMWF/Copernicus homepage (accessed 2020-01-13)  https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5-single-levels?tab=overview</ref> writes the following on the likelihood of errors detected in the ERA5(T) dataset: "''Based on experience with the production of ERA5 so far (and ERA-Interim in the past), our expectation is that such an event would not occur more than once every few years, if at all.''"


== Update plan ==
'''Performance: "ERA5 Gaussian Grid" vs "ERA5(T) Rectangular Grid'''<br>
The dataset holds data back to 1999 and will be updated with new data twice a month with a one week delay. The first update will be around the 8th of a month where all data from the previous month will be available. The second update will be around the 23rd of a month where the first half of the month will be available.
EMD has benchmarked the two datasets: the native "ERA5 Gaussian Grid" and the interpolated "ERA5(T) Rectangular Grid" to investigate any performance differences in the two quite similar datasets. The benchmark is made against high-quality wind-speed data from more than 300 tall meteorological masts with a global distribution. As the evaluation-metrics, we consider the wind-speed corelation (R2) at different averaging times: hourly, daily and monthly. Our benchmark shows, that the two datasets correlate equally well against the local meteorological mast data, with a slight advantage to the interpolated "ERA5(T) Rectangular Grid" dataset - see the table below. Since small differences are seen, we do not recommend to mix ERA5 data originating from different grids as inconsistencies then might occur.
<br>
 
Once the final ERA5 data becomes available (after 2-3 months) the dataset will be updated with that data. According to ECMWF they do not expect changes to happen very often and in the event that it does happen it will be communicated to users. At EMD we will communicate any such changes to the windPRO users as well.
== Dataset Overview ==
[[File:ERA5_WindsAt100m.png|thumb|right|300px|ERA5 - Wind Speeds @ 100m]]
* '''Content:''': Renewable energy focused data subset of ERA5 and ERA5(T). ERA5(T) data is flagged - see the paramateter list below. 
* '''Spatial Resolution:''' The dataset grid derived from an interpolated, regular geographical (lat/lon) grid at 0.25x0.25 resolution (around 27.8 km).
* '''Temporal Resolution:''' Hourly values.
* '''Coverage:''' Coverage is global for land-areas and coastal regions. In windPRO, offshore coverage is at a distance of approximately 300 km from any known coastlines.
* '''Period:''' 30+ years. The EMD databases will hold data back to 1988.
* '''Update schedule:'''
**Currently, EMD are updating this dataset once a month - with data being available with a one week delay from real-time. The monthly update will take place around the 7th or 8th of the month - where all data from the previous month will be available.  
**Once the final ERA5 data becomes available (after 2-3 months) the dataset will be updated with those data. According to ECMWF they do not expect changes to happen very often - and in the event that it does happen - it will be communicated to users. At EMD we will communicate any such changes to the windPRO users as well.
 
== Known Issues and Usage Notes ==
*'''2024-10-25:'''
** '''Issue:''' Final, validated ERA5 data from Jul-Oct 2024 differs from the preliminary ERA5T.
** '''Solution:''' <br>1. Read the notification from ECMWF [https://help.emd.dk/mediawiki/images/f/f4/20241024_ERA5T_AlpsSnowIssue.pdf here].<br>2.An analysis with recommendation from EMD is available [https://help.emd.dk/mediawiki/images/3/32/20241023_ERA5T_Issue_July-2024.pdf here]. This issue is very similar to the one from 2021-12-27 as documented below.<br>3. The EMD newsletter with recommendations is available [https://www.emd-international.com/newsletters/windpro/windPRO4.1_ERA5_Update.html here].<br>4 . A presentation from a wind-energy expert meeting on 2024-11-20 (vindkraftnet) is available [https://help.emd.dk/mediawiki/images/f/ff/20241120_ERA5-ERA5T_SnowDepthIssue_FINAL.pdf here].
*'''2021-12-27:'''
** '''Issue:''' Final, validated ERA5 data from Oct-Nov 2021 is now available (and it differs from preliminary ERA5T).
** '''Solution:'''<br> 1. Read an in-depth and updated [https://help.emd.dk/mediawiki/images/0/01/20211227_ERA5T_Issue_September-November-2021.pdf EMD-Analysis (pdf)] on impact on 100m global wind speeds (updated 2021-12-27). <br>2: If you have used ERA5T data from months Sep-Dec 2021 - then read and follow the recommendation wihin our original [https://www.emd-international.com/newsletters/windpro/windPRO3.5_ERA5_Notification.html EMD-Notification].
*'''2021-12-07:'''
** '''Issue:''' Final, validated ERA5 data from Sep-Nov 2021 differs from preliminary ERA5T.
** '''Solution:'''<br>1: Read and follow the recommendation wihin our [https://www.emd-international.com/newsletters/windpro/windPRO3.5_ERA5_Notification.html EMD-Notification]. <br> 2. Read a more in-depth [https://help.emd.dk/mediawiki/images/0/0d/20211206_ERA5T_Issue_September-November-2021.pdf EMD-Analysis (pdf)] on impact on 100m global wind speeds. <br>3: Read the ECMWF notification email, [https://help.emd.dk/mediawiki/images/5/53/20211201_ECMWF_ERA5T_Notification.pdf here] or follow the status on the ECMWF homepage, [https://confluence.ecmwf.int/display/CUSF/Final+validated+ERA5+product+to+differ+from+ERA5T+for+Sep-Dec+2021 here].<br>4: See an EMD-animation of differences of the 100m winds over Central Europe [https://help.emd.dk/knowledgebase/content/Animations/2021_September_ERA5_ERA5T_CentralEurope_FullOpti.gif gif-full-resolution] and [https://help.emd.dk/knowledgebase/content/Animations/2021_September_ERA5_ERA5T_CentralEurope_SmallOpti.gif gif-coarse-resolution].
*'''2021-12-05:'''
** '''Issue:''' How can I check if my data for a given month is preliminary ERA5T or final, validated ERA5?
** '''Solution:''' Time series downloaded in windPRO has a column called era5t which will be 1 for samples that are era5t data and it will be 0 for final data. It is also indicated in the header of the month time series files. Since the data is updated every month, it will be enough to check the ERA5T indication within header for a given month.
* '''For other issues:'''
** Please see the section on ''Known Issues'' on our [[ERA5_Gaussian_Grid|ERA5 Gaussian Grid]] wiki-page.


== Data Parameters ==
== Data Parameters ==
EMD extracts a focused subset of climate-parameters from the ERA5 datasets - this subset focuses on wind-energy modelling. When the data have been processed at EMD, then the following parameters are available from within the online-data services within windPRO.
EMD extracts a focused subset of climate-parameters from the ERA5 datasets - with special emphasis on renewable energy and wind-energy modelling. When the data have been processed at EMD, then the following parameters are available from within the online-data services within windPRO.


<center>
<center>
Line 23: Line 52:
| time
| time
|  
|  
| UTC time stamp
| UTC time stamp (YYYY-MM-DD HH:MM)
|-
|-
| wSpeed.10
| wSpeed.10
Line 90: Line 119:
|}
|}
</center>
</center>
== Required Modules/Licenses ==
To access the ERA5 reanalysis data from within windPRO - the following licenses/modules are required in your windPRO setup:
* BASIS
* METEO
When the license fee is paid, you then have access to the full dataset without further cost. Downloading of data is unrestricted for licenced users with an active service subscription, however, a "fair use" policy applies. Visit [http://www.emd.dk/windpro/online-ordering/ EMD online ordering] to purchase the necessary licences.
== Attribution ==
If data from this dataset is used within any private or public disseminations, then EMD and its data providers must be acknowledged.
<pre>
Source:
ERA5(T) Rectangular Grid. Distribution through EMD and windPRO - EMD International A/S, 2020.
This dataset uses ERA5 and ERA5(T) which is being developed through the Copernicus Climate Change Service (C3S).
Data processing and distribution for ERA5 is carried out by ECMWF.
</pre>


== External Links ==
== External Links ==
* What is ERA5 - at ECMWF - [https://software.ecmwf.int/wiki/display/CKB/What+is+ERA5 HERE]
* ERA5 Data Documentation - at ECMWF - [https://confluence.ecmwf.int/display/CKB/ERA5%3A+data+documentation HERE]
* ERA5 Page - at the Copernicus Climate Change Service - [https://climate.copernicus.eu/products/climate-reanalysis HERE]
* Release note for ERA5T data - from ECMWF - [https://confluence.ecmwf.int/display/CUSF/Release+of+ERA5T HERE]


* ECMWF - What is ERA5 - [https://software.ecmwf.int/wiki/display/CKB/What+is+ERA5 https://software.ecmwf.int/wiki/display/CKB/What+is+ERA5]
== References ==
* ERA5 Data Documentation at ECMWF - [https://software.ecmwf.int/wiki/display/CKB/ERA5+data+documentation https://software.ecmwf.int/wiki/display/CKB/ERA5+data+documentation]
<references>
* Copernicus Climate Change Service - ERA5 Page - [https://climate.copernicus.eu/products/climate-reanalysis https://climate.copernicus.eu/products/climate-reanalysis]
 
== Acknowledgements and Credits==
* ERA5 is being developed through the Copernicus Climate Change Service (C3S).
* Data processing and distribution for ERA5 is carried out by ECMWF.

Latest revision as of 14:57, 21 November 2024

ERA5 - most recent data available in windPRO: ERA5 (red, until 2019-10-30) and ERA5(T) (green, until 2019-12-31). Site is Gunfleet Sands Offshore Wind Farm, UK. Data accessed 2020-01-11.

Introduction

"ERA5(T) Rectangular Grid" is the most up-to date ERA5 data available in windPRO, provided by a seamless merge of the ERA5 data and the preliminary ERA5(T) data. The dataset hold a focused climate-data parameter subset of the ERA5 data dating back more than 20 years. Data available in windPRO are updated twice a month. For a general introduction to the ERA5 data in windPRO along with validation reports, please refer to our ERA5 Gaussian Grid wiki-page.

Why an additional "ERA5(T) Rectangular Grid" dataset in windPRO?
The first ERA5-based dataset in windPRO was released in summer 2017 and now named ERA5 Gaussian Grid. The original ERA5 data is based on model output from a reduced gaussian grid, which has been a recommended first choise for many modellers and analysts. Unfortuantely, the newly released ERA5(T) data from December 2019 (preliminary data, but updated with a shorter delay) is currently unavailable for the reduced gaussian grid. As such, ERA5(T) data is currently only offered by ECMWF on an interpolated, rectangular geographical grid in a 0.25x0.25 degree resolution. From our understanding, it is due to current technical limitations in the data-distribution systems at ECMWF and CDS. The windPRO "ERA5(T) Rectangular Grid" dataset is created in order to provide the most recent and up-to date data in windPRO - with both ERA5 data and ERA5(T) included.

From Preliminary ERA5T to Final ERA5 Data
ERA5(T) is initial release data with availability 5 days behind real time. ERA5 is final data with availability 2-3 months behind real time. In the "ERA5(T) Rectangular Grid" dataset of windPRO, data from ERA5(T) and ERA5 are seamlessly merged, with ERA5(T) data being the most recent 3 months. Users can easily verify which data is ERA5 or ERA5(T) as the windPRO-data is flagged by the 'era5t' binary parameter - see the table below. In the rather unlikely event that ECMWF detects serious errors or flaws in the ERA5(T) data, the final ERA5 data could be different to the preliminary ERA5(T) data. If corrections are applied then users will be notified by ECMWF (via their mailing list and forum) and by EMD (by wiki and direct email). ECMWF[1] writes the following on the likelihood of errors detected in the ERA5(T) dataset: "Based on experience with the production of ERA5 so far (and ERA-Interim in the past), our expectation is that such an event would not occur more than once every few years, if at all."

Performance: "ERA5 Gaussian Grid" vs "ERA5(T) Rectangular Grid
EMD has benchmarked the two datasets: the native "ERA5 Gaussian Grid" and the interpolated "ERA5(T) Rectangular Grid" to investigate any performance differences in the two quite similar datasets. The benchmark is made against high-quality wind-speed data from more than 300 tall meteorological masts with a global distribution. As the evaluation-metrics, we consider the wind-speed corelation (R2) at different averaging times: hourly, daily and monthly. Our benchmark shows, that the two datasets correlate equally well against the local meteorological mast data, with a slight advantage to the interpolated "ERA5(T) Rectangular Grid" dataset - see the table below. Since small differences are seen, we do not recommend to mix ERA5 data originating from different grids as inconsistencies then might occur.

Dataset Overview

ERA5 - Wind Speeds @ 100m
  • Content:: Renewable energy focused data subset of ERA5 and ERA5(T). ERA5(T) data is flagged - see the paramateter list below.
  • Spatial Resolution: The dataset grid derived from an interpolated, regular geographical (lat/lon) grid at 0.25x0.25 resolution (around 27.8 km).
  • Temporal Resolution: Hourly values.
  • Coverage: Coverage is global for land-areas and coastal regions. In windPRO, offshore coverage is at a distance of approximately 300 km from any known coastlines.
  • Period: 30+ years. The EMD databases will hold data back to 1988.
  • Update schedule:
    • Currently, EMD are updating this dataset once a month - with data being available with a one week delay from real-time. The monthly update will take place around the 7th or 8th of the month - where all data from the previous month will be available.
    • Once the final ERA5 data becomes available (after 2-3 months) the dataset will be updated with those data. According to ECMWF they do not expect changes to happen very often - and in the event that it does happen - it will be communicated to users. At EMD we will communicate any such changes to the windPRO users as well.

Known Issues and Usage Notes

  • 2024-10-25:
    • Issue: Final, validated ERA5 data from Jul-Oct 2024 differs from the preliminary ERA5T.
    • Solution:
      1. Read the notification from ECMWF here.
      2.An analysis with recommendation from EMD is available here. This issue is very similar to the one from 2021-12-27 as documented below.
      3. The EMD newsletter with recommendations is available here.
      4 . A presentation from a wind-energy expert meeting on 2024-11-20 (vindkraftnet) is available here.
  • 2021-12-27:
    • Issue: Final, validated ERA5 data from Oct-Nov 2021 is now available (and it differs from preliminary ERA5T).
    • Solution:
      1. Read an in-depth and updated EMD-Analysis (pdf) on impact on 100m global wind speeds (updated 2021-12-27).
      2: If you have used ERA5T data from months Sep-Dec 2021 - then read and follow the recommendation wihin our original EMD-Notification.
  • 2021-12-07:
    • Issue: Final, validated ERA5 data from Sep-Nov 2021 differs from preliminary ERA5T.
    • Solution:
      1: Read and follow the recommendation wihin our EMD-Notification.
      2. Read a more in-depth EMD-Analysis (pdf) on impact on 100m global wind speeds.
      3: Read the ECMWF notification email, here or follow the status on the ECMWF homepage, here.
      4: See an EMD-animation of differences of the 100m winds over Central Europe gif-full-resolution and gif-coarse-resolution.
  • 2021-12-05:
    • Issue: How can I check if my data for a given month is preliminary ERA5T or final, validated ERA5?
    • Solution: Time series downloaded in windPRO has a column called era5t which will be 1 for samples that are era5t data and it will be 0 for final data. It is also indicated in the header of the month time series files. Since the data is updated every month, it will be enough to check the ERA5T indication within header for a given month.
  • For other issues:

Data Parameters

EMD extracts a focused subset of climate-parameters from the ERA5 datasets - with special emphasis on renewable energy and wind-energy modelling. When the data have been processed at EMD, then the following parameters are available from within the online-data services within windPRO.

Table: Overview of the windPRO subset of ERA5 parameters.
Parameter Unit Description
time UTC time stamp (YYYY-MM-DD HH:MM)
wSpeed.10 m/s Wind speed @ 10m
wDir.10 deg Wind direction @ 10m
wSpeed.100 m/s Wind speed @ 100m
wDir.100 deg Wind direction @ 100m
temperature.2 deg.celcius Temperature @ 2m
dewPointTemp.2 deg.celcius Dew point temperature @ 2m
rh.2 % Relative humidity @ 2m
psfc pa Surface pressure
msl pa Mean sea-level pressure
blh m Boundary layer height
tcc % Total cloud cover
heatFlux w/m2 Instantaneous surface sensible heat flux
zust m/s Friction velocity
cbh m Cloud base height
ssrdAcc w/m2 Surface solar radiation downwards
era5t 0/1 Binary variable indicating whether the sample is ERA5T data

Required Modules/Licenses

To access the ERA5 reanalysis data from within windPRO - the following licenses/modules are required in your windPRO setup:

  • BASIS
  • METEO

When the license fee is paid, you then have access to the full dataset without further cost. Downloading of data is unrestricted for licenced users with an active service subscription, however, a "fair use" policy applies. Visit EMD online ordering to purchase the necessary licences.

Attribution

If data from this dataset is used within any private or public disseminations, then EMD and its data providers must be acknowledged.

Source: 
ERA5(T) Rectangular Grid. Distribution through EMD and windPRO - EMD International A/S, 2020.
This dataset uses ERA5 and ERA5(T) which is being developed through the Copernicus Climate Change Service (C3S). 
Data processing and distribution for ERA5 is carried out by ECMWF.

External Links

  • What is ERA5 - at ECMWF - HERE
  • ERA5 Data Documentation - at ECMWF - HERE
  • ERA5 Page - at the Copernicus Climate Change Service - HERE
  • Release note for ERA5T data - from ECMWF - HERE

References

<references>